AI Agent Evaluation Engineer

Evaluate advanced AI agents remotely with Outlier, applying technical feedback, debugging, and complex systems integration expertise.

This freelance opportunity supports projects training large language models to function as proactive, multi-step agents. Contributors help improve autonomous systems by analyzing complex behaviors and real-world architectural workflows. The work includes writing evaluation rubrics, debugging agent traces, and stress testing agent performance.

Key Responsibilities:

• Analyze complex autonomous agent system behaviors
• Write detailed evaluation rubrics for agents
• Debug multi-turn agent traces and interactions
• Stress test agents using live environments
• Identify subtle failures in agent behavior

Requirements:

• Two or more years of relevant experience
• Strong attention to technical detail
• Experience building production-grade modular software
• Experience with live tools and real interactions
• Clear, high-density technical feedback skills

Use agent debugging, evaluation, and systems expertise to help Outlier improve proactive AI systems.