Autonomous Agent Evaluation Engineer

Evaluate autonomous agents remotely using production-grade software skills and precise technical feedback. Outlier offers freelance work with flexible scheduling.

This opportunity supports projects training large language models to function as proactive, multi-step agents. Contributors assess complex system behaviors through rubrics, trace debugging, and stress testing. Experience with backend engineering, AI automation, or systems integration supports the work.

Key Responsibilities:

• Develop rubrics for evaluating agent outputs
• Inspect and debug traces from agent interactions
• Test agents against complex workflow scenarios
• Identify subtle failures in system behavior

Requirements:

• Two or more years of relevant experience
• Production-grade software development with modular separation
• Practical experience with live system interactions
• Clear, detailed feedback on complex technical behaviors
• Experience with multi-stage coordination tasks preferred

Use agent evaluation, systems integration, and technical analysis skills to improve proactive AI systems through Outlier projects.