Autonomous Agent Evaluation Engineer

Evaluate autonomous agents with Outlier through remote freelance work focused on rubrics, trace debugging, and stress testing. Set your schedule within project needs.

This AI Agent Engineer opportunity supports training large language models to function as proactive, multi-step agents. Projects involve designing, coordinating, and optimizing complex real-world architectural workflows. You will provide human feedback on agent behavior while working with live systems and multi-turn interactions. Outlier accepts applicants in Australia, Canada, Finland, France, Italy, Norway, the United Kingdom, and the United States.

Key Responsibilities:

• Develop evaluation rubrics for agent outputs
• Inspect and debug detailed agent traces
• Test agents against demanding real-world workflows
• Identify subtle failures in system behavior
• Deliver clear, high-density technical feedback

Requirements:

• Two or more years of relevant experience
• Experience building production-grade modular software
• Backend, AI automation, or complex integration background
• Practical live-environment and multi-turn interaction experience
• Strong attention to technical detail

Use agent evaluation, systems integration, and technical feedback skills to improve Outlier’s advanced generative AI systems.