Autonomous Agent Evaluation Engineer

Evaluate autonomous agents with rubric writing and trace debugging skills. This freelance opportunity is remote, flexible, and available through Outlier.

Outlier helps innovative companies improve AI agents by providing human feedback. The work focuses on teaching large language models to design, coordinate, and optimize complex real-world workflows. Contributors assess agent behavior, test systems, and explain subtle failures with high-density technical feedback.

Key Responsibilities:

• Create evaluation rubrics for agent performance
• Investigate and debug detailed agent traces
• Stress test agents against complex workflows
• Assess behavior across live system interactions

Requirements:

• Two or more years of relevant experience
• Complex systems integration or AI automation background
• Ability to build maintainable production-grade software
• Experience handling persistent state and session discovery
• Strong attention to technical behavioral details

Apply evaluation, debugging, and systems expertise to help Outlier improve proactive multi-step AI agents like OpenClaw.