Autonomous Agent Evaluation Engineer

Evaluate autonomous agents through rubrics, trace debugging, and stress testing with Outlier as a remote freelance contributor.

Outlier collaborates with leading AI organizations to improve large language models and proactive, multi-step agents. This opportunity is designed for professionals with two or more years in backend engineering, AI automation, or complex systems integration. Your feedback will address complex system behaviors across real-world workflows.

Key Responsibilities:

• Create evaluation rubrics for agent behavior
• Debug traces from autonomous agent interactions
• Stress test agents in live environments
• Assess multi-turn system interactions carefully
• Identify subtle failures in agent behavior

Requirements:

• Two or more years in backend or AI work
• Production-grade software development experience
• Experience with modular service separation
• Ability to provide high-density technical feedback
• Recognize privacy leaks and prompt injections

Bring precise evaluation, debugging, and systems expertise to help Outlier improve advanced generative agent systems.