AI Agent Evaluation Engineer Outlier

Evaluate AI agents for Outlier remotely, using backend engineering and system integration expertise to improve autonomous model behavior.

This freelance opportunity supports projects training large language models as proactive, multi-step agents. Contributors write evaluation rubrics, debug agent traces, and stress test systems for complex architectural workflows. The work suits engineers with two or more years in backend engineering, AI automation, or complex systems integration, including experience with production-grade software.

Key Responsibilities:

• Write rubrics evaluating agent performance and behavior
• Debug traces from autonomous agent workflows
• Stress test agents against complex scenarios
• Assess multi-step coordination and system interactions

Requirements:

• Two-plus years in backend engineering or AI automation
• Experience with modular production software architectures
• Practical experience using live, non-mocked environments
• Ability to deliver clear technical feedback
• Attention to subtle system failures

Help Outlier improve autonomous agents by combining AI evaluation, backend engineering, and complex systems integration experience.