AI Agent Evaluation Engineer Outlier

Help Outlier improve autonomous agents using agent debugging, stress testing, and technical feedback through remote freelance work.

This contributor opportunity supports AI organizations training large language models to perform proactive, multi-step tasks. You will evaluate complex architectural workflows and identify subtle failures in live interactions. The work suits software developers and orchestration specialists with two or more years of relevant experience.

Key Responsibilities:

• Debug agent traces across multi-turn interactions
• Stress test autonomous agents and workflows
• Write clear evaluation rubrics for agent performance
• Identify privacy leaks and authority escalation
• Detect indirect prompt injection failures

Requirements:

• Two or more years in backend engineering or AI automation
• Experience integrating agents with live tools and APIs
• Build modular, production-grade software systems
• Handle persistent state and session discovery
• Provide precise technical feedback on complex behavior

Bring agent evaluation, backend engineering, and systems integration skills to Outlier’s autonomous-agent training projects.