Autonomous Agent Evaluation Engineer

Evaluate autonomous agents with Outlier using technical feedback, trace debugging, and system stress testing. Work remotely on a freelance schedule.

Outlier collaborates with leading AI organizations to improve large language models and proactive, multi-step agents. This work suits engineers with two or more years in backend engineering, AI automation, or complex systems integration. You will assess how agents coordinate architectural workflows, interact with live tools, and respond across multiple turns.

Key Responsibilities:

• Assess complex agent behaviors through written evaluation rubrics
• Inspect and debug detailed agent traces
• Stress test autonomous systems under varied conditions
• Identify privacy leaks and authority escalation
• Detect indirect prompt injection failures

Requirements:

• Two or more years of relevant engineering experience
• Experience building and maintaining production-grade software
• Practical live-environment and multi-turn interaction experience
• Ability to deliver clear, high-density technical feedback
• Experience with persistent state is useful

Use agent evaluation, debugging, and security analysis skills to help Outlier advance autonomous AI systems.