AI Agent Evaluation Engineer

Evaluate autonomous AI agents remotely using technical feedback, trace debugging, and stress testing for Outlier’s generative AI projects.

Outlier collaborates with AI organizations to improve large language models through human feedback. This freelance work focuses on teaching systems to design, coordinate, and optimize real-world architectural workflows. You will assess complex behaviors while working with live, multi-turn agent interactions.

Key Responsibilities:

• Develop rubrics for evaluating agent performance
• Inspect and debug detailed agent traces
• Stress test autonomous agents across workflows
• Identify privacy leaks and authority escalation risks
• Detect indirect prompt injection failures

Requirements:

• Two or more years of relevant professional experience
• Experience building production-grade modular software systems
• Practical live-environment development beyond mocked systems
• Strong attention to detail and technical communication
• Agent integrations with Supabase, Gmail, or APIs welcomed

Use agent evaluation, systems integration, and failure analysis expertise to improve OpenClaw-like autonomous systems through Outlier.