Autonomous AI Agent Evaluation Engineer

Evaluate autonomous AI agents with Outlier using technical feedback, agent debugging, and complex systems integration. This freelance work is remote.

Outlier helps innovative companies improve AI agents by collecting human feedback. As an engineer, you will help train large language models to operate as proactive, multi-step systems. Projects involve architectural workflows, live tools, and system behaviors requiring careful analysis.

Key Responsibilities:

• Debug agent traces and investigate system behavior
• Write clear evaluation rubrics for agent outputs
• Stress test agents against complex interactions
• Identify privacy leaks and authority escalation failures
• Review indirect prompt injection risks

Requirements:

• At least two years of relevant professional experience
• Backend engineering, AI automation, or integration background
• Maintain production-grade modular software systems
• Handle persistent state and multi-turn interactions
• Provide precise technical feedback on complex behaviors

Use agent evaluation, debugging, and security analysis skills to help Outlier improve autonomous AI systems like OpenClaw.