AI Agent Evaluation Software Engineer

Evaluate AI agents with software engineering and technical feedback skills. Outlier offers remote freelance work for contributors in listed countries.

This work supports leading AI organizations training large language models as proactive, multi-step agents. You will assess complex system behaviors and help improve real-world architectural workflows. The work suits developers with at least two years in backend engineering, AI automation, or complex systems integration, plus experience with live environments.

Key Responsibilities:

• Debug agent traces across multi-turn interactions
• Write detailed evaluation rubrics for agent performance
• Stress test autonomous agents against complex workflows
• Identify subtle failures in system behavior

Requirements:

• At least two years of relevant professional experience
• Maintain modular, production-grade software systems
• Work effectively in live, non-mocked environments
• Give high-density technical feedback on complex behaviors
• Experience with backend engineering or AI automation

Use software engineering judgment, agent debugging, and evaluation skills to strengthen autonomous AI systems through Outlier projects.