AI Agent Evaluation Engineer

Evaluate AI agents remotely for Outlier using technical feedback, trace debugging, and stress testing across real-world interactions.

Outlier collaborates with AI organizations to train Large Language Models as proactive, multi-step agents. This freelance work centers on assessing how systems design, coordinate, and optimize complex workflows. Candidates need at least two years of relevant experience and strong attention to subtle system behaviors.

Key Responsibilities:

• Stress test autonomous agents across demanding scenarios
• Debug traces from multi-step system interactions
• Write clear, precise evaluation rubrics
• Identify subtle failures in agent behavior

Requirements:

• Two or more years in relevant technical fields
• Production-grade software development with modular separation
• Experience handling live environments without mocks
• Clear technical feedback on complex system behavior
• Applicants must be authorized in their residence country

Use evaluation, debugging, and technical analysis skills to improve autonomous AI agents through Outlier’s remote freelance projects.