LLM Agent Evaluation Engineer

Evaluate LLM agents using technical feedback, trace debugging, and stress testing while working remotely as a freelance Outlier contributor.

Outlier supports AI organizations training Large Language Models as proactive, multi-step agents. This work focuses on teaching systems to coordinate complex, real-world architectural workflows through human feedback. The opportunity suits engineers with two or more years in backend engineering, AI automation, or complex systems integration.

Key Responsibilities:

• Create clear evaluation rubrics for agent performance
• Inspect and debug agent traces
• Stress-test agents against complex scenarios
• Identify subtle system behavior failures

Requirements:

• Two or more years of relevant professional experience
• Production-grade modular software development experience
• Live tool and multi-turn interaction experience
• Strong attention to detail
• Clear, high-density technical communication

Use LLM evaluation, agent debugging, and systems expertise to help improve autonomous software through Outlier.