AI Benchmarking Architect

Callosum is building a unified benchmarking system for evaluating agentic and algorithmic AI work. The role focuses on reproducible, contamination‑controlled evaluation across motifs and decomposition strategies, grounded in real execution traces and sandboxed grading.

You will architect and implement pipelines that measure task success, quality and robustness, influencing which approaches ship and how we communicate proof points to customers.