Who&When Pro: Benchmark for Automated Failure Attribution in AI Agents

Jul 14, 2026

Researchers have introduced Who&When Pro, a large-scale benchmark designed for automated failure attribution in agentic systems. The benchmark comprises 12,326 failed trajectories with golden labels, spanning 3 modalities and 26 benchmarks. The study analyzes how large language models attribute failures, uncovering systematic patterns and offering empirical guidance for future systems.

Why it matters: As AI agents become more capable, automated failure attribution is increasingly important for debugging and reliability.

Full story at: arXiv AI/ML ↗