OAT: Lightweight Unsupervised Failure Attribution for LLM Agentic Systems

Jul 15, 2026

Researchers introduce OAT, a method for identifying error steps in large language model (LLM)-based agentic systems by training solely on successful trajectories. OAT uses neural controlled differential equations to model the dynamics of success and assigns anomaly scores to steps in failed trajectories, enabling unsupervised failure attribution. Experiments show OAT is 200–5000× faster than prompting-based baselines and achieves higher F1 scores: +20% in-domain and +7% out-of-distribution.

Why it matters: This approach offers a scalable and efficient way to debug agentic LLM systems without requiring costly failure annotations, potentially improving their reliability.

Full story at: arXiv AI/ML ↗