Critic Experience Bank: Self-Evolving Step-Level Confidence Estimation for LLM Agents

Jul 15, 2026

Researchers introduce Critic Experience Bank (CEB), a training-free framework that enables large language model (LLM) agents to estimate step-level confidence by leveraging accumulated evidence from past actions and their outcomes. CEB employs a hindsight LLM to pseudo-label each step as productive or unproductive after observing the full execution, storing these experiences in a memory bank for retrieval during similar future steps. The method demonstrates up to a 54% reduction in expected calibration error (ECE) compared to the strongest training-free baseline across three agent benchmarks and three critic backbones.

Why it matters: This approach advances reliable LLM agent deployment by enabling calibrated, pre-execution confidence estimates for individual actions, which can help mitigate costly errors in interactive environments.

Full story at: arXiv AI/ML ↗