Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0

Jul 16, 2026

A new preprint evaluates whether agent-optimization methods yield compounding improvements when agents are continually optimized as new tasks appear, using the challenging Terminal-Bench 2.0 suite. The study finds that most methods, including GEPA and Meta Harness, show initial gains but fail to compound or even regress when faced with new tasks. Only RELAI-VCL, which incorporates regression control, achieves both positive transfer to new tasks and continued improvement, reaching a 76.4% lifelong average pass rate.

Why it matters: This work highlights the limitations of static benchmark evaluations and demonstrates the importance of continual-learning assessments for robust, real-world agent deployment.

Full story at: arXiv AI/ML ↗