Study Finds GRPO Fails to Improve Small Web Agents, Reveals Learning-Rate-Dependent Degradation

Jul 15, 2026

A controlled study of Group Relative Policy Optimization (GRPO) on 4B-8B parameter language and vision-language web agents found that no configuration improved upon a strong supervised baseline. Moderate to high learning rates led to credible performance degradation, particularly on text-based tasks. The study attributes this failure to learning-rate-gated degradation and collapse regimes, with effects that are dependent on model scale.

Why it matters: This work challenges the assumption that reinforcement learning with verifiable rewards reliably improves small agents, identifying specific failure modes tied to learning rate and model scale.

Full story at: arXiv AI/ML ↗