Looped State-Space Language Models Show Benefits of Recurrent Computation

Jul 14, 2026

Researchers investigated looped Mamba and hybrid Mamba-Transformer architectures, which repeatedly apply a shared block for recurrent computation. On reasoning tasks, looped models consistently outperformed parameter-matched non-looped baselines and, in several settings, matched or exceeded non-looped models of equal effective depth. In language model pre-training, looped models remained competitive with substantially fewer distinct parameters, though deeper non-looped models retained an advantage in validation perplexity under strict iso-FLOPs comparisons.

Why it matters: This work extends looped architectures to state-space models, demonstrating that recurrent computation can improve reasoning and efficiency, potentially reducing parameter counts in large language models.

Full story at: arXiv AI/ML ↗