Selective State-Space Adaptation and Retrieval for Language Model Reasoning
A new preprint introduces MaLoRA and MaRA, two adapter families that add selective state-space recurrence at token and context levels to improve reasoning in frozen large language models (LLMs). Evaluated on three LLM backbones and two multi-hop reasoning benchmarks, these methods achieve up to +9.3 F1 improvement over standard LoRA adapters, with consistent gains across all tested settings. The results suggest that dynamic, stateful adaptation can outperform static low-rank updates for complex reasoning tasks.
Why it matters: This work demonstrates a practical and effective way to boost LLM reasoning accuracy without modifying the base model, potentially enabling more efficient and adaptable AI systems.
Full story at: arXiv Computation and Language ↗