MADA-RL: Multi-Agent Debate-Aware Reinforcement Learning Boosts Compact Model Reasoning
MADA-RL is a post-training framework that divides compact language models into generator and critic roles, fine-tuning only a small subset of parameters using LoRA adapters. By introducing a counterfactual critic advantage, it explicitly trains critics to improve over generator consensus, resulting in a 2.0 percentage point accuracy gain on the DeepSeek-R1-Distill-Qwen-1.5B model with 16 times fewer trainable parameters than full fine-tuning. The method achieves competitive performance on mathematical reasoning benchmarks and offers a parameter-efficient alternative to existing approaches.
Why it matters: This work presents a significant advance in parameter-efficient training for compact language models, enabling improved reasoning performance under limited computational resources.
Full story at: arXiv Multiagent Systems ↗