BIRD: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning
Researchers introduce BIRD, a two-stage self-reasoning distillation method designed to improve the efficiency of large reasoning models by reducing redundant chain-of-thought traces. Applied to the Qwen3-8B model, BIRD increases MATH-500 benchmark accuracy from 86.2% to 92.0% while cutting average response length from 3,099 to 1,115 tokens. The approach addresses initialization bottlenecks in on-policy distillation by first bootstrapping concise, correct solutions before applying reverse-KL distillation. This results in a better accuracy-efficiency trade-off compared to previous methods.
Why it matters: This work provides a practical advance for making large reasoning models more efficient without sacrificing accuracy, which is important for deploying AI in resource-constrained settings.
Full story at: arXiv Computation and Language ↗