CANON: Consensus-Anchored Self-Distillation Improves LLM Reasoning Without Labels

Jul 16, 2026

Researchers introduce CANON, a label-free training method that leverages consensus among multiple LLM-generated solutions to provide dense, token-level supervision. On mathematical and scientific reasoning benchmarks, CANON improves pass@1 by up to 12 points, surpasses label-free reinforcement learning by 6 points at a fraction of the compute cost, and approaches the performance of models trained with gold labels. The method also generalizes to held-out benchmarks, matching the effectiveness of gold-label training.

Why it matters: CANON demonstrates a compute-efficient approach to enhancing LLM reasoning without human-annotated labels, potentially reducing the need for costly data annotation.

Full story at: arXiv Machine Learning ↗