Optimal Self-Distillation for Rectified Flow via Linear Probing

Jul 17, 2026

A new preprint presents a theoretical and practical framework for optimal self-distillation in rectified flow generative models. The authors derive a closed-form solution for mixing teacher and true velocities, providing a provable improvement in velocity risk under certain conditions. Their method eliminates the need for grid search by introducing a one-shot tuning procedure, and experiments demonstrate improved velocity risk and generation quality over both the teacher and standard distillation approaches.

Why it matters: This work offers a principled and efficient approach to self-distillation in generative models, with theoretical guarantees and demonstrated empirical benefits.

Full story at: arXiv Statistical ML ↗