ReOPD: Efficient Multi-Turn On-Policy Distillation for LLM Agents

Jul 17, 2026

Researchers introduce ReOPD, a method for on-policy distillation of large language model (LLM) agents that reuses pre-collected teacher trajectories as replayed prefixes, reducing the need for costly environment interactions. The approach addresses the 'prefix trap,' where student-on-policy histories can reduce teacher reliability, by employing a step-decaying sampling schedule. Experiments show that ReOPD achieves comparable or improved accuracy to standard on-policy distillation, is at least 4x faster per rollout, and requires zero tool calls during student training.

Why it matters: ReOPD makes scalable distillation of LLM agents more practical by transforming expensive agent-environment interactions into a reusable offline resource, enabling more efficient training across diverse tasks and environments.

Full story at: arXiv Statistical ML ↗