Sub-1B On-Device Distillation for Structured Text Enrichment: Reasoning Teachers Boost Summary Quality but May Reduce Factual Grounding

Jul 10, 2026

Researchers distilled an 8B reasoning teacher (deepseek-r1:8b) into a 0.6B student (Qwen3-0.6B) for structured text enrichment, achieving 0.8 seconds per article compared to the teacher's 39 seconds. The student recovered 58% of the summary quality gap and outperformed baselines, but a reasoning-lineage student showed reduced factual grounding on short, thin-source articles. The study provides a per-field routing map for on-device enrichment.

Why it matters: This work demonstrates that small on-device models can achieve practical performance for structured extraction tasks, but the choice of teacher model significantly impacts different capabilities, highlighting the need for task-specific distillation strategies.

Full story at: arXiv AI/ML ↗