Score-Only Distillation Enables Faster, Compact Dense Retrieval Models

Jul 14, 2026

Researchers propose a distillation method that trains a compact dense retriever using only teacher score vectors, without requiring access to teacher hidden states. Their 0.6B parameter student model recovers up to 50% of the base-to-teacher performance gap on a standard evaluation panel, while achieving 4.7× faster query encoding and 9.7× faster document encoding compared to sequential teacher fusion. However, external-transfer performance after distillation is mixed.

Why it matters: This approach offers a practical path to compress large retrieval models for more efficient online serving, though its effectiveness may vary across tasks.

Full story at: arXiv Information Retrieval ↗