TAKE: Trajectory-Aware Knowledge Estimation for Text Dataset Distillation

Jul 15, 2026

Researchers introduce TAKE, a framework for text dataset distillation that reduces large text corpora to as little as 0.1% of their original size while maintaining performance on downstream tasks. TAKE leverages influence functions and trajectory-aware knowledge estimation to identify and select the most informative samples using an optimal transport objective. Evaluations on text classification and natural language inference tasks demonstrate that TAKE achieves extreme compression without significant loss in accuracy.

Why it matters: This work offers a principled approach to drastically reducing the size of text datasets, potentially lowering storage and computational costs for NLP training and fine-tuning.

Full story at: arXiv Computation and Language ↗