Tracing LLM Behavior to the Training Data with Empirical Next-Token Distributions

Jul 17, 2026

This preprint investigates how closely large language models' (LLMs) next-token predictions align with the empirical next-token distributions (ENTD) derived from their training data. The authors find that for many input contexts, LLMs' output distributions closely match the ENTD, with this agreement improving as model size and training compute increase. However, they also identify a substantial set of cases where the model's predictions diverge from the ENTD, attributing these discrepancies to factors such as model architecture, training procedures, and noise in the ENTD estimation.

Why it matters: The work introduces and demonstrates 'data-centric mechanistic interpretability,' providing a new approach to understanding how LLM behaviors are shaped by their training data.

Full story at: arXiv AI/ML ↗