Knowledgeless Language Models: Suppressing Parametric Recall for Evidence-Grounded Language Modeling

Jul 15, 2026

Researchers introduce Knowledge-Less Language Models (KLLMs), which are pretrained on corpora with anonymized named entities to reduce the model's reliance on memorized factual knowledge. KLLMs demonstrate reduced closed-book factual recall but outperform standard models on contextual question answering, fact verification, and hallucination detection, especially in retrieval-grounded settings with imperfect evidence, achieving up to 20–25% relative gains. The models also show improved calibration and more reliable abstention behavior, indicating a shift toward evidence-grounded reasoning.

Why it matters: This work shows that controlling knowledge acquisition during pretraining can produce language models that are more robust and reliable in evidence-grounded tasks, potentially improving real-world AI applications.

Full story at: arXiv Computation and Language ↗