Study Reveals Why LLMs Memorize but Fail to Generalize New Knowledge

Jul 10, 2026

A new paper formalizes the 'Knowing–Using Gap' in LLM fine-tuning, where models can memorize new facts but often fail to use them for reasoning tasks. Using a 'self-patching' intervention, researchers found that memorized knowledge may reside in circuit locations that are ineffective for computation. A simple heuristic was able to recover 58–75% of the generalization gap.

Why it matters: This mechanistic insight could lead to more effective fine-tuning methods that ensure new knowledge is actually usable by LLMs.

Full story at: arXiv AI/ML ↗