Quantized LLMs Exhibit Silent Reasoning Failures Undetected by Accuracy Metrics

Jul 14, 2026

A new preprint study demonstrates that post-training quantization can silently alter the reasoning processes of large language models (LLMs), even when their task accuracy remains stable. The researchers introduce a taxonomy of six reasoning failure modes and find that 'Hollow Convergence'—cases where correct answers are reached through incomplete or unverifiable reasoning—shifts significantly under low-precision quantization, particularly in smaller models. These shifts are not captured by standard accuracy benchmarks, and Hollow Convergence cannot be reliably detected from surface-level text features.

Why it matters: This work highlights a critical blind spot in LLM evaluation, showing that accuracy metrics alone may miss important reasoning failures in quantized models, which has implications for their safe deployment.

Full story at: arXiv Computation and Language ↗