Saturation Makes Quantization Error Additive: A Coverage Model with a Certificate

Jul 15, 2026

A new preprint investigates the premise that quantization loss in large language models can be effectively predicted from per-layer sensitivities. The authors show that per-layer effects account for 85–93% of loss variance at 4-bit precision, and introduce a coverage model and additive predictor that achieve low KL divergence on models ranging from 30B to 355B parameters. Notably, their approach maintains performance on code and reasoning tasks below four bits, outperforming gradient-based methods in these regimes.

Why it matters: This work offers a theoretically grounded and empirically validated approach for mixed-precision quantization, potentially enabling more efficient deployment of large language models with minimal accuracy loss.

Full story at: arXiv Machine Learning ↗