ExTernD: Ternary LLM Quantization That Approaches Full-Precision Accuracy

Jul 16, 2026

A new method called ExTernD enables post-training quantization of large language models (LLMs) by decomposing weight matrices into ternary factors with an expanded inner rank. This approach allows the quantized model's accuracy to approach that of full-precision (bf16) models arbitrarily closely, overcoming limitations of fixed bit-width quantization. ExTernD achieves Q4_K-level accuracy at 5.2–5.5 effective bits per weight on models like Gemma-4-E2B and Qwen3.5-4B, with a full Qwen3.5-4B conversion reaching 10.10 perplexity versus 9.78 for bf16 (+3.2%).

Why it matters: ExTernD provides a flexible, near-lossless quantization method for LLMs, enabling more efficient deployment without significant accuracy loss.

Full story at: arXiv Machine Learning ↗