LaCache: Exact Caching and Precision-Adaptive Inference for Diffusion Large Language Models
LaCache is a training-free acceleration framework for diffusion-based large language models (DLLMs) that reduces redundant computation by caching intermediate results, including embeddings, pre-attention states, and softmax statistics. It also introduces per-group FP8 quantization for feed-forward network layers to further improve efficiency. Experiments show LaCache achieves about 1.3x speedup on its own and up to 40.2x when combined with other acceleration methods, without compromising task accuracy.
Why it matters: Improving inference efficiency in diffusion LLMs can make these models more practical for real-world deployment by reducing computational costs without sacrificing output quality.
Full story at: arXiv AI/ML ↗