LiteTopK: Fused Indexer-TopK Kernel for Efficient Sparse Attention

Jul 15, 2026

Researchers introduce LiteTopK, a fused GPU kernel designed to accelerate Indexer-TopK operations in long-context sparse attention by leveraging the curse of dimensionality. LiteTopK samples data to estimate score ranges and partitions candidates into bins, reducing memory traffic and overhead while maintaining exact top-k correctness. Experiments show a 1.2x speedup in the prefill stage of GLM 5.2 during real-world deployment, along with lower memory usage.

Why it matters: This work offers a practical advance in the efficiency of sparse attention for large language models, enabling faster and more memory-efficient processing of long contexts.

Full story at: arXiv Machine Learning ↗