KV-Cache Compression Rankings Reverse Under Query-Agnostic Protocol, Audit Finds

Jul 15, 2026

A new audit of six KV-cache compression methods finds that their performance rankings reverse when compression is performed before seeing the query (query-agnostic) compared to after (query-aware). In the more realistic query-agnostic setting, only KeyDiff consistently outperforms trivial baselines, while the widely used SnapKV method underperforms a simple 'keep start and recent window' baseline.

Why it matters: This study challenges the validity of current evaluation protocols for KV-cache compression, suggesting that some popular methods may be less effective in real-world LLM deployments than previously thought.

Full story at: arXiv Machine Learning ↗