Cache-Aware Prompt Compression (CAPC) Significantly Reduces LLM API Costs
A new method called Cache-Aware Prompt Compression (CAPC) is introduced, combining query-agnostic compression with explicit cache control to optimize prompt caching for large language model (LLM) APIs. On Anthropic's Sonnet 4.6 API, CAPC achieves mean cost savings of 49% over cache-only and 64% over query-aware compression, with output quality within 0.05 of uncompressed baselines. CAPC is validated on three production workloads and is the most cost-effective strategy across all 16 LongBench-v2 configurations.
Why it matters: This work challenges the prevailing assumption that query-aware compression is optimal, demonstrating that it can be less cost-effective than caching alone, and introduces a practical approach that yields substantial real-world savings.
Full story at: arXiv AI/ML ↗