Together AI unveils CPD architecture for up to 40% faster long-context LLM serving

Jul 11, 2026

Together AI introduced Cache-aware Prefill–Decode Disaggregation (CPD), a new inference architecture that separates warm and cold workloads. The approach delivers up to 40% higher throughput and significantly reduces time-to-first-token for long-context LLM serving.

Why it matters: This technique addresses a key bottleneck in serving long prompts, enabling faster responses for applications like document analysis and code generation.

Full story at: Together AI Blog ↗