Together AI Introduces FlashAttention-4 with Pipelining and Hybrid Softmax

Jul 11, 2026

Together AI has announced FlashAttention-4, a new kernel design that addresses asymmetric hardware scaling by introducing pipelining for maximum overlap, 2-CTA MMA modes to reduce shared memory traffic, and a hardware-software hybrid approach to softmax exponentials. The technique aims to keep pace with GPU throughput outpacing memory bandwidth.

Why it matters: FlashAttention-4 could significantly improve the efficiency of attention mechanisms in large language models, enabling faster training and inference as hardware continues to evolve.

Full story at: Together AI Blog ↗