ParallelKernelBench: Frontier LLMs can't write fast multi-GPU kernels (yet)

Jul 11, 2026

Together AI released ParallelKernelBench, a benchmark that tests LLMs on writing fast multi-GPU CUDA kernels across 87 real workloads. The best-performing model solves under a third of the tasks, though some generated kernels outperform any public implementation.

Why it matters: This benchmark highlights both the current limitations and emerging potential of LLMs in high-performance computing code generation.

Full story at: Together AI Blog ↗