AI21 Labs Uncovers Silent Integer Overflow in vLLM CUDA Kernel Affecting GRPO Training

Jul 11, 2026

AI21 Labs discovered a 32-bit integer overflow in a vLLM CUDA kernel that led to logprob mismatches during GRPO training of their Jamba 3B model. The bug only manifested when the number of cache slots exceeded approximately 47,935 and was resolved with a minor code change. The issue persisted undetected for several weeks.

Why it matters: This case underscores how subtle, low-level bugs can silently compromise AI training, highlighting the importance of thorough infrastructure testing.

Full story at: AI21 Labs ↗