Accuracy Without Grounding: Diagnosing Visual Dependency Dissociation in Video LLM Benchmarks

Jul 16, 2026

A new preprint introduces the Visual Dependency Gap (VDG) metric to assess whether video LLM benchmarks truly measure visual understanding. By evaluating 20 models across various architectures, the study finds that benchmark accuracy can be dissociated from genuine visual dependency, with temporal order contributing little to performance. The authors propose VDG as a standard audit for visually grounded capability in video LLMs.

Why it matters: This work challenges the assumption that high benchmark scores in video LLMs reflect real visual understanding, highlighting the need for more rigorous evaluation methods.

Full story at: arXiv Computer Vision ↗