Auditing Data Leakage in Whole-Slide Image Multimodal Benchmarks

Jul 15, 2026

A new preprint audits data leakage in whole-slide image (WSI) multimodal benchmarks and finds that patient-level and institutional-level overlaps significantly compromise reported zero-shot performance of vision-language models. The authors document case-level train-test overlaps of 92.3–100% on TCGA-derived benchmarks and show that this leakage is linearly decodable from foundation-model features. They recommend concrete steps for contamination-free evaluation practices.

Why it matters: This work reveals that widely-used WSI VQA benchmarks may not accurately measure genuine multimodal reasoning, calling into question reported advances in computational pathology.

Full story at: arXiv Computer Vision ↗