VanillaBench Reveals Hidden Accuracy Cost of Adversarial Robustness

Jul 15, 2026

A new benchmark, VanillaBench, systematically quantifies the accuracy cost of adversarial robustness by comparing 186 robust models against vanilla baselines. The study finds that the mean clean accuracy gap ranges from -7.7 to -29.5 percentage points, with even the best robust models trailing their vanilla counterparts by 4.0-21.0 points. The authors recommend that future robustness evaluations should routinely report vanilla-referenced accuracy gaps.

Why it matters: This benchmark makes explicit the substantial accuracy trade-off of adversarial robustness, providing critical information for practitioners and decision-makers that is often missing from individual papers.

Full story at: arXiv Cryptography and Security ↗