Forgetful Attention: A Trainable Support-Vector Memory with Certified Selection and Exact Unlearning

Jul 15, 2026

Researchers present Support Vector Attention (SV-Attention), a novel max-margin memory mechanism for attention models that enables certified token eviction and exact unlearning. SV-Attention achieves higher rare-item recall (0.86 vs. 0.32) and better retention (0.80 vs. 0.05 deterioration hours) on MIMIC-IV data streams compared to baselines, and demonstrates improved compression (2.178 BPC vs. 2.383) on enwik8 relative to a sliding-window Transformer. The method also supports surgical forgetting, exact editing, and patient-record deletion in real-world scenarios.

Why it matters: This work introduces a principled and practical approach to certified unlearning in attention mechanisms, addressing a key challenge in machine learning privacy and data management.

Full story at: arXiv Machine Learning ↗