Fisher Rank Inflation: A Spectral Signature of Memorization under Label Noise

Jul 15, 2026

Researchers have identified 'Fisher Rank Inflation,' a spectral signature in neural network gradients that marks the transition from learning clean data to memorizing corrupted labels. The effective rank of per-example gradients expands during memorization and contracts afterward, with the peak rank increasing as label corruption becomes more severe. This phenomenon enables detection of corrupted examples and, in some cases, the onset of memorization can be detected before test performance degrades.

Why it matters: This work provides a principled and interpretable method to detect and quantify memorization of label noise in deep networks, which could improve data auditing and robust training practices.

Full story at: arXiv Statistical ML ↗