How the Hessian Spectrum of Neural Networks Depends on Data

Jul 16, 2026

Researchers have derived the eigenvalues of the Hessian matrix for linear neural networks with arbitrary width, depth, and dataset size. For classification tasks using mean squared error (MSE) loss, they show that the sharpness of the solution is directly linked to the maximum proportion of samples in any class. Their theoretical predictions remain robust even as simplifying assumptions are relaxed and some nonlinearities are introduced.

Why it matters: This work provides a theoretical connection between data properties and the loss landscape in neural networks, which could inform future research on optimization and generalization in deep learning.

Full story at: arXiv Machine Learning ↗