Singular Bayesian Neural Networks Achieve Competitive Performance with Far Fewer Parameters
A new approach to Bayesian neural networks uses low-rank weight parameterization, reducing the number of parameters from O(mn) to O(r(m+n)). This method achieves up to 33 times fewer parameters than 5-member Deep Ensembles, while maintaining competitive predictive performance. It also shows substantial improvements in out-of-distribution (OOD) detection and often better calibration compared to mean-field and perturbation baselines.
Why it matters: This work could make Bayesian deep learning more practical by significantly lowering the computational cost of uncertainty estimation.
Full story at: arXiv Statistical ML ↗