Partial Fusion of Neural Networks Enables Flexible Tradeoffs Between Ensembles and Weight Aggregation
Researchers propose partial fusion of neural networks, a method that interpolates between full ensembles and weight aggregation to balance computational cost and model performance. The technique aggregates only the most similar neurons across networks, using partial optimal transport for matching, and generalizes to a form of pruning that allows for isolating, deleting, or combining neurons. This approach provides a flexible framework for managing the tradeoff between accuracy and efficiency in neural network deployment.
Why it matters: This method could make neural network ensembles more practical by reducing their computational demands while preserving much of their performance.
Full story at: arXiv Statistical ML ↗