Train separately, merge together: Modular post-training with mixture-of-experts

Jul 11, 2026

The Allen Institute for AI has introduced BAR, a modular post-training method that enables domain experts to be trained independently and then merged into a single mixture-of-experts model. This approach allows for upgrading individual experts without affecting the performance of others.

Why it matters: BAR offers a scalable and efficient way to enhance language models by decoupling the development of different capabilities, potentially reducing retraining costs and improving adaptability.

Full story at: Allen Institute for AI ↗