EMO: Pretraining Mixture of Experts for Emergent Modularity

Jul 11, 2026

The Allen Institute for AI has introduced EMO, a mixture-of-experts model in which modular expert groups emerge from data during pretraining. This design allows users to select small, task-specific expert subsets while maintaining performance close to that of the full model.

Why it matters: EMO could reduce computational costs and improve accessibility by enabling efficient, task-specific model usage without retraining.