EMO: Expert Mixture Models for Emergent Modular Pretraining

EMO：为涌现模块化预训练的专家混合模型

AllenAI introduced EMO, a mixture-of-experts model with 14B total parameters and 1B active parameters, trained on 1 trillion tokens and able to use only 12.5% of its experts for specific tasks while retaining near-full-model performance.

Why it matters: HKR-H/K/R all pass, but this is an AllenAI/Hugging Face research release rather than a frontier model launch. The 14B/1B and 12.5% expert-activation claims justify the low featured band.