Sumi: Open Uniform Diffusion Language Model from Scratch
Sumi:从头训练的7B开源均匀扩散语言模型
Sumi is the first uniform diffusion language model pretrained from scratch at 7B parameters on 1.5T tokens, with weights and full training recipe released openly. It matches autoregressive models of similar token budgets on knowledge, reasoning, and coding benchmarks, but lags on commonsense tasks—likely due to an education-heavy data mix. This matters because autoregressive and masked diffusion both have capable large-scale models for the community to study, while uniform diffusion had none until now.
Why it matters: A fully open 7B diffusion language model, with weights, checkpoints and the training recipe published.