Google open-sources DiffusionGemma: a diffusion-based Gemma 4 hitting 1,456 tok/s decode, with a clear reasoning trade-off

Google 发布 DiffusionGemma：当语言模型不再只能从左到右写

Google converted the fully post-trained Gemma 4 26B-A4B weights into a discrete polynomial diffusion model and open-sourced the weights on Hugging Face. On a single H100 at FP8 with batch size 1, decode hits 1,456 tok/s—over 7× the original AR model—by processing 256 tokens per forward pass and cutting memory-bandwidth overhead at low concurrency. The trade-off: AIME 2026 drops from 88.3 to 69.1, and MRCR 128K from 44.1 to 32.0. An AR fallback mode recovers AIME to 84.2, showing the base knowledge survived but the diffusion generation mode itself caused part of the quality loss. Additional training used under 10% of the original token budget, but absolute token count, FLOPs, and GPU hours are not disclosed. In real serving, TTFT rises from 53 ms to 489 ms, and at high concurrency AR total throughput overtakes diffusion.

Why it matters: Google open-sourced a diffusion-converted Gemma 4 that hits 1456 tok/s on a single H100 — 7x the original — but AIME math drops from 88.3 to 69.1. The speed-vs-capability tradeoff is backed by concrete numbers, directly useful for inference engineers. Not 85+ because the capab...