AI HOT (Curated Pool)Aug 5, 2026, 01:51 UTC+8

SpecForge v0.3: LMSYS releases a disaggregated speculative decoding training stack and new open draft models

SpecForge v0.3.0 发布：统一解耦与共置投机解码栈，新增开放 SpecBundle 草稿模型

SpecForge v0.3 decouples target-model inference from draft-model training. Patched SGLang servers capture features, Mooncake transports tensors, and trainer workers consume them independently. On an 8×H20 testbed, 3 servers + 5 trainers deliver ~10% higher end-to-end training throughput than the previous colocated design. The runtime now supports six speculative decoding families—EAGLE3, DFlash, Domino, DSpark, and more—and ships community-contributed draft models trained entirely on open data.

Why it matters: LMSYS disaggregated speculative decoding training into three independent contracts — capture, delivery, lifecycle — and showed ~10% throughput gain on 8×H20 with 3 servers + 5 training nodes. The architecture is clean and the numbers are concrete, but the audience is narrow (i...

Read the original ↗Export Markdown

Take

The useful bit here is the engineering split: before v0.3, training a draft model meant colocating it with the frozen target model in the same process, which led to GPU idle time and resource contention. Now LMSYS breaks it into three independent pieces—patched SGLang servers capture features from the target model, Mooncake handles tensor transport, and trainer workers just consume features. On an 8×H20 testbed, 3 servers + 5 trainers got ~10% higher end-to-end training throughput than the old colocated setup.

They also added support for six drafting families—EAGLE3, DFlash, Domino, DSpark, and more—and released community-contributed draft models trained on Qwen3.6-27B and Kimi-K3. That means you can grab a pre-trained draft model and plug it into SGLang for speculative decoding without building the training pipeline yourself.

I'd hold off on extrapolating the 10% gain too far—it's from a small 8×H20 testbed, and we don't know how it scales to larger clusters. The serving performance numbers are also only shown at concurrency=1, so the real-world speedup under load is still an open question.

Source: AI HOT (Curated Pool) · lmsys.org
Published: Aug 5, 2026, 01:51 UTC+8 · Aug 5, 2026