SGLang Integrates DSpark: Confidence-Driven, Variable-Length Speculative Decoding
SGLang 集成 DSpark 推测解码:置信度驱动的可变长度验证
LMSYS integrated DSpark speculative decoding into SGLang. The draft model's confidence head decides how many tokens to verify per request, instead of always verifying a full block. On H200 with DeepSeek-V4-Flash, it beats both MTP and non-spec baselines in throughput and per-user decode speed. The key engineering move is a ragged, variable-length verify under full CUDA graphs: when the scheduler trims the verify budget, the engine replays a genuinely smaller graph, not a padded one. A cost-table profiler lets the scheduler size each request's budget online, and a cap-accept mode exposes the acceptance ceiling hidden by trimming.
Why it matters: LMSYS plugs DSpark into SGLang, shortening verification for requests with low acceptance probability based on the draft model's confidence to save compute.