Skip to content
r/LocalLLaMA

Benchmarking vLLM vs SGLang vs llama.cpp on a mixed Blackwell/Ada cluster

The author benchmarked long-context prefill on a 7-GPU mixed Blackwell/Ada cluster; on Qwen3.5-397B-A17B with 75k tokens, vLLM reached 9.8s TTFT and 7,683 t/s, while llama.cpp took 57.2s and 1,319 t/s.

Why it matters: A hands-on test of long-context prefill across seven mixed Blackwell and Ada GPUs, with vLLM hitting 9.8s first-token latency at 75k tokens and 7,683 tok/s prefill, roughly 4 to 6 times llama.cpp.

Read the original ↗Export Markdown