Skip to content
r/LocalLLaMA

MTP + TurboQuant Running: Qwen3.6-27B Hits 80+ t/s on a Single RTX 4090

Got MTP + TurboQuant running — Qwen3.6-27B -- 80+ t/s at 262K context on a single RTX 4090

indrasmirror ran Qwen3.6-27B-Heretic-v2 on a single RTX 4090 with 262K context, TBQ4_0 KV cache, and MTP draft 3, improving throughput from about 43 t/s to 80-87 t/s with roughly 73% MTP draft acceptance.

Why it matters: indrasmirror pushed Qwen3.6-27B to 262K context on one RTX 4090, using MTP speculative decoding to lift generation from about 43 t/s to 80-87 t/s at a 73% draft acceptance rate.

Read the original ↗Export Markdown