r/LocalLLaMAMay 9, 2026, 05:15 UTC+8

MTP + TurboQuant Running: Qwen3.6-27B Hits 80+ t/s on a Single RTX 4090

Got MTP + TurboQuant running — Qwen3.6-27B -- 80+ t/s at 262K context on a single RTX 4090

indrasmirror ran Qwen3.6-27B-Heretic-v2 on a single RTX 4090 with 262K context, TBQ4_0 KV cache, and MTP draft 3, improving throughput from about 43 t/s to 80-87 t/s with roughly 73% MTP draft acceptance.

Why it matters: HKR-H/K/R all pass, backed by a numbered first-person experiment. The Reddit-only source and niche local-inference focus keep it below the 78–84 band for broader industry releases.

Read the original ↗Export Markdown

Take

I’d down-rank this before celebrating consumer-GPU long-context inference. The hard claim is specific: Qwen3.6-27B-Heretic-v2 on one RTX 4090, 262K context, TBQ4_0 KV cache, MTP draft 3, moving from 43 t/s to 80-87 t/s with about 73% draft acceptance. The body is only a Reddit 403, so there are no commands, quant settings, VRAM numbers, prompt mix, or prefill/decode split.

If the claim holds, the story is MTP plus compressed KV cache making a 27B long-context model usable on hobbyist hardware. It is not a fresh proof that Qwen suddenly jumped. DeepSeek already made the MTP engineering dividend obvious; the fragile part is whether that 73% acceptance survives code tasks, Chinese long-doc prompts, and multi-turn agent loops. Without reproduction, this is an attractive performance receipt, not a benchmark.

Source: r/LocalLLaMA · reddit.com
Published: May 9, 2026, 05:15 UTC+8 · May 9, 2026