Skip to content
r/LocalLLaMA

MTP speculative decoding with Gemma 4: assistant model choice makes or breaks speed gains

Not All MTP Assistants Are Created Equal

A user tested MTP speculative decoding with Gemma 4 Heretic models in llama.cpp and found assistant model selection is everything. A 26B Q8 jumped from 30 t/s to 62 t/s; a 12B Q4 went from 12 t/s to 54 t/s. Two GGUFs with the same name aren't always identical. Unquantized assistants consistently beat Q4/Q8 assistants by roughly 10 t/s. Draft count of 1 gave the best results across the board. Always check logs to confirm MTP actually initialized—otherwise you're benchmarking the base model by accident.

Why it matters: A user benchmarked Gemma 4 MTP speculative decoding on llama.cpp, and the assistant model picked changed how much speed was gained.

Read the original ↗Export Markdown