24+ tok/s from ~30B MoE models on an old GTX 1080
24+ tok/s from ~30B MoE models on an old GTX 1080 (8 GB VRAM, 128k context)
User mdda ran Qwen 3.6 35B-A3B on an i7-6700, GTX 1080, and 32GB RAM machine at about 24 tok/s with 128k context; the setup uses llama.cpp MoE offloading plus TurboQuant/RotorQuant KV cache quantization, with PCIe 3.0 x16 saturated and GPU utilization at about 40–50%.
Why it matters: A user runs roughly 30B-parameter MoE models on an i7-6700 with a GTX 1080 at about 24 tokens per second and 128k context, with the exact setup and numbers.