r/LocalLLaMAMay 20, 2026, 08:41 UTC+8

Running DeepSeek-V4 locally on 4 legacy RTX 2080 Ti GPUs with W8A8 at 255 prefill tok/s

Running DeepSeek-V4 locally with 4x legacy RTX 2080 Ti ($2k budget setup). Custom Turing kernels, W8A8 quantization, and 255 prefill tok/s!

A Reddit user ran DeepSeek-V4-Flash locally on 4 RTX 2080 Ti GPUs, reporting 284B total parameters, 13B active parameters, a sub-$2,500 build, custom Turing CUDA kernels, W8A8 quantization, 1TB DDR4 ECC RAM, and about 255 prefill tokens/s.

Why it matters: HKR-H/K/R all pass: this is a numeric first-person local-inference experiment. Single-source Reddit provenance and custom Turing kernels keep it in the lower featured band.

Read the original ↗Export Markdown

Take

A 4×RTX 2080 Ti box running DeepSeek-V4-Flash at 255 prefill tok/s hits the inference cost curve, not just the hobbyist-flex lane. The title claims 284B total parameters, 13B active parameters, W8A8 quantization, custom Turing CUDA kernels, 1TB DDR4 ECC RAM, and a sub-$2,500 build. Reddit 403 blocks the body, so code, batch size, context length, and decode tok/s are not verifiable here.

I’d treat this as suspicious in number, correct in direction. The 2080 Ti is old Turing hardware, without the clean modern INT8 path people expect on newer NVIDIA cards. If the result holds, the trick is likely MoE sparsity, CPU memory spill, and a generous prefill metric. The pattern matches what vLLM and llama.cpp have done all year: open inference does not need to beat H100 clusters to matter; it only needs to make “good enough” local serving cheap on discarded hardware.

Source: r/LocalLLaMA · reddit.com
Published: May 20, 2026, 08:41 UTC+8 · May 20, 2026