Skip to content
r/LocalLLaMA

PFlash: 10x prefill speedup over llama.cpp at 128K on an RTX 3090

PFlash: 10x prefill speedup over llama.cpp at 128K on a RTX 3090

PFlash cuts Qwen3.6-27B Q4_K_M 128K TTFT to 24.8s on an RTX 3090, versus 248.4s cold for llama.cpp. It uses a Qwen3-0.6B drafter to score token importance, keeps 5% of spans, and runs C++/CUDA without Python, Triton, or PyTorch. The quality caveat is clear: only NIAH single-needle passes from 32K to 128K; RULER and multi-needle results are not disclosed.

Why it matters: PFlash uses Qwen3-0.6B to pick the 5% of chunks that matter, cutting first-token latency on a 128K-token RTX 3090 run from 248.4s to 24.8s.

Read the original ↗Export Markdown