PFlash: 10x prefill speedup over llama.cpp at 128K on an RTX 3090
PFlash: 10x prefill speedup over llama.cpp at 128K on a RTX 3090
PFlash cuts Qwen3.6-27B Q4_K_M 128K TTFT to 24.8s on an RTX 3090, versus 248.4s cold for llama.cpp. It uses a Qwen3-0.6B drafter to score token importance, keeps 5% of spans, and runs C++/CUDA without Python, Triton, or PyTorch. The quality caveat is clear: only NIAH single-needle passes from 32K to 128K; RULER and multi-needle results are not disclosed.
Why it matters: PFlash uses Qwen3-0.6B to pick the 5% of chunks that matter, cutting first-token latency on a 128K-token RTX 3090 run from 248.4s to 24.8s.