Interesting Paper Advocates Quantized Prefilling and Precise Decoding
arXiv 2605.20315 argues for W4A4 quantization during prefilling to target a theoretical 4x gain, while keeping decoding on the original high-precision path because activation errors can perturb sampled tokens and accumulate across autoregressive generation.
Why it matters: The paper proposes W4A4 quantization during prefill while keeping decoding at high precision, claiming up to 4x faster input processing without letting activation quantization error accumulate through autoregressive generation.