Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps

RTPurbo converts full-attention LLMs to sparse inference with a few hundred adaptation steps. It keeps the full KV cache only for retrieval heads, uses a 16-dimensional token indexer, and reports up to 9.36x prefill speedup at 1M context plus about 2.01x decode speedup on long-context and reasoning benchmarks.

Why it matters: HKR-H/K/R all pass: the hook is counterintuitive, the post gives 1M-context speedup numbers, and inference cost resonates. Reddit-only sourcing and missing model/code details keep it in the 78–84 band.