LRU is harder to beat than KV-cache papers suggest, tested on 393 Claude Code sessions
LRU is harder to beat than the KV-cache papers suggest
This repo replays 68k requests from 393 real Claude Code sessions to test agentic KV-cache eviction policies. LRU is harder to beat than papers claim—many new policies look good on paper benchmarks but fall apart on real agent traces. The post doesn't give exact hit-rate numbers, but the core finding is that real access patterns differ sharply from academic benchmarks. Don't rush to replace LRU in production.
Why it matters: The author replays prefix caching over 68,000 requests from 393 real Claude Code sessions; three attempts to beat the production baseline failed, and recomputation under capacity pressure comes mostly from tool-call loops seconds apart.