LRU is harder to beat than KV-cache papers suggest, tested on 393 Claude Code sessions

LRU is harder to beat than the KV-cache papers suggest

This repo replays 68k requests from 393 real Claude Code sessions to test agentic KV-cache eviction policies. LRU is harder to beat than papers claim—many new policies look good on paper benchmarks but fall apart on real agent traces. The post doesn't give exact hit-rate numbers, but the core finding is that real access patterns differ sharply from academic benchmarks. Don't rush to replace LRU in production.

Why it matters: Replays real agent traces to stress-test KV-cache eviction policies, directly pushing back on papers that only cite academic benchmarks. 393 sessions and 68k requests is solid scale, but the repo doesn't disclose specific hit-rate numbers, so the score stays at the featured th...