Skip to content
Hacker News front page

MemStitch: Zero-copy KV cache stitching for vLLM cuts multi-agent TTFT by up to 25x

Show HN: MemStitch – Zero-copy context bridging for vLLM (25x TTFT speedup)

DaqulaLin open-sourced MemStitch, a gateway that sits in front of vLLM and stitches KV caches across requests at the memory level using PagedAttention. It skips redundant prefill in multi-agent workflows, cutting TTFT by up to 25x and saving over 40% VRAM. The post doesn't disclose the test model or GPU setup, so I'd discount the 25x claim until those details surface.

Why it matters: MemStitch stitches KV caches from different requests directly in vLLM's memory layer, skipping repeated prefill in multi-agent collaboration, cutting time to first token by up to 25x and memory use by over 40%.

Read the original ↗Export Markdown