r/LocalLLaMAMay 2, 2026, 21:44 UTC+8

I built Semvec: A constant-cost semantic memory for LLMs, looking for testers

I built "Semvec": A Constant-Cost Semantic Memory for LLMs (Looking for testers!)

A developer released Semvec, replacing unbounded chat history with fixed-size semantic state. Its 48-turn benchmark claims about 76% token reduction, with identical input footprint at turn 10 and 10,000. It supports OpenAI-compatible LLMs, MCP, Claude Code, Cursor, and multi-agent shared state.

Why it matters: HKR-H/K/R all pass, but this is a Reddit self-release with author benchmarks only. Treat it as an interesting indie memory tool, not a same-day industry story.

Read the original ↗Export Markdown

Take

Semvec is betting against endless context growth by compressing dialogue into fixed-size semantic state. The summary gives two hard hooks: a 48-turn benchmark claims about 76% fewer tokens, and turn 10 has the same input footprint as turn 10,000. The Reddit body is blocked by 403, so the task, model, scoring, and failure cases are missing.

I like the direction, especially the OpenAI-compatible API, MCP, Claude Code, Cursor, and shared multi-agent state. That is closer to developer workflow than another generic vector-memory wrapper. The catch is brutal: fixed state always drops information. Which facts get dropped, when they get dropped, and whether the system can recover them decide whether Semvec is infrastructure or a neat demo. MemGPT, Zep, and LangGraph memory have all hit that wall.

Source: r/LocalLLaMA · reddit.com
Published: May 2, 2026, 21:44 UTC+8 · May 2, 2026