AI memory benchmarks favor recall, but products need write precision

AI Memory 的 Benchmark 偏向 Recall，产品却更需要 Precision

Most AI memory benchmarks start with a preloaded history and test retrieval, but real products face an earlier decision: should this sentence be stored at all. PASB research found that when agents autonomously write to long-term state, sycophantic errors persist at 72% vs. 45% when kept in-session—a 27-point gap. A Mem0 deployment case saw 10,134 memories cleaned down to 224, with only 38 needing no edits. ChatGPT, Claude, and Gemini are all expanding cross-session memory, but none have published adoption rates, retention impact, or correction frequency. The post argues product teams should first measure the ceiling value of ideal memory, then compare auto-write, AI-suggest-with-confirmation, and manual-save modes, weighing write precision and downstream harm against time saved.

Why it matters: The article uses PASB's experimental data (72% vs 45%) to clearly articulate the recall bias in memory benchmarks and maps gaps in existing evaluations. The argument is data-backed, not hand-waving. Deduction because it's a personal blog analysis rather than original research,...