Do Models Need Sleep? CMU Paper Lets LLMs Consolidate Memory During “Sleep”
模型也需要「睡觉」?CMU新论文让LLM在梦中「巩固记忆」
CMU and the University of Maryland propose Language Models Need Sleep: when each L-token context window fills, the model runs N offline recurrent forward passes and updates SSM fast weights before evicting the KV cache. On GSM-Infinite, Jet-Nemotron 2B with 6 sleep loops improves 6-step arithmetic accuracy from 0.742 to 0.812.
Why it matters: CMU and University of Maryland researchers gave LLMs a sleep mechanism, compressing context into SSM fast weights via offline recurrent forward passes when the window fills, then clearing the cache.