Language Models Need Sleep: Let AI Nap Before Continuing Inference
大模型也需要睡觉!让AI打个盹,醒来更聪明
Carnegie Mellon University and the University of Maryland propose a “sleep” mechanism for language models: when the context window is nearly full, the model stops accepting new tokens, runs multiple offline recursive forward passes to compress accumulated context into fast weights, clears the KV cache, and then resumes inference; tests cover cellular automata, multi-hop graph retrieval, and GSM-Infinite reasoning tasks.
Why it matters: Carnegie Mellon and the University of Maryland give LLMs a sleep-like mechanism that pauses and compresses memory as context fills, a transferable approach to long-context memory management.