Neuralwatt analyzes agent framework caching and energy use
What happened
On October 17, Hacker News' front page featured Neuralwatt's analysis of how agent runtimes, caching and energy use relate. Looking at cloud traffic and experimental requests, Neuralwatt found that a runtime changing existing prompts breaks the cache and adds recomputation and energy use. With the same model and similar input and output lengths, requests that mostly miss the cache use about twice the energy of cache hits, and about 2.5 times for GLM-5.3. The analysis also says that past 128k tokens of context, one broad re-read costs roughly the prefill compute of 600 ordinary turns.
Written by AI from the coverage · updated 2 hours ago
Coverage
Follow the reports to see the story from different sides.
- Hacker News front pagePickNeuralwatt on how agent harnesses affect energy use and token efficiency
Neuralwatt analyzed cloud traffic and experimental requests and found that an agent harness editing an existing prompt breaks the cache, adding recomputation and energy use. With the same model and similar input and output lengths, requests that mostly miss the cache use about twice the energy of cache-hit requests, and about 2.5 times for GLM-5.3. When context passes 128k tokens, one broad re-read costs prefill compute roughly equal to 600 ordinary turns.
Heat over time
Not enough continuous observations to draw a trend yet.