Screen memory's third path: store pointers, not pixels

Ambient Context, a two-day open-source prototype, grabs foreground window text and file paths every 5 seconds and writes them to a local Markdown file—no screenshots, no network. It splits records into two layers: low-fidelity window text as an index, and high-fidelity originals accessed on demand via file: paths. Compared with Microsoft Recall requiring dedicated chips and Rewind shutting down its screen-recording feature, this pointer-over-pixel approach sidesteps both privacy and cost burdens. The post runs the visual-token math: sampling a 2K screen once per second burns ~250K tokens per hour, while a full day of deduplicated text lands in the tens of thousands. The rule is clear—store pointers when the original file still lives on disk, store bytes for ephemeral meetings, and use low-fidelity text to decide where to jump back.

Why it matters: Ambient Context splits screen memory into a third path between full-pixel recording and text-only logs: store local file paths as pointers, with grabbed text as an index. The article includes code verification and cost calculations — solid information density. Not scoring high...