TurboFieldfare: Run Gemma 4 26B on any M-series Mac with 2 GB RAM
Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac
A Swift + Metal inference engine runs 4-bit Gemma 4 26B-A4B-IT using about 2 GB RAM. The 14 GB weights won't fit conventional tools on 8 GB Macs. It keeps shared layers and KV cache in RAM, streams routed experts per token from SSD, and hides SSD latency with a small expert cache plus parallel preads. Hits 5–6 tok/s on an 8 GB M2 MacBook Air, 31–35 tok/s on an M5 MacBook Pro. Includes an experimental OpenAI-compatible local server with streaming and tool calls. The post doesn't spell out quantization details or expert cache hit rates.
Why it matters: An open-source inference engine in Swift and Metal squeezes Gemma 4 26B, still 14GB after 4-bit quantization, into about 2GB of memory so an 8GB Mac can run it.