Hacker News front pageSep 2, 2026, 00:42 UTC+8

Slotstream runs the 104GB Qwen3.8-Flash-Next on a 48GB Mac at ~12 tok/s

Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s

carloslfu open-sourced Slotstream, an MLX + Swift tool that runs the 125B-parameter MoE model Qwen3.8-Flash-Next (104GB at 4-bit) on a Mac with only 48GB RAM. It streams expert modules from SSD on demand instead of loading everything into memory. Speed is ~12 tok/s, and it exposes an Ollama-compatible API. The post doesn't disclose time-to-first-token or the SSD model used, so real-world feel is still an open question.

Why it matters: Streams MoE experts from SSD on demand via MLX + Swift, letting a 104GB Qwen 125B model hit ~12 tok/s on a 48GB Mac. Clean engineering with an Ollama-compatible API that lowers the trial barrier. Docked a few points because it's a solo project with no community validation or m...

Read the original ↗Export Markdown

Take

This caught my eye because it pushes the hardware floor for running large models locally even lower. Qwen3.8-Flash-Next is a 125B-parameter MoE model that's still 104GB after 4-bit quantization — normally you'd need at least that much VRAM or unified memory. Slotstream only loads the active expert modules from SSD into RAM and discards the rest, so a 48GB Mac can handle it.

~12 tok/s isn't fast, but it's readable. I'd discount the real-world feel a bit: the post doesn't disclose time-to-first-token or which SSD was used. If it's the built-in slow drive, you might wait several seconds before the first word appears. The Ollama-compatible API is a practical touch — it slots right into existing toolchains.

Don't read this as "any large model runs on a 48GB Mac." The more accurate take: for MoE architectures where only a small fraction of parameters activate per inference, on-demand streaming can save a ton of memory. Local access to 100B+ models just got more options.

Source: Hacker News front page · github.com
Published: Sep 2, 2026, 00:42 UTC+8 · 28 days ago