Slotstream runs the 104GB Qwen3.8-Flash-Next on a 48GB Mac at ~12 tok/s
Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s
carloslfu open-sourced Slotstream, an MLX + Swift tool that runs the 125B-parameter MoE model Qwen3.8-Flash-Next (104GB at 4-bit) on a Mac with only 48GB RAM. It streams expert modules from SSD on demand instead of loading everything into memory. Speed is ~12 tok/s, and it exposes an Ollama-compatible API. The post doesn't disclose time-to-first-token or the SSD model used, so real-world feel is still an open question.
Why it matters: Developer carloslfu open-sourced slotstream, which streams expert weights from SSD to run a 104GB Qwen model on a 48GB Mac at about 12 tok/s, with an Ollama-compatible API.