A 27B model runs on iPhone—two paths for what on-device LLMs are actually good for
Bonsai 27B compresses Qwen3.6-27B to ~1.125 bit/weight, fits a ~3.9 GB working set on iPhone 17 Pro Max, and scores 76.11 average on 15 thinking-mode benchmarks—keeping ~89.5% of the base model’s capability but dropping noticeably on vision and tool use. MiniCPM-V 4.6 takes the other path: 1.3B total params optimized for on-device OCR, screenshots, and UI understanding, where vision prefill dominates latency. The post frames the real question as “what is it useful for”: text reasoning favors a large base with extreme quantization; reading receipts and documents favors vision-encoding efficiency; multi-step agents also need tool reliability, permissions, and thermal stability. No side-by-side measurements on the same iPhone are provided.
Why it matters: Bonsai 27B putting a 27B model on iPhone with real benchmark numbers marks a shift from 'can it run' to product-level discussion. The article goes beyond scores to explain the four engineering bottlenecks: memory, thermals, vision prefill, and reliability. Downside: the MiniCP...