Hacker News front pageSep 22, 2026, 02:53 UTC+8

Frontier AI on Your Own Hardware

Tim Dettmers's dlab is open-sourcing a full stack this week to run frontier AI on local hardware. An agent auto-optimized Metal kernels to run Qwen 3.6 35B-A3B at 1.5 bits per weight, hitting 450 tokens/s on a Mac. The core argument: the unit of research is no longer the paper but a coherent ecosystem. Full details are still under wraps, but the release includes an autonomous research agent, efficient test-time scaling, and auto-compaction that beats Claude Code on token savings.

Why it matters: Tim Dettmers is a key figure in quantization, and this isn't a single paper but a full toolchain release with concrete numbers (1.5 bits, 450 tok/s) and a reproducible path. The deduction: it's a blog announcement — actual usability and compatibility won't be clear until the o...

Read the original ↗Export Markdown

Take

I clicked because Tim Dettmers has real credibility in quantization and local inference. He's not dropping a single paper—he's releasing a stack where each piece feeds the next: an agent that auto-optimizes Metal kernels, running Qwen 3.6 35B-A3B at 1.5 bits per weight and hitting 450 tokens/s on a Mac. That speed for a 35B model on local hardware is genuinely fast.

His bigger claim is that the unit of research is now the ecosystem, not the paper. I partly buy it—when individual experiments take weeks instead of a year, the hard part is making components that don't step on each other.

What's missing: full evals and real-world usability checks. The post says the releases roll out this week. I'd wait for the code to land before judging. If the 1.5-bit quality loss is tolerable, this is a real cost-saver for teams running frontier models locally.

Source: Hacker News front page · timdettmers.com
Published: Sep 22, 2026, 02:53 UTC+8 · 8 days ago