Shoehorn: Quantize any model to fit your exact memory budget, down to the byte
Show HN: Shoehorn – Quantize any model down to run on your machine
Shoehorn is an open-source quantizer that starts from your available memory, subtracts inference overhead, then solves a per-tensor mixed-precision assignment that routinely uses over 99.99% of the budget. It avoids preset quantization tiers that either waste hundreds of megabytes or fail at load time. The local web UI measures your machine, streams the fit, shows perplexity cost, and launches a chat. It requires llama.cpp on PATH, outputs standard GGUF v3, and runs on macOS Apple Silicon, Linux, and Windows. The quantizer is written from scratch in Rust.
Why it matters: The open-source quantization tool Shoehorn measures available machine memory first, then assigns mixed precision layer by layer, leaving 13 KB of headroom on a 519.2 MiB budget in the official example.