Skip to content
Computing Life · Share · Yage

Tencent Hunyuan Hy3 1-bit quantization: fits on one GPU, but is it usable?

Tencent released IQ1_M quants for the 295B Hy3 MoE model, fitting into a single 96GB GPU at ~85.5 GiB. Actual BPW is ~2.4, not 1-bit. With 64K q8 KV cache, only ~2 GiB headroom remains—official config is marked tight. Agent and coding abilities dip slightly vs BF16; multi-step reliability is untested. Community GGUF hits ~23–25 tok/s on M3 Max, but 8K prompt prefill takes ~200s. Worth experimenting if you already own the hardware, not a reason to buy it.

Why it matters: Tencent squeezed its 295B-parameter Hunyuan Hy3 to about 85.5 GiB, fitting one 96GB GPU, with 1-bit averaging about 2.4 bits per weight.

Read the original ↗Export Markdown