Skip to content
r/LocalLLaMA

Unweight: how we compressed an LLM 22% without sacrificing quality

Cloudflare released Unweight, a lossless system that compresses LLM weights by 15% to 22% with bit-exact outputs preserved. The snippet says it targets memory-bandwidth bottlenecks on GPUs like NVIDIA H100 by compressing only the BF16 exponent byte; over 99% of weights in a typical layer use 16 exponent values, saving about 3 GB VRAM on an 8B model. The key detail is on-chip decompression plus four autotuned execution paths; the post does not disclose throughput results or model coverage in the excerpt.

Why it matters: Cloudflare's open-source Unweight compresses only the exponent bits of BF16 weights, decompresses to output identical to the original model, and saves about 3GB of VRAM on an 8B model.

Read the original ↗Export Markdown