Skip to content
QbitAI · WeChat

Beyond TurboQuant: Together AI Brings 2-bit KV Cache to Real Serving

超越TurboQuant:Together AI把2-bit KV Cache推向真实服务

Together AI, the University of Sydney, and UIUC introduced OSCAR, a 2-bit KV Cache quantization method that uses about 2.28 effective bits per KV element and scores 71.86 on Qwen3-4B-Thinking, 40.1 points above TurboQuant.

Why it matters: OSCAR compresses each KV cache element to about 2.28 bits and scores 71.86 on Qwen3-4B-Thinking, 40.1 points above TurboQuant in live serving.

Read the original ↗Export Markdown