Beyond TurboQuant: Together AI Brings 2-bit KV Cache to Real Serving
超越TurboQuant:Together AI把2-bit KV Cache推向真实服务
Together AI, the University of Sydney, and UIUC introduced OSCAR, a 2-bit KV Cache quantization method that uses about 2.28 effective bits per KV element and scores 71.86 on Qwen3-4B-Thinking, 40.1 points above TurboQuant.
Why it matters: OSCAR compresses each KV cache element to about 2.28 bits and scores 71.86 on Qwen3-4B-Thinking, 40.1 points above TurboQuant in live serving.