HalBench tests 29 open models on sycophancy and hallucination; Qwen 3.6 and Gemma 4 punch far above their weight

HalBench: 29 OSS models tested on a custom built Sycophancy and Hallucination Benchmark, Qwen 3.6 and Gemma 4 scoring far above their weight! (While Meta keeps proving they forgot how to spend their money...)

HalBench is an open benchmark that gives models a false premise and measures whether they push back or play along. v2.3 covers 33 models, 29 of them open. Only Sonnet 4.6 (65.1%) and Grok 4.3 (50.9%) clear 50% pushback. The best open model is Qwen 3.6 (~27B dense) at 36.6%, beating GPT-5.4 and Gemini 3.1 Pro. Gemma 4 26B follows at 29.2%. Model size barely predicts performance; phi-4 sits dead last at 2.3%. Dataset, scoring code, and Space are all open.

Why it matters: A community benchmark with concrete numbers and rankings, where Qwen 3.6 outperforms GPT-5.4 on refusal rate, is real signal. Not p1 because it's a self-built eval without peer review yet — treating it as a strong recommendation.