Show HN: A New Benchmark for Testing LLMs for Deterministic Outputs
Show HN: A new benchmark for testing LLMs for deterministic outputs
Interfaze released Structured Output Benchmark, scoring schema pass rate, types, and value accuracy across text, image, and audio. Each record has a JSON Schema and human plus LLM-checked ground truth; GLM-4.7 ranks No. 2 overall. The key bug is field-level value error: GPT-5.4 ranks 3rd on text and 9th on images.
Why it matters: Interfaze released the SOB benchmark, testing structured output across 5,000 text items, 209 images and 115 audio clips, checking whether field values are correct rather than just whether the JSON parses.