I catalogued every way local models break JSON output and built a repair library across 288 model calls
I catalogued every way local models break JSON output and built a repair library, here's what I found across 288 model calls
Reddit user kexxty ran 288 structured-output calls through OpenRouter models, including Llama 3, Mistral, Command R, DeepSeek, and Qwen, and found similar JSON failure categories across local and API-only models. The MIT-licensed Python library outputguard validates against JSON Schema, applies 15 ordered repair strategies, includes 2,001 tests, and has no LLM provider dependency.
Why it matters: Reddit user kexxty ran 288 structured output tests across Llama 3, Mistral, DeepSeek and others, categorized how local models break JSON, and open-sourced a repair library, outputguard.