OpenRouter launches Ori Eval: benchmark models against your own prompts to find the best fit
OpenRouter 推出 Ori Eval:用你的数据证明哪款模型最适合你的项目
OpenRouter released Ori Eval on July 31, a tool that benchmarks models directly inside your codebase. It scans every place your code calls a model, asks whether you care more about accuracy, latency, or cost, then auto-generates eval files and runs your real prompts against five recent models. The output is a table showing bug catch rate, p50 latency, and cost per PR — the post's example lists Claude Opus 5 at 94% catch, 38s p50, $0.041 per PR. The eval file is code you can run in CI to block regressions and re-run when new models drop. You start by telling your coding agent a single curl command; no eval-writing experience needed.
Why it matters: OpenRouter launched Ori Eval, letting developers compare 5 models on real prompts from their own codebase and see bug catch rate, median latency and cost per task.