AI Chat-Group Daily (群聊日报)Jul 26, 2026, 15:27 UTC+8

Opus 5 Day 2: Saturation self-testing trades cost for quality, total cost may beat Fable

2026-07-25 群聊日报

Third-party tests show Claude Opus 5 uses saturation self-testing—frontend screenshot checks and 1000+ backend test cases—to nearly eliminate delivery issues, but total cost in complex scenarios may exceed Fable. With self-testing off, bug rates don't beat the previous model. Anthropic's strategy: long-chain debugging over one-shot correctness. The official model card advises against max effort for the first time; FrontierBench peaks at xhigh. Another test reveals ~80% of Claude Code's system prompt was cut. Group sentiment is positive, some calling it smoother than Fable. OpenAI had a full 503 outage overnight; reset cards landed the next day. WSJ reports US companies are mixing cheaper models to control costs, with Cursor as a beneficiary.

Why it matters: Third-party testing delivers the most concrete behavioral and cost data on Opus 5 so far — the self-testing tradeoff is a real signal. Slight discount for being a group-chat digest rather than the original review, but density clears the featured bar.

Read the original ↗Export Markdown

Take

This is worth opening because the third-party tests nail down Opus 5's behavioral shift: the frontend checks every function against requirement screenshots, the backend writes 1000+ test cases, and most delivery issues just vanish. The tradeoff is massive time and token burn—total cost in complex scenarios may exceed Fable. Turn off self-testing and the raw code bug rate doesn't beat the previous model. Anthropic's bet is long-chain debugging over one-shot correctness.

Another detail that matters: the official model card shows FrontierBench peaks at xhigh, not max. Anthropic explicitly tells users to redo effort sweeps instead of blindly copying Opus 4.8 configs. Someone in the group confirmed sol's max isn't always better than xhigh either—overthinking is real.

On the system prompt side, about 80% of Claude Code's prompt got cut—excess examples, hard bans, and things the model already handles on its own. This mirrors OpenAI's earlier simplification that lifted scores 10–15% and dropped tokens 41–66%. Group sentiment is positive; one person said it feels smoother than Fable.

I'd discount this a bit: these are personal tests from a chat group, not standardized benchmarks, and the task definitions aren't fully public. But if the saturation self-testing behavior holds, it's more informative than benchmark scores—you're no longer paying for raw intelligence, you're paying for the model's patience to check its own work.

Source: AI Chat-Group Daily (群聊日报) · louyu2015.github.io
Published: Jul 26, 2026, 15:27 UTC+8 · Jul 26, 2026