Hacker News front pageSep 5, 2026, 11:38 UTC+8

CodeRabbit evaluates GPT-6 Astra: 20% more cross-file bugs caught than Sol

GPT-6 Astra in code review: Gains, privacy, and cost

CodeRabbit benchmarked OpenAI's GPT-6 Astra on code review. It caught ~4% more actionable bugs overall vs GPT-5.6 Sol, and 20% more on hard cross-file reviews. API pricing is steep: $10/1M input tokens, $50/1M output—roughly 2.5× Sol's cost for a 100K-input-token task. The post doesn't disclose benchmark size, bug-type breakdown, or false-positive rate, so treat the absolute numbers as directional.

Why it matters: CodeRabbit's own benchmark shows GPT-6 Astra pulling ahead on cross-file review — the hardest sub-task — by 20% over Sol and 33% over Opus 5, with real cost and privacy data. Capped below 85 because it's a single-vendor eval, not an independent benchmark, and Astra itself isn'...

Read the original ↗Export Markdown

Take

CodeRabbit ran GPT-6 Astra on their internal code-review benchmark. The headline is cross-file reviews—the hard stuff where a change in one file breaks something elsewhere. Astra caught 20% more actionable bugs than GPT-5.6 Sol and 33% more than Opus 5. That's a real signal that the model is getting better at connecting scattered evidence across a codebase.

But the pricing is steep: $10/1M input tokens, $50/1M output. For a 100K-input-token task, that's roughly 2.5× Sol's cost. Teams will need to decide whether the extra bugs caught justify the bill.

I'd discount the absolute numbers a bit. The post doesn't share benchmark size, bug-type breakdown, or false-positive rate, so treat these as directional. The cross-file gain is the useful bit; the rest needs more data before you build a business case around it.

Source: Hacker News front page · coderabbit.ai
Published: Sep 5, 2026, 11:38 UTC+8 · 25 days ago