Frontier models are much better at physics than benchmarks suggest—expert re-grading shows why
How good are frontier models at physics?
Researchers at Yale and other institutions had physics faculty and PhDs re-grade six widely used physics benchmarks. Most answers previously marked wrong turned out to be grader errors, incorrect reference solutions, or ambiguous questions. For GPT-5.6-Sol, corrected mean@4 jumped from 47.3% to 78.7% on HLE-Physics and from 61.0% to 87.2% on CMT-Benchmark. The near-saturation on these closed-ended tasks signals an urgent need for harder, expert-validated evaluations.
Why it matters: Researchers at Yale and elsewhere had physics experts re-grade six common physics benchmarks and found most wrong answers came from the scoring code, reference answers or question wording, not from model reasoning.