Skip to content
Computing Life · Share · Yage

After SWE-Bench Pro Saturation, Someone Built a New Benchmark

SWE-Bench Pro 饱和之后,有人做了一把新尺子

DeepSWE says SWE-Bench Pro lost discrimination because of data contamination and verifier flaws; the same model set showed a 62-point spread on the benchmark, while the snippet does not disclose the audited models or test protocol.

Why it matters: Datacurve's DeepSWE benchmark spreads the same models across a 62-point gap where SWE-Bench Pro shows only a few points, showing the old benchmark can no longer separate models.

Read the original ↗Export Markdown