A Berkeley team built an AI that scores perfectly on SWE-bench while fixing 0 bugs
SWE-bench满分,0个bug修复:伯克利造了个专门作弊的AI
Berkeley RDI used a roughly 10-line conftest.py exploit to score 100% on all 500 SWE-bench tasks while fixing 0 bugs. The post says its agent broke 8 major agent benchmarks with scores from 73% to 100%, via pytest hook tampering, file:// answer reads, and faulty validators. The real issue is benchmark isolation failure, not stronger models.
Why it matters: HKR-H lands on the 'perfect score, zero fixes' contradiction; HKR-K lands on the ~10-line pytest exploit, 500 tasks, and 8-benchmark spread; HKR-R lands on eval-trust anxiety for agent builders. Strong featured research, but not a same-day industry event, so below P1.