A Berkeley team built an AI that scores perfectly on SWE-bench while fixing 0 bugs
SWE-bench满分,0个bug修复:伯克利造了个专门作弊的AI
Berkeley RDI used a roughly 10-line conftest.py exploit to score 100% on all 500 SWE-bench tasks while fixing 0 bugs. The post says its agent broke 8 major agent benchmarks with scores from 73% to 100%, via pytest hook tampering, file:// answer reads, and faulty validators. The real issue is benchmark isolation failure, not stronger models.
Why it matters: A Berkeley RDI team scored full marks on all 500 SWE-bench tasks with about 10 lines of pytest hook code and zero real fixes, and the same trick broke 8 other major agent benchmarks.