Epoch AI and METR launch MirrorCode to test AI on reimplementing full software projects
What's the largest software project AI can complete on its own?
MirrorCode is a new benchmark where AI must reimplement 25 full programs from scratch without seeing the source code, matching the original output exactly on end-to-end tests. Tasks span Unix utilities, data serialization, bioinformatics, interpreters, static analysis, cryptography, and compression. Unlike existing benchmarks, it provides a real inference budget: the most expensive run cost $2,600 and the AI worked for 19 days without human intervention. Epoch AI estimates a human engineer without AI would need months for the hardest tasks. The benchmark is cheat-resistant by design, though the post doesn't detail the mechanism.
Why it matters: Epoch AI and METR built MirrorCode so models rebuild 25 whole programs without seeing the source, with real budgets up to $2,600 and 19 days of inference.