Hugging Face used 1,200 people + coding agents to reproduce 2,200 ICML 2026 papers
What We Learned by Reproducing 2,200 papers from ICML
Hugging Face ran a 19-day hackathon where 1,200+ participants used coding agents like Claude Code and Codex to reproduce claims from ICML 2026 papers. They covered 2,226 papers, roughly a third of the conference. One spotlight paper had a reviewer admitting they didn't check the proofs carefully; the reproduction later caught real issues. The core question: when agents can run experiments and write papers at scale, what role do humans play in research?
Why it matters: Hugging Face got over 1,200 people to reproduce 2,226 ICML 2026 papers with coding agents in 19 days, covering nearly a third of submissions and surfacing proof gaps reviewers missed.