MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering

OpenAI released MLE-bench, a benchmark built from 75 Kaggle competitions to measure ML engineering ability in AI agents. The best setup, o1-preview with AIDE scaffolding, reached at least Kaggle bronze-medal level on 16.9% of tasks; the benchmark code is open-source.

Why it matters: Strong HKR-H/K/R: OpenAI moves evaluation from exam-style tasks to real ML engineering, anchored by 75 Kaggle competitions and a 16.9% bronze-level result. Important as a benchmark release with concrete numbers, but still research rather than a major product launch, so featured,