MLE-bench what it measures, how it is scored and who leads it

Name: MLE-bench
Creator: OpenAI
Published: 2024-10-09

Coding · OpenAI · introduced Oct 9, 2024 · Any medal rate

About this benchmark

Task: Act as a machine-learning engineer on Kaggle competitions: prepare data, train, submit.
Dataset: 75 Kaggle competitions.
Method: Agents submit to each competition offline; scored by the medal their result would have earned.
Metric: Any medal rate
Organization: OpenAI
Introduced: Oct 9, 2024
Paper: MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering (arXiv 2410.07095)
Home: github.com/openai/mle-bench

Newest first. New versions are added, never rewritten.

Version	Date	What changed
1Released with the paper.	Oct 9, 20242 years ago	Released with the paper.

Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.