LiveCodeBench what it measures, how it is scored and who leads it
Coding · UC Berkeley, MIT and Cornell · introduced Mar 12, 2024 · pass@1
About this benchmark
- Task
- Solve competitive programming problems published after a model was trained; also self-repair and test-output prediction.
- Dataset
- Problems collected over time from LeetCode, AtCoder and Codeforces.
- Method
- Generated code is run against tests; results can be sliced by problem date to avoid contamination.
- Metric
- pass@1
- Organization
- UC Berkeley, MIT and Cornell
- Introduced
- Mar 12, 2024
- Official leaderboard
- livecodebench.github.io/leaderboard.html
Versions
Newest first. New versions are added, never rewritten.
| Version | Date | |
|---|---|---|
| release_v61,055 problems up to April 2025. | Apr 20251 year ago | 1,055 problems up to April 2025. |
| release_v5880 problems up to January 2025. | Jan 20251 year ago | 880 problems up to January 2025. |
| release_v4713 problems up to September 2024. | Sep 20242 years ago | 713 problems up to September 2024. |
| release_v3612 problems up to July 2024. | Jul 20242 years ago | 612 problems up to July 2024. |
| release_v2511 problems up to May 2024. | May 20242 years ago | 511 problems up to May 2024. |
| release_v1400 problems published May 2023 to March 2024. | Mar 20242 years ago | 400 problems published May 2023 to March 2024. |
Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.