MMLU what it measures, how it is scored and who leads it
Knowledge · UC Berkeley · introduced Sep 7, 2020 · Accuracy · saturated
About this benchmark
- Task
- Answer four-option multiple-choice questions across 57 subjects, from elementary math to law.
- Dataset
- 57 tasks, about 16,000 questions from practice exams and textbooks.
- Method
- Zero-shot or few-shot prompting; the chosen letter is compared with the key.
- Metric
- Accuracy
- Organization
- UC Berkeley
- Introduced
- Sep 7, 2020
Versions
Newest first. New versions are added, never rewritten.
| Version | Date | |
|---|---|---|
| 1Released with the paper. | Sep 7, 20206 years ago | Released with the paper. |
Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.