GSM8K what it measures, how it is scored and who leads it
Math · OpenAI · introduced Oct 27, 2021 · Accuracy · saturated
About this benchmark
- Task
- Solve grade-school math word problems that take two to eight steps.
- Dataset
- 8,500 linguistically diverse problems written by people, with a 1,319-problem test set.
- Method
- Free-form answer; the final number is compared with the key.
- Metric
- Accuracy
- Organization
- OpenAI
- Introduced
- Oct 27, 2021
Versions
Newest first. New versions are added, never rewritten.
| Version | Date | |
|---|---|---|
| 1Released with the paper. | Oct 27, 20214 years ago | Released with the paper. |
Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.