GSM8K what it measures, how it is scored and who leads it

Name: GSM8K
Creator: OpenAI
Published: 2021-10-27

Math · OpenAI · introduced Oct 27, 2021 · Accuracy · saturated

About this benchmark

Task: Solve grade-school math word problems that take two to eight steps.
Dataset: 8,500 linguistically diverse problems written by people, with a 1,319-problem test set.
Method: Free-form answer; the final number is compared with the key.
Metric: Accuracy
Organization: OpenAI
Introduced: Oct 27, 2021
Paper: Training Verifiers to Solve Math Word Problems (arXiv 2110.14168)
Home: github.com/openai/grade-school-math

Newest first. New versions are added, never rewritten.

Version	Date	What changed
1Released with the paper.	Oct 27, 20215 years ago	Released with the paper.

Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.