MGSM what it measures, how it is scored and who leads it

Name: MGSM
Creator: Google
Published: 2022-10-06

Math · Google · introduced Oct 6, 2022 · Accuracy

About this benchmark

Task: Solve grade-school math word problems in ten typologically diverse languages.
Dataset: 250 GSM8K problems translated by people into ten languages.
Method: Chain-of-thought prompting; exact match on the final number.
Metric: Accuracy
Organization: Google
Introduced: Oct 6, 2022
Paper: Language Models are Multilingual Chain-of-Thought Reasoners (arXiv 2210.03057)
Home: github.com/google-research/url-nlp

Newest first. New versions are added, never rewritten.

Version	Date	What changed
1Released with the paper.	Oct 6, 20223 years ago	Released with the paper.

Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.