AI2 Reasoning Challenge (ARC) what it measures, how it is scored and who leads it
Reasoning · Allen Institute for AI · introduced Mar 14, 2018 · Accuracy · saturated
About this benchmark
- Task
- Answer grade-school science multiple-choice questions; the Challenge set defeats simple retrieval.
- Dataset
- Grade-school science questions split into an Easy set and a Challenge set.
- Method
- Multiple choice.
- Metric
- Accuracy
- Organization
- Allen Institute for AI
- Introduced
- Mar 14, 2018
Versions
Newest first. New versions are added, never rewritten.
| Version | Date | |
|---|---|---|
| 1Released with the paper. | Mar 14, 20188 years ago | Released with the paper. |
Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.