ARC-AGI-1 what it measures, how it is scored and who leads it
Reasoning · ARC Prize Foundation (created by François Chollet) · introduced Nov 5, 2019 · Tasks solved
About this benchmark
- Task
- Infer the rule behind a few input-output grid pairs and apply it to a new grid.
- Dataset
- Public training and evaluation sets of abstract grid puzzles, plus semi-private and private evaluation sets.
- Method
- Two attempts per task (pass@2); an answer counts only if every cell of the output grid is right.
- Metric
- Tasks solved
- Organization
- ARC Prize Foundation (created by François Chollet)
- Introduced
- Nov 5, 2019
- Official leaderboard
- arcprize.org/leaderboard
Versions
Newest first. New versions are added, never rewritten.
| Version | Date | |
|---|---|---|
| Public leaderboardARC Prize opens a public leaderboard with semi-private evaluation. | Jun 27, 20242 years ago | ARC Prize opens a public leaderboard with semi-private evaluation. |
| 1Published with the paper as the Abstraction and Reasoning Corpus. | Nov 5, 20196 years ago | Published with the paper as the Abstraction and Reasoning Corpus. |
Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.