GAIA what it measures, how it is scored and who leads it
Agents and tools · Meta, Hugging Face and AutoGPT · introduced Nov 21, 2023 · Accuracy
About this benchmark
- Task
- Answer real-world questions that need browsing, tools, files and several steps.
- Dataset
- 466 questions with unambiguous answers in three difficulty levels; most answers are held out.
- Method
- Exact match on a short final answer. People scored 92%, GPT-4 with plugins 15% at release.
- Metric
- Accuracy
- Organization
- Meta, Hugging Face and AutoGPT
- Introduced
- Nov 21, 2023
- Official leaderboard
- huggingface.co/spaces/gaia-benchmark/leaderboard
Versions
Newest first. New versions are added, never rewritten.
| Version | Date | |
|---|---|---|
| 1Released with the paper. | Nov 21, 20232 years ago | Released with the paper. |
Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.