WebArena what it measures, how it is scored and who leads it
Agents and tools · Carnegie Mellon University · introduced Jul 25, 2023 · Task success
About this benchmark
- Task
- Complete tasks on realistic self-hosted websites: shopping, forums, code hosting, maps, content management.
- Dataset
- Long-horizon tasks on functional copies of popular site types.
- Method
- Programmatic checks of the final site state or answer. People reached 78%, the best GPT-4 agent 14% at release.
- Metric
- Task success
- Organization
- Carnegie Mellon University
- Introduced
- Jul 25, 2023
- Home
- webarena.dev/
Versions
Newest first. New versions are added, never rewritten.
| Version | Date | |
|---|---|---|
| 1Released with the paper. | Jul 25, 20233 years ago | Released with the paper. |
Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.