SciCode what it measures, how it is scored and who leads it
Coding · University of Illinois and collaborators · introduced Jul 18, 2024 · Main problems solved
About this benchmark
- Task
- Write code for research problems from 16 natural-science subfields.
- Dataset
- 80 main problems decomposed into 338 subproblems, curated by scientists.
- Method
- Code is run against scientist-written tests, per subproblem and per main problem.
- Metric
- Main problems solved
- Organization
- University of Illinois and collaborators
- Introduced
- Jul 18, 2024
Versions
Newest first. New versions are added, never rewritten.
| Version | Date | |
|---|---|---|
| 1Released with the paper. | Jul 18, 20242 years ago | Released with the paper. |
Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.