Aider Polyglot what it measures, how it is scored and who leads it
Coding · Aider · introduced Dec 2024 · Pass rate
Scores from Aider Polyglot, published Oct 3, 2025
About this benchmark
- Task
- Edit code to solve hard programming exercises in six languages, through the Aider coding assistant.
- Dataset
- 225 of the hardest Exercism exercises in C++, Go, Java, JavaScript, Python and Rust.
- Method
- Two tries per exercise; the second sees the failing test output. Edits must apply cleanly in Aider's edit format.
- Metric
- Pass rate
- Organization
- Aider
- Introduced
- Dec 2024
- Official leaderboard
- aider.chat/docs/leaderboards/
Versions
Newest first. New versions are added, never rewritten.
| Version | Date | |
|---|---|---|
| Polyglot225 exercises in six languages. | Dec 20241 year ago | 225 exercises in six languages. |
| Code editing (Python)The earlier Aider benchmark on Python Exercism exercises. | 20233 years ago | The earlier Aider benchmark on Python Exercism exercises. |
Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.