BBQ what it measures, how it is scored and who leads it
Safety and honesty · New York University · introduced Oct 15, 2021 · Accuracy and bias score
About this benchmark
- Task
- Answer questions where a social stereotype could bias the answer.
- Dataset
- Hand-built question sets across nine social dimensions, with ambiguous and disambiguated contexts.
- Method
- Accuracy and a bias score comparing stereotype-consistent and inconsistent errors.
- Metric
- Accuracy and bias score
- Organization
- New York University
- Introduced
- Oct 15, 2021
Versions
Newest first. New versions are added, never rewritten.
| Version | Date | |
|---|---|---|
| 1Released with the paper. | Oct 15, 20214 years ago | Released with the paper. |
Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.