XSTest what it measures, how it is scored and who leads it
Safety and honesty · Bocconi University and University of Oxford · introduced Aug 2, 2023 · Over-refusal rate
About this benchmark
- Task
- Answer safe prompts that only look risky, and refuse the unsafe contrasts.
- Dataset
- 250 safe prompts in ten types and 200 unsafe contrast prompts.
- Method
- Responses classified as full compliance, full refusal or partial.
- Metric
- Over-refusal rate
- Organization
- Bocconi University and University of Oxford
- Introduced
- Aug 2, 2023
Versions
Newest first. New versions are added, never rewritten.
| Version | Date | |
|---|---|---|
| 1Released with the paper. | Aug 2, 20233 years ago | Released with the paper. |
Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.