τ-bench (tau-bench) what it measures, how it is scored and who leads it
Agents and tools · Sierra · introduced Jun 17, 2024 · pass^1
About this benchmark
- Task
- Serve a simulated customer through tools while following a domain policy.
- Dataset
- Retail and airline domains with databases, tools and policies; τ²-bench adds a telecom domain where the user also acts.
- Method
- A model plays the user; success means the final database state matches the goal. pass^k measures consistency over k tries.
- Metric
- pass^1
- Organization
- Sierra
- Introduced
- Jun 17, 2024
- Official leaderboard
- taubench.com/
Versions
Newest first. New versions are added, never rewritten.
| Version | Date | |
|---|---|---|
| τ²-benchAdds the dual-control telecom domain (arXiv 2506.07982). | Jun 9, 20251 year ago | Adds the dual-control telecom domain (arXiv 2506.07982). |
| τ-benchRetail and airline, released with the paper. | Jun 17, 20242 years ago | Retail and airline, released with the paper. |
Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.