IFEval what it measures, how it is scored and who leads it

Name: IFEval
Creator: Google
Published: 2023-11-14

Chat and instructions · Google · introduced Nov 14, 2023 · Instruction-level accuracy

About this benchmark

Task: Follow verifiable formatting instructions such as "write more than 400 words".
Dataset: About 500 prompts using 25 types of verifiable instruction.
Method: Each instruction is checked by a program, no model judge.
Metric: Instruction-level accuracy
Organization: Google
Introduced: Nov 14, 2023
Paper: Instruction-Following Evaluation for Large Language Models (arXiv 2311.07911)
Home: github.com/google-research/google-research/tree/master/instruction_following_eval

Newest first. New versions are added, never rewritten.

Version	Date	What changed
1Released with the paper.	Nov 14, 20232 years ago	Released with the paper.

Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.