GPQA (Diamond) what it measures, how it is scored and who leads it

Name: GPQA (Diamond)
Creator: New York University
Published: 2023-11-20

Knowledge · New York University · introduced Nov 20, 2023 · Accuracy

Leaderboard Paper

Scores from GPQA Diamond (Epoch AI runs), published Nov 6, 2026

Rank	Model	Accuracy
1	Gemini 4 CarbonGoogle · as “gemini-4-carbon_high”	96.5%
2	Gemini 4 ArgonGoogle · as “gemini-4-argon_high”	96.4%
3	GPT-6 AstraOpenAI · as “gpt-6-astra_max”	95.8%
4	Claude Sonnet 5.5Anthropic · as “claude-sonnet-5-5_max”	95.6%
5	Gemini 3.8 FlashGoogle · as “gemini-3.8-flash_high”	95.4%
5	GPT-6.1 SolOpenAI · as “gpt-6.1-sol_max”	95.4%
7	Gemini 3.7 FlashGoogle · as “gemini-3.7-flash_high”	94.8%
8	GPT-5.4 ProOpenAI · as “gpt-5.4-pro-2026-03-05_xhigh”	94.6%
9	Gemini 3.1 Pro PreviewGoogle · as “gemini-3.1-pro-preview_high”	94.4%
10	GPT-6.1 TerraOpenAI · as “gpt-6.1-terra_max”	94.3%
10	GPT-6 SolOpenAI · as “gpt-6-sol_max”	94.3%
12	Kimi K3.1 Pro TogetherMoonshot AI · as “kimi-k3.1-pro-together_max”	94.2%
13	Gemini 3.6 FlashGoogle · as “gemini-3.6-flash_high”	94.1%
14	GPT-5.5 Pre ReleaseOpenAI · as “gpt-5.5-pre-release_xhigh”	94.0%
14	Grok 4.6SpaceXAI · as “grok-4.6_high”	94.0%
16	GPT-5.5 Pro Pre ReleaseOpenAI · as “gpt-5.5-pro-pre-release_xhigh”	93.9%
17	Claude Fable 5.2Anthropic · as “claude-fable-5-2_max”	93.9%
17	Claude Opus 5Anthropic · as “claude-opus-5_max”	93.9%
19	GPT-5.6 SolOpenAI · as “gpt-5.6-sol_max”	93.5%
20	Grok 4.5SpaceXAI · as “grok-4.5_high”	93.4%
21	GPT-5.6 TerraOpenAI · as “gpt-5.6-terra_max”	93.3%
22	GPT-5.4OpenAI · as “gpt-5.4-2026-03-05_xhigh”	93.3%
23	DeepSeek V4.1 FlashDeepSeek · as “deepseek-v4.1-flash-1017_max”	93.2%
24	Doubao Seed EvolvingByteDance · as “doubao-seed-evolving_high”	93.1%
24	Kimi K3Moonshot AI · as “kimi-k3_max”	93.1%

About this benchmark

Task: Answer graduate-level biology, physics and chemistry questions that non-experts cannot look up.
Dataset: 448 multiple-choice questions written by domain experts; the Diamond subset keeps the 198 hardest and cleanest.
Method: Four-option multiple choice. Experts with PhDs reached 65%, skilled non-experts with web access 34%.
Metric: Accuracy
Organization: New York University
Introduced: Nov 20, 2023
Paper: GPQA: A Graduate-Level Google-Proof Q&A Benchmark (arXiv 2311.12022)
Official leaderboard: epoch.ai/benchmarks/gpqa-diamond
Home: github.com/idavidrein/gpqa

Versions

Newest first. New versions are added, never rewritten.

Version	Date	What changed
1Main set of 448 questions and the 198-question Diamond subset.	Nov 20, 20232 years ago	Main set of 448 questions and the 198-question Diamond subset.

Sources: each benchmark's paper (arXiv) and its official site or leaderboard. Scores are the boards captured here every day, exactly as published.