Skip to content

Benchmark

GPQA Diamond

Science benchmarks measure how well models answer science questions. Scores are percentages. Higher is better.

Models and systems measured
282
Scores recorded
1623
Sources
4
IQ weight
1.72

Score distribution

79.3%
Mean
84.5%
Median
15.7%
Std dev
56.8%
10th percentile
92.2%
90th percentile
0.168%
Min
0.168%96.8%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.72 for chat models, measured over 272 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model or system
1551GPT-4o miniOpenRouter harness, openai/gpt-4o-mini, Azure, public 90-day aggregate· OpenRouter benchmark exports43.1%
1552Qwen2.5 VL 72B InstructOpenRouter harness, qwen/qwen2.5-vl-72b-instruct, auto-routing, public 90-day aggregate· OpenRouter benchmark exports42.9%
1553GPT-4o miniOpenRouter harness, openai/gpt-4o-mini, OpenAI, public 90-day aggregate· OpenRouter benchmark exports42.8%
1554GPT 4 1106 Preview· Epoch AI Benchmarking Hub data42.4%
1555GPT-4-0125-preview· Epoch AI Benchmarking Hub data42.3%
1556Mistral Small 3.1· Epoch AI Benchmarking Hub data41.9%
1557Qwen Turbo· Epoch AI Benchmarking Hub data41.8%
1558Llama 3.2 90B Vision Instruct· Epoch AI Benchmarking Hub data41%
1559Qwen2 72B Instruct· Epoch AI Benchmarking Hub data40.8%
1560Claude 3 Sonnet· Epoch AI Benchmarking Hub data40.6%
1561Gemma 3 12B ITOpenRouter harness, google/gemma-3-12b-it, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports40.6%
1562Llama 3 70B Instruct· Epoch AI Benchmarking Hub data40.6%
1563gemini-1.5-flash-001· Epoch AI Benchmarking Hub data40.4%
1564Gemma 3 12B IT· Epoch AI Benchmarking Hub data39.5%
1565Mistral Large2402· Epoch AI Benchmarking Hub data38.8%
1566Gemma 4 26B A4BOpenRouter harness, google/gemma-4-26b-a4b-it-20260403, Venice, public 90-day aggregate· OpenRouter benchmark exports38.6%
1567Nova MicroOpenRouter harness, amazon/nova-micro-v1, Amazon Bedrock, public 90-day aggregate· OpenRouter benchmark exports38.6%
1568Claude Haiku 3.5· Epoch AI Benchmarking Hub data38.1%
1569Gemma 3 12B ITOpenRouter harness, google/gemma-3-12b-it, NextBit, public 90-day aggregate· OpenRouter benchmark exports38%
1570Qwen3 1.7B· Epoch AI Benchmarking Hub data38%
1571Gemma 4 31B ITOpenRouter harness, google/gemma-4-31b-it-20260402, Io Net, public 90-day aggregate· OpenRouter benchmark exports37.9%
1572GPT-4o mini· Epoch AI Benchmarking Hub data37.7%
1573Gemma 2 27B IT· Epoch AI Benchmarking Hub data36.5%
1574Claude 3 Haiku· Epoch AI Benchmarking Hub data36.3%
1575GPT 4· Epoch AI Benchmarking Hub data35.7%
1576Qwen2.5 7B Instruct· Epoch AI Benchmarking Hub data35.5%
1577DeepSeek V4 FlashOpenRouter harness, deepseek/deepseek-v4-flash-20260423, Reka, public 90-day aggregate· OpenRouter benchmark exports34.7%
1578Claude 2· Epoch AI Benchmarking Hub data34.7%
1579DeepSeek R1 Distill Qwen 1.5B· Epoch AI Benchmarking Hub data33.6%
1580Claude 2.1· Epoch AI Benchmarking Hub data33%
1581gemini-1.5-flash-8b-001· Epoch AI Benchmarking Hub data33%
1582DBRX-instruct· Epoch AI Benchmarking Hub data32.9%
1583Yi 1.5 34B Chat· Epoch AI Benchmarking Hub data32%
1584Qwen1.5-32B-Chat· Epoch AI Benchmarking Hub data30.7%
1585Qwen3 1.7Bnone· Epoch AI Benchmarking Hub data30.6%
1586Mixtral 8x7B Instruct v0.1· Epoch AI Benchmarking Hub data30.6%
1587Llama 3.1 8B Instruct· OpenEvals Leaderboard data· Reported by the model's developer. The source relaying it did not verify it.
Values from 2 sources
30.4%
1588Open Mistral Nemo· Epoch AI Benchmarking Hub data29.9%
1589open-mixtral-8x7b· Epoch AI Benchmarking Hub data29.8%
1590Llama 3.1 8B InstructOpenRouter harness, meta-llama/llama-3.1-8b-instruct, Groq, public 90-day aggregate· OpenRouter benchmark exports29.2%
1591Qwen1.5-72B-Chat· Epoch AI Benchmarking Hub data28.8%
1592Llama 3.2 3BOpenRouter harness, meta-llama/llama-3.2-3b-instruct, Cloudflare, public 90-day aggregate· OpenRouter benchmark exports28.6%
1593GPT-3.5 Turbo· Epoch AI Benchmarking Hub data28%
1594Phi-3 Medium 128K Instruct· Epoch AI Benchmarking Hub data27.6%
1595Gemma 2 9B IT· Epoch AI Benchmarking Hub data27.5%
1596Ministral 8B· Epoch AI Benchmarking Hub data27.1%
1597Gemma 3 4B ITOpenRouter harness, google/gemma-3-4b-it, Parasail, public 90-day aggregate· OpenRouter benchmark exports27.1%
1598Llama 3.1 8B InstructOpenRouter harness, meta-llama/llama-3.1-8b-instruct, auto-routing, public 90-day aggregate· OpenRouter benchmark exports27.1%
1599Llama 3.1 8B InstructOpenRouter harness, meta-llama/llama-3.1-8b-instruct, CoreWeave, public 90-day aggregate· OpenRouter benchmark exports26.7%
1600Llama-2-70b-chat-hf· Epoch AI Benchmarking Hub data26.3%

Showing 1,551–1,600 of 1,621 results.

Not ranked

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or system
MiMo V2 Flash Baseofficial Base table · 5-shot· Reported by Xiaomi MiMo· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.55.1%
Qwen3.8-2.4T-A95Blifecycle-page core-benchmark claim· Reported by Alibaba Cloud Model Studio· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.92.6%

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · OpenRouter public benchmark leaderboard and search-lane exports, CC BY 4.0. Fractional source scores converted to percentages; immutable dated snapshots. (licence) · OpenEvals/leaderboard-data, MIT (licence)