Skip to content

Benchmark

Deep20Bench question score

Lower scores are better.

Models measured
28
Scores recorded
28
Sources
1
IQ weight
1.53

Score distribution

18.1
Mean
18.1
Median
3.6
Std dev
14.6
10th percentile
22.2
90th percentile
13.7
Min
13.729.2
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.53 for chat models, measured over 28 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 4 Argonqualified-core-v1, google-vertex, BX-20261014-B-0003-official-M0034-002· Deep20Bench13.7
2GPT-6 Astraqualified-core-v1, openai, BX-20260909-B-0003-experimental-M0022-002· Deep20Bench13.7
3Gemini 4 Neonqualified-core-v1, google-vertex, BX-20261120-B-0003-official-M0042-001· Deep20Bench13.9
4Claude Fable 5.2qualified-core-v1, anthropic, BX-20261109-official-M0038-001· Deep20Bench14.6
5GPT-6.2 Astraqualified-core-v1, openai, BX-20261121-B-0003-official-M0043-001· Deep20Bench14.6
6GPT-6.2 Solqualified-core-v1, openai, BX-20261117-B-0003-official-M0041-001· Deep20Bench14.7
7Gemini 3.8 Flashqualified-core-v1, google-ai-studio, BX-20260908-B-0003-experimental-M0021-001· Deep20Bench14.9
8Gemini 4 Hydrogenqualified-core-v1, google-vertex, BX-20261113-B-0003-official-M0040-001· Deep20Bench14.9
9Claude Fable 5.1qualified-core-v1, anthropic, BX-20260908-B-0003-experimental-M0020-001· Deep20Bench15.1
10GPT-6.1 Solqualified-core-v1, openai, BX-20260929-B-0003-official-M0031-001· Deep20Bench15.2
11Claude Opus 5.5qualified-core-v1, anthropic, BX-20261103-B-0003-official-M0030-001· Deep20Bench16.4
12Gemini 3.7 Flashqualified-core-v1, google-ai-studio, BX-20260908-B-0003-experimental-M0016-001· Deep20Bench16.6
13Grok 4.6qualified-core-v1, xai, BX-20260908-B-0003-experimental-M0015-001· Deep20Bench17.1
14GPT-6 Solqualified-core-v1, openai, BX-20260926-B-0003-official-M0029-001· Deep20Bench18
15Claude Opus 5qualified-core-v1, anthropic, BX-20260907-B-0003-experimental-M0006-002· Deep20Bench18.1
16GPT-5.6 Solqualified-core-v1, openai, BX-20260908-B-0003-experimental-M0010-001· Deep20Bench18.1
17Claude Sonnet 5.5qualified-core-v1, anthropic, BX-20261103-B-0003-official-M0032-001· Deep20Bench18.2
18Muse Spark 1.3 Contributorqualified-core-v1, meta, BX-20260909-B-0003-experimental-M0026-003· Deep20Bench18.4
19GPT-6.1 Terraqualified-core-v1, openai, BX-20261027-B-0003-official-M0036-001· Deep20Bench18.8
20Muse Wonderqualified-core-v1, meta, BX-20261013-B-0003-OFFICIAL-M0033-002· Deep20Bench19.6
21Claude Haiku 5.5qualified-core-v1, anthropic, BX-20261023-B-0003-official-M0035-001· Deep20Bench20
22GPT-5.6 Lunaqualified-core-v1, openai, BX-20260908-B-0003-experimental-M0001-001· Deep20Bench21
23GPT-6 Lunaqualified-core-v1, openai, BX-20260926-B-0003-official-M0028-001· Deep20Bench21.2
24GPT OSS 120Bqualified-core-v1, cerebras, BX-20260908-B-0003-experimental-M0002-001· Deep20Bench22.2
25Grok 4.7qualified-core-v1, xai, BX-20260926-B-0003-official-M0027-001· Deep20Bench22.2
26DeepSeek V4.1 Flash 1017qualified-core-v1, deepinfra, BX-20261028-B-0003-official-M0037-001· Deep20Bench22.7
27GLM 5.3 Flashqualified-core-v1, z-ai, BX-20260908-B-0003-experimental-M0023-001· Deep20Bench23.3
28MiniMax M3qualified-core-v1, coreweave, BX-20260908-B-0003-experimental-M0024-001· Deep20Bench29.2

Data credits: Deep20Bench edition 1.1, Patrick Heusser. Results CC BY 4.0; native question scores and fraction-to-percent success rates. (licence)