Skip to content

Benchmark

Deep20Bench success rate

Scores are percentages. Higher is better.

Models measured
28
Scores recorded
28
Sources
1
IQ weight
3.27

Score distribution

90.7%
Mean
96.7%
Median
12.2%
Std dev
73.3%
10th percentile
100%
90th percentile
50%
Min
50%100%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 3.27 for chat models, measured over 28 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Fable 5.2qualified-core-v1, anthropic, BX-20261109-official-M0038-001· Deep20Bench100%
2Gemini 3.8 Flashqualified-core-v1, google-ai-studio, BX-20260908-B-0003-experimental-M0021-001· Deep20Bench100%
3Gemini 4 Argonqualified-core-v1, google-vertex, BX-20261014-B-0003-official-M0034-002· Deep20Bench100%
4Gemini 4 Hydrogenqualified-core-v1, google-vertex, BX-20261113-B-0003-official-M0040-001· Deep20Bench100%
5Gemini 4 Neonqualified-core-v1, google-vertex, BX-20261120-B-0003-official-M0042-001· Deep20Bench100%
6GPT-6 Astraqualified-core-v1, openai, BX-20260909-B-0003-experimental-M0022-002· Deep20Bench100%
7GPT-6 Solqualified-core-v1, openai, BX-20260926-B-0003-official-M0029-001· Deep20Bench100%
8GPT-6.2 Astraqualified-core-v1, openai, BX-20261121-B-0003-official-M0043-001· Deep20Bench100%
9GPT-6.2 Solqualified-core-v1, openai, BX-20261117-B-0003-official-M0041-001· Deep20Bench100%
10Grok 4.6qualified-core-v1, xai, BX-20260908-B-0003-experimental-M0015-001· Deep20Bench100%
11Claude Opus 5.5qualified-core-v1, anthropic, BX-20261103-B-0003-official-M0030-001· Deep20Bench96.7%
12GPT-5.6 Solqualified-core-v1, openai, BX-20260908-B-0003-experimental-M0010-001· Deep20Bench96.7%
13GPT-6.1 Solqualified-core-v1, openai, BX-20260929-B-0003-official-M0031-001· Deep20Bench96.7%
14Muse Spark 1.3 Contributorqualified-core-v1, meta, BX-20260909-B-0003-experimental-M0026-003· Deep20Bench96.7%
15Muse Wonderqualified-core-v1, meta, BX-20261013-B-0003-OFFICIAL-M0033-002· Deep20Bench96.7%
16Claude Fable 5.1qualified-core-v1, anthropic, BX-20260908-B-0003-experimental-M0020-001· Deep20Bench93.3%
17Claude Opus 5qualified-core-v1, anthropic, BX-20260907-B-0003-experimental-M0006-002· Deep20Bench93.3%
18Gemini 3.7 Flashqualified-core-v1, google-ai-studio, BX-20260908-B-0003-experimental-M0016-001· Deep20Bench93.3%
19GPT-5.6 Lunaqualified-core-v1, openai, BX-20260908-B-0003-experimental-M0001-001· Deep20Bench90%
20GPT-6.1 Terraqualified-core-v1, openai, BX-20261027-B-0003-official-M0036-001· Deep20Bench90%
21Claude Sonnet 5.5qualified-core-v1, anthropic, BX-20261103-B-0003-official-M0032-001· Deep20Bench86.7%
22Claude Haiku 5.5qualified-core-v1, anthropic, BX-20261023-B-0003-official-M0035-001· Deep20Bench83.3%
23DeepSeek V4.1 Flash 1017qualified-core-v1, deepinfra, BX-20261028-B-0003-official-M0037-001· Deep20Bench83.3%
24GPT-6 Lunaqualified-core-v1, openai, BX-20260926-B-0003-official-M0028-001· Deep20Bench80%
25GLM 5.3 Flashqualified-core-v1, z-ai, BX-20260908-B-0003-experimental-M0023-001· Deep20Bench73.3%
26Grok 4.7qualified-core-v1, xai, BX-20260926-B-0003-official-M0027-001· Deep20Bench73.3%
27GPT OSS 120Bqualified-core-v1, cerebras, BX-20260908-B-0003-experimental-M0002-001· Deep20Bench66.7%
28MiniMax M3qualified-core-v1, coreweave, BX-20260908-B-0003-experimental-M0024-001· Deep20Bench50%

Data credits: Deep20Bench edition 1.1, Patrick Heusser. Results CC BY 4.0; native question scores and fraction-to-percent success rates. (licence)