Skip to content

Benchmark

Deepswe Pass@4

Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.

Models measured
26
Scores recorded
69
Sources
1
IQ weight
1.95

Score distribution

76%
Mean
80.5%
Median
15.5%
Std dev
60.2%
10th percentile
88.5%
90th percentile
4.4%
Min
4.4%90.3%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.95 for chat models, measured over 26 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
1GPT 5.5high· Epoch AI Benchmarking Hub data90.3%
2GPT-5.6 Lunamax· Epoch AI Benchmarking Hub data90.3%
3Claude Opus 5medium· Epoch AI Benchmarking Hub data89.4%
4Kimi K3max· Epoch AI Benchmarking Hub data89.4%
5Claude Fable 5xhigh· Epoch AI Benchmarking Hub data88.5%
6Claude Opus 5max· Epoch AI Benchmarking Hub data88.5%
7GPT 5.5xhigh· Epoch AI Benchmarking Hub data88.5%
8GPT-5.6 Terramax· Epoch AI Benchmarking Hub data88.5%
9Claude Opus 5high· Epoch AI Benchmarking Hub data87.6%
10GLM 5.3max· Epoch AI Benchmarking Hub data87.6%
11Claude Fable 5high· Epoch AI Benchmarking Hub data86.7%
12GPT-5.6 Solhigh· Epoch AI Benchmarking Hub data86.7%
13Claude Opus 5xhigh· Epoch AI Benchmarking Hub data85.8%
14Gemini 3.8 Flashhigh· Epoch AI Benchmarking Hub data85.8%
15GPT-5.6 Solmax· Epoch AI Benchmarking Hub data85.8%
16GPT-5.6 Solxhigh· Epoch AI Benchmarking Hub data85.8%
17Claude Opus 5low· Epoch AI Benchmarking Hub data85%
18GLM 5.3 Flashmax· Epoch AI Benchmarking Hub data85%
19Grok 4.6high· Epoch AI Benchmarking Hub data85%
20Grok 4.6xhigh· Epoch AI Benchmarking Hub data85%
21Claude Fable 5max· Epoch AI Benchmarking Hub data84.1%
22Grok 4.6medium· Epoch AI Benchmarking Hub data84.1%
23Claude Fable 5medium· Epoch AI Benchmarking Hub data83.2%
24Gemini 3.7 Flashmedium· Epoch AI Benchmarking Hub data83.2%
25Gemini 3.8 Flashmedium· Epoch AI Benchmarking Hub data83.2%
26Qwen3.8 Maxxhigh· Epoch AI Benchmarking Hub data83.2%
27Gemini 3.7 Flashhigh· Epoch AI Benchmarking Hub data82.3%
28GPT-6 Astrahigh· Epoch AI Benchmarking Hub data82.3%
29GPT-6 Astramedium· Epoch AI Benchmarking Hub data82.3%
30Claude Fable 5low· Epoch AI Benchmarking Hub data81.4%
31Muse Spark 1.2xhigh· Epoch AI Benchmarking Hub data81.4%
32Claude Opus 4.8xhigh· Epoch AI Benchmarking Hub data80.5%
33GPT-5.6 Solmedium· Epoch AI Benchmarking Hub data80.5%
34GPT-5.6 Terrahigh· Epoch AI Benchmarking Hub data80.5%
35GPT-5.6 Terraxhigh· Epoch AI Benchmarking Hub data80.5%
36GPT-6 Astraxhigh· Epoch AI Benchmarking Hub data80.5%
37Claude Sonnet 5high· Epoch AI Benchmarking Hub data79.6%
38GPT-6 Astralow· Epoch AI Benchmarking Hub data79.6%
39GPT-6 Astramax· Epoch AI Benchmarking Hub data79.6%
40Muse Spark 1.1· Epoch AI Benchmarking Hub data79.6%
41Claude Opus 4.8max· Epoch AI Benchmarking Hub data79.3%
42Claude Sonnet 5max· Epoch AI Benchmarking Hub data78.8%
43GPT-5.6 Lunaxhigh· Epoch AI Benchmarking Hub data78.8%
44Claude Opus 4.8high· Epoch AI Benchmarking Hub data77.9%
45Gemini 3.7 Flashlow· Epoch AI Benchmarking Hub data77.9%
46GPT 5.4xhigh· Epoch AI Benchmarking Hub data77.9%
47GPT 5.5medium· Epoch AI Benchmarking Hub data77.9%
48Grok 4.5high· Epoch AI Benchmarking Hub data77.9%
49GLM 5.2max· Epoch AI Benchmarking Hub data77%
50Claude Opus 4.8medium· Epoch AI Benchmarking Hub data76.1%

Showing 1–50 of 69 results.

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)