Skip to content

Benchmark

Deepswe Pass@1

Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.

Models measured
26
Scores recorded
69
Sources
1
IQ weight
2.17

Score distribution

54.9%
Mean
58.1%
Median
16.7%
Std dev
30.5%
10th percentile
72.8%
90th percentile
1.5%
Min
1.5%74.1%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.17 for chat models, measured over 26 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
1GPT-6 Astraxhigh· Epoch AI Benchmarking Hub data74.1%
2Gemini 3.8 Flashhigh· Epoch AI Benchmarking Hub data73.8%
3Claude Opus 5max· Epoch AI Benchmarking Hub data73.6%
4GPT-6 Astrahigh· Epoch AI Benchmarking Hub data73.2%
5GPT-6 Astramax· Epoch AI Benchmarking Hub data73.2%
6Claude Opus 5xhigh· Epoch AI Benchmarking Hub data73.2%
7Claude Opus 5high· Epoch AI Benchmarking Hub data72.8%
8GPT-6 Astramedium· Epoch AI Benchmarking Hub data72.8%
9GPT-5.6 Solmax· Epoch AI Benchmarking Hub data72.7%
10Gemini 3.8 Flashmedium· Epoch AI Benchmarking Hub data71%
11GPT-5.6 Solxhigh· Epoch AI Benchmarking Hub data70.7%
12Claude Fable 5xhigh· Epoch AI Benchmarking Hub data69.9%
13Claude Fable 5max· Epoch AI Benchmarking Hub data69.7%
14GPT-5.6 Terramax· Epoch AI Benchmarking Hub data69.6%
15GPT-5.6 Solhigh· Epoch AI Benchmarking Hub data69.4%
16GLM 5.3max· Epoch AI Benchmarking Hub data69%
17Claude Opus 5medium· Epoch AI Benchmarking Hub data68.9%
18Claude Fable 5high· Epoch AI Benchmarking Hub data68.6%
19Kimi K3max· Epoch AI Benchmarking Hub data68.5%
20Grok 4.6medium· Epoch AI Benchmarking Hub data67.5%
21GPT-5.6 Lunamax· Epoch AI Benchmarking Hub data67.2%
22GPT 5.5xhigh· Epoch AI Benchmarking Hub data67%
23GPT-6 Astralow· Epoch AI Benchmarking Hub data67%
24Grok 4.6xhigh· Epoch AI Benchmarking Hub data66.7%
25Gemini 3.7 Flashmedium· Epoch AI Benchmarking Hub data65.5%
26Claude Fable 5medium· Epoch AI Benchmarking Hub data65.4%
27Gemini 3.7 Flashhigh· Epoch AI Benchmarking Hub data65.3%
28Grok 4.6high· Epoch AI Benchmarking Hub data65.2%
29GPT 5.5high· Epoch AI Benchmarking Hub data64.4%
30GLM 5.3 Flashmax· Epoch AI Benchmarking Hub data63.4%
31GPT-5.6 Solmedium· Epoch AI Benchmarking Hub data61.1%
32GPT-5.6 Terraxhigh· Epoch AI Benchmarking Hub data60.2%
33Claude Fable 5low· Epoch AI Benchmarking Hub data59.6%
34Claude Opus 4.8max· Epoch AI Benchmarking Hub data59%
35Claude Opus 5low· Epoch AI Benchmarking Hub data58.1%
36Qwen3.8 Maxxhigh· Epoch AI Benchmarking Hub data57.5%
37GPT-5.6 Lunaxhigh· Epoch AI Benchmarking Hub data56.9%
38Muse Spark 1.2xhigh· Epoch AI Benchmarking Hub data54.9%
39Claude Opus 4.8xhigh· Epoch AI Benchmarking Hub data54.4%
40GPT 5.5medium· Epoch AI Benchmarking Hub data54%
41Claude Sonnet 5max· Epoch AI Benchmarking Hub data53.8%
42Gemini 3.7 Flashlow· Epoch AI Benchmarking Hub data53.8%
43GPT-5.6 Terrahigh· Epoch AI Benchmarking Hub data53.8%
44Grok 4.5high· Epoch AI Benchmarking Hub data53.8%
45Muse Spark 1.1· Epoch AI Benchmarking Hub data53.3%
46Claude Opus 4.8high· Epoch AI Benchmarking Hub data51.8%
47GPT 5.4xhigh· Epoch AI Benchmarking Hub data51.8%
48Claude Sonnet 5xhigh· Epoch AI Benchmarking Hub data49.7%
49Claude Opus 4.8medium· Epoch AI Benchmarking Hub data48.7%
50Claude Sonnet 5high· Epoch AI Benchmarking Hub data48.2%

Showing 1–50 of 69 results.

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)