Skip to content

Benchmark

SWE-bench Verified (Epoch)

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
31
Scores recorded
32
Sources
1
IQ weight
3.61

Score distribution

70.7%
Mean
73.8%
Median
10.2%
Std dev
57.9%
10th percentile
78.7%
90th percentile
31%
Min
31%83.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 3.61 for chat models, measured over 31 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.7max· Epoch AI Benchmarking Hub data83.5%
2Gemini 3.5 Flashhigh· Epoch AI Benchmarking Hub data79.3%
3Claude Opus 4.6· Epoch AI Benchmarking Hub data78.7%
4GLM 5.2max· Epoch AI Benchmarking Hub data78.7%
5DeepSeek V4 Promax· Epoch AI Benchmarking Hub data77.6%
6Qwen3.7 Max· Epoch AI Benchmarking Hub data77.3%
7GPT 5.4high· Epoch AI Benchmarking Hub data76.9%
8Claude Opus 4.5· Epoch AI Benchmarking Hub data76.7%
9Kimi K2.6· Epoch AI Benchmarking Hub data76.7%
10Qwen3.6 Max Preview· Epoch AI Benchmarking Hub data76.7%
11Gemini 3.1 Pro Previewcustomtools· Epoch AI Benchmarking Hub data75.6%
12Gemini 3 Flash· Epoch AI Benchmarking Hub data75.4%
13Claude Sonnet 4.6· Epoch AI Benchmarking Hub data75.2%
14GPT-5.3 Codexhigh· Epoch AI Benchmarking Hub data74.8%
15GLM 5.1· Epoch AI Benchmarking Hub data74.2%
16GPT 5.2high· Epoch AI Benchmarking Hub data73.8%
17Kimi K2.5· Epoch AI Benchmarking Hub data73.8%
18GPT 5high· Epoch AI Benchmarking Hub data73.6%
19Claude Opus 4.1· Epoch AI Benchmarking Hub data73.3%
20Gemini 3 Pro Preview· Epoch AI Benchmarking Hub data72.9%
21GLM 5· Epoch AI Benchmarking Hub data72.1%
22GPT 5medium· Epoch AI Benchmarking Hub data71.5%
23Claude Sonnet 4.5· Epoch AI Benchmarking Hub data71.3%
24Claude Opus 4· Epoch AI Benchmarking Hub data70.7%
25GPT 5.1high· Epoch AI Benchmarking Hub data68%
26GPT-5 minimedium· Epoch AI Benchmarking Hub data64.7%
27OpenAI o3medium· Epoch AI Benchmarking Hub data62.3%
28Claude Sonnet 3.7· Epoch AI Benchmarking Hub data61%
29Qwen3.6 Plus· Epoch AI Benchmarking Hub data57.9%
30Gemini 2.5 Pro· Epoch AI Benchmarking Hub data57.6%
31GPT 4.1· Epoch AI Benchmarking Hub data48.5%
32GPT 4o· Epoch AI Benchmarking Hub data31%

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)