Skip to content

Benchmark

SWE-rebench 2026-06

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
8
Scores recorded
8
Sources
1
IQ weight
0.94

Score distribution

35.4%
Mean
35.2%
Median
15.3%
Std dev
20%
10th percentile
55.7%
90th percentile
13.8%
Min
13.8%58.1%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.94 for chat models, measured over 8 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
1GLM 5.2tools, high· SWE-rebench58.1%
2Grok 4.5tools, high· SWE-rebench55.7%
3MiniMax M3tools· SWE-rebench43.8%
4DeepSeek V4 Protools, high· SWE-rebench35.2%
5MiMo V2.5 Protools· SWE-rebench33.8%
6Qwen3.6 27Btools· SWE-rebench22.9%
7Qwen3.6 35B A3Btools· SWE-rebench20%
8Qwen3.5 35B A3Btools· SWE-rebench13.8%

Tokens per task, input and output together, as measured by the source. Comparable within this benchmark only.