Skip to content

Benchmark

SWE-rebench 2026-01

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
30
Scores recorded
30
Sources
1
IQ weight
2.47

Score distribution

36%
Mean
37.9%
Median
10.9%
Std dev
21.7%
10th percentile
47.1%
90th percentile
7.1%
Min
7.1%51.7%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.47 for chat models, measured over 30 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
1Claude Opus 4.6tools, high· SWE-rebench51.7%
2GPT 5.2tools, xhigh· SWE-rebench51.7%
3GPT-5.1 Codex Maxtools· SWE-rebench48.5%
4Claude Sonnet 4.5tools· SWE-rebench47.1%
5Gemini 3 Flashtools· SWE-rebench46.7%
6Gemini 3 Pro Previewtools· SWE-rebench46.7%
7GPT-5.2 Codextools· SWE-rebench45%
8Claude Opus 4.5tools· SWE-rebench43.8%
9Kimi K2 Thinkingtools· SWE-rebench43.8%
10GPT-5.1 Codextools· SWE-rebench42.9%
11GLM 5tools· SWE-rebench42.1%
12GLM 4.7tools· SWE-rebench41.3%
13Qwen3 Coder Nexttools· SWE-rebench40%
14MiniMax M2.5tools· SWE-rebench39.6%
15Kimi K2.5tools· SWE-rebench37.9%
16Devstral 2 123b Instruct 2512tools· SWE-rebench37.5%
17DeepSeek V3.2tools· SWE-rebench37.5%
18GLM 4.6tools· SWE-rebench37.1%
19GPT-5 minitools, high· SWE-rebench35%
20Kimi K2 Instructtools· SWE-rebench34.3%
21GLM 4.5 Airtools· SWE-rebench31.8%
22MiniMax M2.1tools· SWE-rebench31.7%
23Qwen3 Codertools· SWE-rebench31.7%
24GLM 4.7 Flashtools· SWE-rebench25.4%
25GPT OSS 120Btools· SWE-rebench24.6%
26Qwen3 235B A22B Instruct 2507tools· SWE-rebench23.8%
27DeepSeek R1tools· SWE-rebench21.7%
28Qwen3 Coder 30B A3B Instructtools· SWE-rebench18%
29Qwen3 Next 80B A3B Instructtools· SWE-rebench15.4%
30Qwen3 30B A3B Instructtools· SWE-rebench7.1%

Tokens per task, input and output together, as measured by the source. Comparable within this benchmark only.