Skip to content

Benchmark

SWE-rebench 2026-02

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
26
Scores recorded
27
Sources
1
IQ weight
2.59

Score distribution

54.4%
Mean
58.2%
Median
8.6%
Std dev
41.6%
10th percentile
62.3%
90th percentile
33.3%
Min
33.3%65.3%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.59 for chat models, measured over 26 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
1Claude Opus 4.6tools, high· SWE-rebench65.3%
2GLM 5tools· SWE-rebench62.8%
3GLM 5.1tools· SWE-rebench62.7%
4Gemini 3.1 Pro Previewtools· SWE-rebench62.3%
5DeepSeek V3.2tools· SWE-rebench60.9%
6Claude Sonnet 4.6tools· SWE-rebench60.7%
7Claude Sonnet 4.5tools· SWE-rebench60%
8Qwen3.5 397B A17Btools· SWE-rebench59.9%
9Step 3.5 Flashtools· SWE-rebench59.6%
10Qwen3.5 27Btools· SWE-rebench58.9%
11GLM 4.7tools· SWE-rebench58.7%
12GPT-5.3 Codextools, xhigh· SWE-rebench58.6%
13Kimi K2.5tools· SWE-rebench58.5%
14GPT-5.3 Codextools· SWE-rebench58.2%
15Kimi K2 Thinkingtools· SWE-rebench57.4%
16GPT-5.2 Codextools· SWE-rebench56.8%
17MiniMax M2.5tools· SWE-rebench54.6%
18Qwen3 Coder Nexttools· SWE-rebench54.4%
19Qwen3.5 35B A3Btools· SWE-rebench53.7%
20Gemini 3 Flashtools· SWE-rebench52.5%
21MiniMax M2.7tools· SWE-rebench51.9%
22Devstral 2 123b Instruct 2512tools· SWE-rebench48.8%
23Qwen3 Codertools· SWE-rebench44.7%
24Gemma 4 31B ITtools· SWE-rebench41.6%
25GLM 4.5 Airtools· SWE-rebench38.3%
26GLM 4.7 Flashtools· SWE-rebench34%
27GPT OSS 120Btools· SWE-rebench33.3%

Tokens per task, input and output together, as measured by the source. Comparable within this benchmark only.