Skip to content

Benchmark

OpenHands Index: swe-bench

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
31
Scores recorded
31
Sources
1
IQ weight
3.46

Score distribution

73.4%
Mean
74.2%
Median
7.1%
Std dev
62.4%
10th percentile
78.6%
90th percentile
56.8%
Min
56.8%95.8%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 3.46 for chat models, measured over 31 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
1Claude Fable 5· OpenHands Index data95.8%
2Claude Opus 4.8· OpenHands Index data83.8%
3Claude Opus 4.7· OpenHands Index data81.6%
4Gemini 3.5 Flash· OpenHands Index data78.6%
5GPT 5.5· OpenHands Index data78.2%
6Claude Opus 4.6· OpenHands Index data76.8%
7Gemini 3.1 Pro Preview· OpenHands Index data76.8%
8Claude Opus 4.5· OpenHands Index data76.6%
9MiniMax M3· OpenHands Index data76.4%
10GPT 5.4· OpenHands Index data75.6%
11GLM 5.1· OpenHands Index data75%
12Gemini 3 Flash· OpenHands Index data74.6%
13GPT 5.2· OpenHands Index data74.6%
14Kimi K2.6· OpenHands Index data74.6%
15Claude Sonnet 4.6· OpenHands Index data74.4%
16Claude Sonnet 4.5· OpenHands Index data74.2%
17Qwen3.6 Plus· OpenHands Index data74.2%
18GPT-5.2 Codex· OpenHands Index data73.8%
19GLM 4.7· OpenHands Index data73.4%
20GLM 5· OpenHands Index data73.4%
21DeepSeek V4 Pro· OpenHands Index data73.2%
22MiniMax M2.5· OpenHands Index data72.6%
23Gemini 3 Pro Preview· OpenHands Index data70.6%
24Kimi K2 Thinking· OpenHands Index data69.2%
25Kimi K2.5· OpenHands Index data68.8%
26MiniMax M2.1· OpenHands Index data68.8%
27Qwen3 Coder Next· OpenHands Index data66.6%
28Qwen3 Coder· OpenHands Index data62.4%
29Nemotron 3 Super· OpenHands Index data62%
30Qwen3.5 Flash· OpenHands Index data62%
31Trinity Large Thinking· OpenHands Index data56.8%

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: OpenHands Index (All Hands AI), Apache-2.0 (licence)