Skip to content

Benchmark

OpenHands Index: swe-bench-multimodal

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
31
Scores recorded
31
Sources
1
IQ weight
2.27

Score distribution

34.9%
Mean
35.9%
Median
10.4%
Std dev
22.1%
10th percentile
44.1%
90th percentile
16.2%
Min
16.2%70.6%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.27 for chat models, measured over 31 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
1Claude Fable 5· OpenHands Index data70.6%
2Claude Opus 4.8· OpenHands Index data50%
3Claude Opus 4.7· OpenHands Index data48.5%
4Gemini 3.1 Pro Preview· OpenHands Index data44.1%
5Claude Opus 4.6· OpenHands Index data41.8%
6Claude Opus 4.5· OpenHands Index data41.2%
7GLM 5.1· OpenHands Index data41.2%
8Kimi K2.6· OpenHands Index data41.2%
9Gemini 3.5 Flash· OpenHands Index data39.7%
10GPT 5.5· OpenHands Index data38.2%
11Claude Sonnet 4.5· OpenHands Index data36.8%
12DeepSeek V4 Pro· OpenHands Index data36.8%
13Gemini 3 Pro Preview· OpenHands Index data36.8%
14GPT 5.4· OpenHands Index data36.8%
15MiniMax M3· OpenHands Index data36.8%
16GPT-5.2 Codex· OpenHands Index data35.9%
17GLM 5· OpenHands Index data35.3%
18Kimi K2.5· OpenHands Index data32.8%
19Kimi K2 Thinking· OpenHands Index data32.4%
20Claude Sonnet 4.6· OpenHands Index data30.9%
21GPT 5.2· OpenHands Index data30.9%
22Qwen3 Coder Next· OpenHands Index data30.9%
23Qwen3.6 Plus· OpenHands Index data30.9%
24Qwen3.5 Flash· OpenHands Index data27.9%
25MiniMax M2.5· OpenHands Index data25%
26Trinity Large Thinking· OpenHands Index data25%
27Qwen3 Coder· OpenHands Index data23.5%
28Gemini 3 Flash· OpenHands Index data22.1%
29GLM 4.7· OpenHands Index data22.1%
30Nemotron 3 Super· OpenHands Index data20.6%
31MiniMax M2.1· OpenHands Index data16.2%

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: OpenHands Index (All Hands AI), Apache-2.0 (licence)