Skip to content

Benchmark

OpenHands Index: swt-bench

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages.

Models measured
31
Scores recorded
31
Sources
1
IQ weight
2.05

Score distribution

64.7%
Mean
68.6%
Median
15.2%
Std dev
45.7%
10th percentile
81.1%
90th percentile
33.3%
Min
33.3%91.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.05 for chat models, measured over 31 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
1Claude Fable 5· OpenHands Index data91.9%
2Claude Opus 4.8· OpenHands Index data84.3%
3GPT 5.5· OpenHands Index data83.4%
4MiniMax M3· OpenHands Index data81.1%
5Claude Opus 4.7· OpenHands Index data80.8%
6Claude Opus 4.6· OpenHands Index data78.8%
7Claude Opus 4.5· OpenHands Index data78.5%
8Gemini 3.5 Flash· OpenHands Index data76.2%
9Gemini 3.1 Pro Preview· OpenHands Index data75.1%
10GPT 5.2· OpenHands Index data73.2%
11Gemini 3 Flash· OpenHands Index data70.7%
12GPT 5.4· OpenHands Index data70.4%
13Kimi K2.6· OpenHands Index data70.4%
14GLM 5.1· OpenHands Index data70.2%
15Claude Sonnet 4.5· OpenHands Index data68.8%
16Gemini 3 Pro Preview· OpenHands Index data68.6%
17DeepSeek V4 Pro· OpenHands Index data68.1%
18MiniMax M2.5· OpenHands Index data68.1%
19GPT-5.2 Codex· OpenHands Index data67%
20Qwen3.6 Plus· OpenHands Index data62.1%
21Kimi K2.5· OpenHands Index data61.9%
22MiniMax M2.1· OpenHands Index data61.4%
23Claude Sonnet 4.6· OpenHands Index data54%
24GLM 4.7· OpenHands Index data49.4%
25GLM 5· OpenHands Index data47.3%
26Kimi K2 Thinking· OpenHands Index data47.3%
27Nemotron 3 Super· OpenHands Index data45.7%
28Qwen3 Coder Next· OpenHands Index data45.7%
29Qwen3.5 Flash· OpenHands Index data38.3%
30Qwen3 Coder· OpenHands Index data34.9%
31Trinity Large Thinking· OpenHands Index data33.3%

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: OpenHands Index (All Hands AI), Apache-2.0 (licence)