Skip to content

Benchmark

SWE-bench-Live MultiLang Go

Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.

Models measured
13
Scores recorded
24
Sources
1
IQ weight
0.36

Score distribution

38.8%
Mean
32.4%
Median
16.1%
Std dev
22.1%
10th percentile
65.2%
90th percentile
20.6%
Min
20.6%74.6%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.36 for chat models, measured over 13 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.6scaffold: ami-agent, set: go, instances: 138, submitted: 2026-07-16· SWE-bench-Live leaderboard data74.6%
2Claude Sonnet 4.6scaffold: ami-agent, set: go, instances: 138, submitted: 2026-07-23· SWE-bench-Live leaderboard data71.7%
3Claude Haiku 4.5scaffold: ami-agent, set: go, instances: 138, submitted: 2026-07-29· SWE-bench-Live leaderboard data65.2%
4DeepSeek V4.1 Flashscaffold: tianxicode, set: go, instances: 138, submitted: 2026-10-16· SWE-bench-Live leaderboard data63.8%
5Gemini 3.6 Flashscaffold: ami-agent, set: go, instances: 138, submitted: 2026-07-29· SWE-bench-Live leaderboard data56.5%
6Gemini 3.1 Pro Previewscaffold: ami-agent, set: go, instances: 138, submitted: 2026-07-29· SWE-bench-Live leaderboard data52.2%
7Claude Sonnet 4.5scaffold: claude-code, set: go, instances: 68, submitted: 2026-01-08· SWE-bench-Live leaderboard data44.1%
8Claude Sonnet 4.5scaffold: openhands, set: go, instances: 68, submitted: 2026-01-08· SWE-bench-Live leaderboard data39.7%
9Claude Sonnet 4.5scaffold: swe-agent, set: go, instances: 68, submitted: 2026-01-08· SWE-bench-Live leaderboard data39.7%
10GPT 5.5scaffold: claude-code, set: go, instances: 98, submitted: 2026-06-05, effort: medium· SWE-bench-Live leaderboard data37.8%
11GPT 5.5scaffold: swe-agent, set: go, instances: 98, submitted: 2026-06-05, effort: medium· SWE-bench-Live leaderboard data34.7%
12Gemini 3 Flashscaffold: swe-agent, set: go, instances: 68, submitted: 2026-01-08· SWE-bench-Live leaderboard data32.4%
13Gemini 3 Flashscaffold: openhands, set: go, instances: 68, submitted: 2026-01-08· SWE-bench-Live leaderboard data30.9%
14Qwen 3.8 27Bscaffold: ami-agent, set: go, instances: 138, submitted: 2026-10-09· SWE-bench-Live leaderboard data29.7%
15DeepSeek V4 Proscaffold: claude-code, set: go, instances: 98, submitted: 2026-06-05· SWE-bench-Live leaderboard data29.6%
16DeepSeek V4 Proscaffold: swe-agent, set: go, instances: 98, submitted: 2026-06-05· SWE-bench-Live leaderboard data29.6%
17DeepSeek V3.1 Terminusscaffold: claude-code, set: go, instances: 68, submitted: 2026-01-08· SWE-bench-Live leaderboard data29.4%
18GPT 5.2scaffold: claude-code, set: go, instances: 68, submitted: 2026-01-08, effort: medium, thinking· SWE-bench-Live leaderboard data27.9%
19DeepSeek V4 Proscaffold: openhands, set: go, instances: 98, submitted: 2026-06-05· SWE-bench-Live leaderboard data27.6%
20GPT 5.5scaffold: openhands, set: go, instances: 98, submitted: 2026-06-05, effort: medium· SWE-bench-Live leaderboard data25.5%
21GPT 5.2scaffold: swe-agent, set: go, instances: 68, submitted: 2026-01-08, effort: medium, thinking· SWE-bench-Live leaderboard data25%
22DeepSeek V3.1 Terminusscaffold: openhands, set: go, instances: 68, submitted: 2026-01-08· SWE-bench-Live leaderboard data22.1%
23DeepSeek V3.1 Terminusscaffold: swe-agent, set: go, instances: 68, submitted: 2026-01-08· SWE-bench-Live leaderboard data20.6%
24GPT 5.2scaffold: openhands, set: go, instances: 68, submitted: 2026-01-08, effort: medium, thinking· SWE-bench-Live leaderboard data20.6%

Data credits: SWE-bench-Live (Microsoft and contributors), swe-bench-live.github.io. Resolve rates computed from the published report counts.