Skip to content

Benchmark

SWE-bench-Live Verified

Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.

Models measured
2
Scores recorded
2
Sources
1
IQ weight
No weight published

Score distribution

35.2%
Mean
40%
Median
4.8%
Std dev
30.4%
10th percentile
40%
90th percentile
30.4%
Min
30.4%40%
The band covers the middle 80% of scores. The line marks the median.

Leaderboard

#ModelScore
1Claude Sonnet 4.5scaffold: swe-agent, set: verified, instances: 500, submitted: 2025-12-31· SWE-bench-Live leaderboard data40%
2GPT 5scaffold: swe-agent, set: verified, instances: 500, submitted: 2025-12-31, effort: medium· SWE-bench-Live leaderboard data30.4%

Data credits: SWE-bench-Live (Microsoft and contributors), swe-bench-live.github.io. Resolve rates computed from the published report counts.