Benchmark
SWE-bench-Live Full
Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.
- Models measured
- 3
- Scores recorded
- 4
- Sources
- 1
- IQ weight
- No weight published
Score distribution
- 16%
- Mean
- 18.6%
- Median
- 4.1%
- Std dev
- 9.1%
- 10th percentile
- 19.3%
- 90th percentile
- 9.1%
- Min
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | Claude Sonnet 3.7scaffold: openhands, set: full, instances: 1319, submitted: 2025-05-01· SWE-bench-Live leaderboard data | 19.3% |
| 2 | GPT 4.1scaffold: swe-agent, set: full, instances: 1319, submitted: 2025-05-01· SWE-bench-Live leaderboard data | 18.6% |
| 3 | Claude Sonnet 3.7scaffold: swe-agent, set: full, instances: 1319, submitted: 2025-05-01· SWE-bench-Live leaderboard data | 17.1% |
| 4 | GPT 4oscaffold: swe-agent, set: full, instances: 1319, submitted: 2025-05-01· SWE-bench-Live leaderboard data | 9.1% |