Benchmark
SWE-Effi Resolved Rate
Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.
- Models measured
- 3
- Scores recorded
- 15
- Sources
- 1
- IQ weight
- No weight published
Score distribution
- 23.2%
- Mean
- 26%
- Median
- 11.5%
- Std dev
- 10%
- 10th percentile
- 38%
- 90th percentile
- 4%
- Min
Leaderboard
| # | Model | |
|---|---|---|
| 1 | Qwen3 32Bagentless· SWE-Effi data docs | 48% |
| 2 | Qwen3 32Bauto-code-rover· SWE-Effi data docs | 38% |
| 3 | Qwen3 32Bopenhands· SWE-Effi data docs | 34% |
| 4 | Llama 3.3 70B Instructagentless· SWE-Effi data docs | 28% |
| 5 | Llama 3.3 70B Instructauto-code-rover· SWE-Effi data docs | 28% |
| 6 | Qwen3 32Bagentless-mini· SWE-Effi data docs | 28% |
| 7 | Qwen3 32Bswe-agent· SWE-Effi data docs | 28% |
| 8 | GPT-4o miniagentless· SWE-Effi data docs | 26% |
| 9 | GPT-4o miniagentless-mini· SWE-Effi data docs | 20% |
| 10 | Llama 3.3 70B Instructopenhands· SWE-Effi data docs | 20% |
| 11 | GPT-4o miniauto-code-rover· SWE-Effi data docs | 12% |
| 12 | Llama 3.3 70B Instructswe-agent· SWE-Effi data docs | 12% |
| 13 | GPT-4o miniopenhands· SWE-Effi data docs | 11.9% |
| 14 | GPT-4o miniswe-agent· SWE-Effi data docs | 10% |
| 15 | Llama 3.3 70B Instructagentless-mini· SWE-Effi data docs | 4% |
Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.
Data credits: SWE-Effi (Centre for Software Excellence), Apache-2.0; resolved rate converted to percent; resource figures are per-task means (licence)