Benchmark
SWE-fficiency Correctness
Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.
- Models measured
- 16
- Scores recorded
- 16
- Sources
- 1
- IQ weight
- 1.34
Score distribution
- 71.8%
- Mean
- 80.9%
- Median
- 21%
- Std dev
- 59.6%
- 10th percentile
- 89%
- 90th percentile
- 1.2%
- Min
This benchmark has a weight of 1.34 for chat models, measured over 16 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | Claude Opus 4.5openhands· SWE-fficiency data docs | 90.8% |
| 2 | GPT 5.2openhands· SWE-fficiency data docs | 89% |
| 3 | Gemini 3 Pro Previewopenhands· SWE-fficiency data docs | 85.9% |
| 4 | Claude Opus 4.1openhands· SWE-fficiency data docs | 84.7% |
| 5 | Qwen3 Coder Plusopenhands· SWE-fficiency data docs | 82.1% |
| 6 | GPT 5openhands· SWE-fficiency data docs | 81.7% |
| 7 | Claude Sonnet 4.5openhands· SWE-fficiency data docs | 81.3% |
| 8 | DeepSeek V3.1openhands· SWE-fficiency data docs | 80.9% |
| 9 | Claude Opus 4.6openhands· SWE-fficiency data docs | 79.7% |
| 10 | Gemini 3 Flashopenhands· SWE-fficiency data docs | 74.9% |
| 11 | Kimi K2 0905openhands· SWE-fficiency data docs | 73.9% |
| 12 | GLM 4.6openhands· SWE-fficiency data docs | 67.3% |
| 13 | Gemini 2.5 Flashopenhands· SWE-fficiency data docs | 60.8% |
| 14 | Gemini 2.5 Proopenhands· SWE-fficiency data docs | 59.6% |
| 15 | GPT-5 miniopenhands· SWE-fficiency data docs | 54.8% |
| 16 | Claude Sonnet 3.7openhands· SWE-fficiency data docs | 1.2% |
Data credits: SWE-fficiency (swefficiency.com) leaderboard materials, CC BY 4.0; Human Expert reference row omitted (licence)