Benchmark
DeepSWE v1.1
Scores are percentages. Higher is better.
Benchmark sourceScores from BenchBoard docs.
- Models and systems measured
- 46
- Scores recorded
- 151
- Sources
- 1
- IQ weight
- No weight published
Score distribution
- 61.7%
- Mean
- 67%
- Median
- 15.7%
- Std dev
- 42.4%
- 10th percentile
- 73.8%
- 90th percentile
- 1.2%
- Min
Leaderboard
| # | Model or system | Score |
|---|---|---|
| 51 | Claude Fable 5OpenAI launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 69.9% |
| 52 | DeepSeek V4.1 FlashClaude Code · max effort · temp 1.0 · top_p .95 · 1M · max_steps 500 · N=8· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 69.8% |
| 53 | Claude Fable 5max effort · $21.6347 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 69.7% |
| 54 | Claude Fable 5OpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 69.7% |
| 55 | Claude Fable 5Opus 5 System Card capability summary · adaptive thinking · max effort unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 69.7% |
| 56 | GPT-5.6 TerraOpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 69.6% |
| 57 | GPT-5.6 Solhigh effort · $2.66 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 69.4% |
| 58 | Claude Opus 5medium effort · $3.29 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 68.9% |
| 59 | GPT-6 Solmax effort · $2.7439 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 68.8% |
| 60 | Claude Opus 5Opus 5 System Card capability summary · adaptive thinking · max effort unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 68.8% |
| 61 | GPT-6 SolDeepSWE v1.1 · OpenAI release chart · max reasoning · displayed cost $2.74/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 68.8% |
| 62 | Claude Fable 5high effort · $9.1776 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 68.6% |
| 63 | DeepSeek V4.1 FlashDSH PTC · max effort · temp 1.0 · top_p .95 · 1M · max_steps 500 · N=8· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 67.6% |
| 64 | Kimi K3DeepSeek V4.1 launch chart · exact public value · setting not further specified· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 67.5% |
| 65 | Kimi K3Kimi Code · max effort· Reported by Moonshot AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 67.5% |
| 66 | Kimi K3mini-SWE / DeepSeek设置· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 67.5% |
| 67 | Kimi K3mini-swe-agent / Z.ai设置· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 67.5% |
| 68 | Claude Fable 5.1OpenAI launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 67.4% |
| 69 | Kimi K3mini-SWE-agent · official leaderboard snapshot· Reported by Moonshot AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 67.3% |
| 70 | GPT-5.6 LunaOpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 67.2% |
| 71 | GPT-6 Astralow effort · $1.5952 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 67% |
| 72 | GPT 5.5OpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 67% |
| 73 | GPT 5.5xhigh· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 67% |
| 74 | GPT 5.5xhigh · mini-swe-agent fork · no internet · 64GB VM · pass@1 · five-attempt mean· Reported by Meta, Meta Superintelligence Labs· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 67% |
| 75 | GPT-5.6 LunaGemini 3.6 Flash Model Card · July 2026· Reported by Google DeepMind· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 67% |
| 76 | GPT-6 AstraDeepSWE v1.1 · OpenAI release chart · low reasoning · displayed cost $1.60/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 67% |
| 77 | GLM 5.3DeepSeek V4.1 launch chart · exact public value · setting not further specified· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 66.9% |
| 78 | GLM 5.3mini-SWE / DeepSeek设置· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 66.9% |
| 79 | GLM 5.3mini-swe-agent / Z.ai设置· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 66.9% |
| 80 | GPT-6 SolDeepSWE v1.1 · OpenAI release chart · xhigh reasoning · displayed cost $1.00/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 66.6% |
| 81 | GPT-6 Lunamax effort · $0.2169 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 66.6% |
| 82 | GPT-6 Solxhigh effort · $1.0033 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 66.6% |
| 83 | DeepSeek V4.1 FlashPi · max effort · temp 1.0 · top_p .95 · 1M · max_steps 500 · N=8· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 66.2% |
| 84 | DeepSeek V4.1 FlashCodex · max effort · temp 1.0 · top_p .95 · 1M · max_steps 500 · N=8· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 65.6% |
| 85 | DeepSeek V4.1 FlashOpenCode · max effort · temp 1.0 · top_p .95 · 1M · max_steps 500 · N=8· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 65.5% |
| 86 | Claude Fable 5medium effort · $6.0882 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 65.4% |
| 87 | Gemini 3.7 Flashmini-swe-agent · LiteLLM 1.96 · high thinking· Reported by Google DeepMind· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 65.3% |
| 88 | GPT-6 SolDeepSWE v1.1 · OpenAI release chart · high reasoning · displayed cost $0.64/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 65.3% |
| 89 | GPT-6 Solhigh effort · $0.6404 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 65.3% |
| 90 | Claude Opus 5Claude Code · max· Reported by Meta· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 65% |
| 91 | GPT-5.6 TerraCodex · max· Reported by Meta· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 64.8% |
| 92 | GPT-6.1 SolDeepSWE v1.1 · OpenAI release chart · low reasoning · displayed cost $0.17/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 64.4% |
| 93 | Hy4 previewHF Eval Result· Reported by Tencent Hy· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 64.3% |
| 94 | GLM 5.3 Flashmini-swe-agent / 6h· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 63.4% |
| 95 | DeepSeek V4 Pro 0813DeepSeek V4.1 launch chart · exact public value · setting not further specified· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 62.7% |
| 96 | DeepSeek V4 Pro 0813mini-SWE / DeepSeek设置· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 62.7% |
| 97 | DeepSeek V4 Pro 0813mini-swe-agent / Z.ai设置· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 62.7% |
| 98 | GPT-5.6 Lunamax effort · $0.532 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 62.2% |
| 99 | GPT-6 Lunaxhigh effort · $0.1096 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 61.3% |
| 100 | GPT-5.6 Solmedium effort · $1.42 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 61.1% |
Showing 51–100 of 148 results.
Not ranked
The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.
| Model or system | Score |
|---|---|
| Claude Fable 5 with Opus 4.8 fallbackDatacurve mini-SWE-agent · Pass@1 · max effort· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 69.7% |
| Grok 4.6Datacurve mini-SWE-agent · Pass@1 · high effort· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 65.2% |
| Muse Spark (version unspecified)Meta Model API · highest effort · version unspecified · mini-swe-agent fork · no internet · 64GB VM · pass@1 · five-attempt mean· Reported by Meta, Meta Superintelligence Labs· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 10% |