Skip to content

Benchmark

DeepSWE v1.1

Scores are percentages. Higher is better.

Models and systems measured
46
Scores recorded
151
Sources
1
IQ weight
No weight published

Score distribution

61.7%
Mean
67%
Median
15.7%
Std dev
42.4%
10th percentile
73.8%
90th percentile
1.2%
Min
1.2%77.9%
The band covers the middle 80% of scores. The line marks the median.

Leaderboard

#Model or systemScore
51Claude Fable 5OpenAI launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.69.9%
52DeepSeek V4.1 FlashClaude Code · max effort · temp 1.0 · top_p .95 · 1M · max_steps 500 · N=8· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.69.8%
53Claude Fable 5max effort · $21.6347 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.69.7%
54Claude Fable 5OpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.69.7%
55Claude Fable 5Opus 5 System Card capability summary · adaptive thinking · max effort unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.69.7%
56GPT-5.6 TerraOpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.69.6%
57GPT-5.6 Solhigh effort · $2.66 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.69.4%
58Claude Opus 5medium effort · $3.29 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.68.9%
59GPT-6 Solmax effort · $2.7439 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.68.8%
60Claude Opus 5Opus 5 System Card capability summary · adaptive thinking · max effort unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.68.8%
61GPT-6 SolDeepSWE v1.1 · OpenAI release chart · max reasoning · displayed cost $2.74/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.68.8%
62Claude Fable 5high effort · $9.1776 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.68.6%
63DeepSeek V4.1 FlashDSH PTC · max effort · temp 1.0 · top_p .95 · 1M · max_steps 500 · N=8· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67.6%
64Kimi K3DeepSeek V4.1 launch chart · exact public value · setting not further specified· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67.5%
65Kimi K3Kimi Code · max effort· Reported by Moonshot AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67.5%
66Kimi K3mini-SWE / DeepSeek设置· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67.5%
67Kimi K3mini-swe-agent / Z.ai设置· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67.5%
68Claude Fable 5.1OpenAI launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67.4%
69Kimi K3mini-SWE-agent · official leaderboard snapshot· Reported by Moonshot AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67.3%
70GPT-5.6 LunaOpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67.2%
71GPT-6 Astralow effort · $1.5952 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67%
72GPT 5.5OpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67%
73GPT 5.5xhigh· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67%
74GPT 5.5xhigh · mini-swe-agent fork · no internet · 64GB VM · pass@1 · five-attempt mean· Reported by Meta, Meta Superintelligence Labs· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67%
75GPT-5.6 LunaGemini 3.6 Flash Model Card · July 2026· Reported by Google DeepMind· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67%
76GPT-6 AstraDeepSWE v1.1 · OpenAI release chart · low reasoning · displayed cost $1.60/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67%
77GLM 5.3DeepSeek V4.1 launch chart · exact public value · setting not further specified· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.66.9%
78GLM 5.3mini-SWE / DeepSeek设置· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.66.9%
79GLM 5.3mini-swe-agent / Z.ai设置· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.66.9%
80GPT-6 SolDeepSWE v1.1 · OpenAI release chart · xhigh reasoning · displayed cost $1.00/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.66.6%
81GPT-6 Lunamax effort · $0.2169 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.66.6%
82GPT-6 Solxhigh effort · $1.0033 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.66.6%
83DeepSeek V4.1 FlashPi · max effort · temp 1.0 · top_p .95 · 1M · max_steps 500 · N=8· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.66.2%
84DeepSeek V4.1 FlashCodex · max effort · temp 1.0 · top_p .95 · 1M · max_steps 500 · N=8· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.65.6%
85DeepSeek V4.1 FlashOpenCode · max effort · temp 1.0 · top_p .95 · 1M · max_steps 500 · N=8· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.65.5%
86Claude Fable 5medium effort · $6.0882 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.65.4%
87Gemini 3.7 Flashmini-swe-agent · LiteLLM 1.96 · high thinking· Reported by Google DeepMind· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.65.3%
88GPT-6 SolDeepSWE v1.1 · OpenAI release chart · high reasoning · displayed cost $0.64/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.65.3%
89GPT-6 Solhigh effort · $0.6404 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.65.3%
90Claude Opus 5Claude Code · max· Reported by Meta· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.65%
91GPT-5.6 TerraCodex · max· Reported by Meta· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.64.8%
92GPT-6.1 SolDeepSWE v1.1 · OpenAI release chart · low reasoning · displayed cost $0.17/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.64.4%
93Hy4 previewHF Eval Result· Reported by Tencent Hy· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.64.3%
94GLM 5.3 Flashmini-swe-agent / 6h· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.63.4%
95DeepSeek V4 Pro 0813DeepSeek V4.1 launch chart · exact public value · setting not further specified· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.62.7%
96DeepSeek V4 Pro 0813mini-SWE / DeepSeek设置· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.62.7%
97DeepSeek V4 Pro 0813mini-swe-agent / Z.ai设置· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.62.7%
98GPT-5.6 Lunamax effort · $0.532 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.62.2%
99GPT-6 Lunaxhigh effort · $0.1096 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.61.3%
100GPT-5.6 Solmedium effort · $1.42 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.61.1%

Showing 51–100 of 148 results.

Not ranked

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Claude Fable 5 with Opus 4.8 fallbackDatacurve mini-SWE-agent · Pass@1 · max effort· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.69.7%
Grok 4.6Datacurve mini-SWE-agent · Pass@1 · high effort· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.65.2%
Muse Spark (version unspecified)Meta Model API · highest effort · version unspecified · mini-swe-agent fork · no internet · 64GB VM · pass@1 · five-attempt mean· Reported by Meta, Meta Superintelligence Labs· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.10%

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.