Skip to content

Benchmark

DeepSWE v1.0

Scores are percentages. Higher is better.

Models measured
5
Scores recorded
5
Sources
1
IQ weight
No weight published

Score distribution

57.7%
Mean
62%
Median
9.4%
Std dev
40.1%
10th percentile
66.1%
90th percentile
40.1%
Min
40.1%66.1%
The band covers the middle 80% of scores. The line marks the median.

Leaderboard

#ModelScore
1Claude Fable 5Fable · max· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.66.1%
2GPT 5.5xhigh· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.64.3%
3Grok 4.5Datacurve eval · each provider's harness · pass@1· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.62%
4Claude Opus 4.8max· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.55.8%
5Claude Opus 4.7max· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.40.1%

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.