Skip to content

Benchmark

DeepSWE 1.0

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
2
Scores recorded
2
Sources
1
IQ weight
No weight published

Score distribution

54.2%
Mean
62%
Median
7.8%
Std dev
46.3%
10th percentile
62%
90th percentile
46.3%
Min

Leaderboard

#ModelScore
1Grok 4.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim62%
2Mercury 3· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim46.3%

Data credits: Data by LLM Stats