Skip to content

Benchmark

DeepSWE 1.1

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
55
Scores recorded
55
Sources
1
IQ weight
2.55

Score distribution

62.4%
Mean
66.9%
Median
14.1%
Std dev
42.2%
10th percentile
75.4%
90th percentile
12%
Min
12%79%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.55 for chat models, measured over 55 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51Laguna S 2.1· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim40.4%
52Gemini 3.5 Flash· LLM Stats data37%
53Kimi K2.7 Code· LLM Stats data31%
54Claude Sonnet 4.6· LLM Stats data30%
55Gemini 3.1 Pro Preview· LLM Stats data12%

Showing 51–55 of 55 results.

Data credits: Data by LLM Stats