Skip to content

Benchmark

SWE-Lancer

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
4
Scores recorded
4
Sources
1
IQ weight
No weight published

Score distribution

38.6%
Mean
37.3%
Median
17.5%
Std dev
18%
10th percentile
66.3%
90th percentile
18%
Min
18%66.3%
The band covers the middle 80% of scores. The line marks the median.

Leaderboard

#ModelScore
1GPT-5.1 Codex· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim66.3%
2GPT-4.5 Preview· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim37.3%
3GPT 4o· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim32.6%
4o3 Mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim18%

Data credits: Data by LLM Stats