Skip to content

Benchmark

SWE-Lancer offline earned payout

Coding benchmarks compare models on programming tasks. Higher scores are better.

Models measured
2
Scores recorded
2
Sources
1
IQ weight
No weight published

Score distribution

28,562.5
Mean
45,625
Median
17,062.5
Std dev
11,500
10th percentile
45,625
90th percentile
11,500
Min
11,50045,625
The band covers the middle 80% of scores. The line marks the median.

Leaderboard

#ModelScore
1o1offline-198· SWE-Lancer offline leaderboard45,625
2GPT 4ooffline-198· SWE-Lancer offline leaderboard11,500

Data credits: SWE-Lancer community leaderboard. Offline task subset as sized in the publisher README; aggregate accuracy and earned payouts.