Skip to content

Benchmark

SWE-fficiency Speedup Ratio

Coding benchmarks compare models on programming tasks. Higher scores are better.

Models measured
16
Scores recorded
16
Sources
1
IQ weight
1.36

Score distribution

0.091
Mean
0.098
Median
0.056
Std dev
0.039
10th percentile
0.157
90th percentile
0.0244
Min
0.02440.225
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.36 for chat models, measured over 16 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

Data credits: SWE-fficiency (swefficiency.com) leaderboard materials, CC BY 4.0; Human Expert reference row omitted (licence)