Skip to content

Benchmark

MBPP

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
65
Scores recorded
65
Sources
2
IQ weight
1.60

Score distribution

72.3%
Mean
74.4%
Median
15.1%
Std dev
52.4%
10th percentile
89.7%
90th percentile
29.8%
Min
29.8%95.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.60 for chat models, measured over 64 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51Gemma 2 27B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim62.6%
52codellama-7b· EvalPlus Leaderboard data59.5%
53Mixtral 8x7B Instruct v0.1· EvalPlus Leaderboard data59.5%
54Gemma 1.1 7B IT· EvalPlus Leaderboard data57.1%
55Gemma 3n E2b It· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim56.6%
56codegemma-2b· EvalPlus Leaderboard data55.6%
57gemma-7b· EvalPlus Leaderboard data52.6%
58Gemma 2 9B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim52.4%
59mistral-7b· EvalPlus Leaderboard data51.9%
60Gemma 7B IT· EvalPlus Leaderboard data47.1%
61Mistral 7B Instruct v0.2· EvalPlus Leaderboard data44.7%
62SOLAR 10.7B Instruct v1.0· EvalPlus Leaderboard data43.9%
63gemma-2b· EvalPlus Leaderboard data41.8%
64Gemma 3 1b it· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim35.2%
65Gemma 1.1 2B IT· EvalPlus Leaderboard data29.8%

Showing 51–65 of 65 results.

Data credits: EvalPlus Leaderboard and EvalPerf (evalplus.github.io), Apache-2.0; EvalPerf win-rate fractions converted to percentages (licence) · Data by LLM Stats