Skip to content

Benchmark

BigCodeBench Complete

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
77
Scores recorded
77
Sources
1
IQ weight
1.92

Score distribution

44.3%
Mean
47.6%
Median
13.1%
Std dev
25.7%
10th percentile
58.2%
90th percentile
7.9%
Min
7.9%62.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.92 for chat models, measured over 77 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1DeepSeek V3· BigCodeBench Leaderboard data62.2%
2Llama 4 Maverick· BigCodeBench Leaderboard data61.4%
3GPT 4o· BigCodeBench Leaderboard data61.1%
4Gemini 2.0 Flash Experimental· BigCodeBench Leaderboard data59.9%
5DeepSeek Coder V2 Instruct· BigCodeBench Leaderboard data59.7%
6gemini-exp· BigCodeBench Leaderboard data59.3%
7Claude Haiku 3.5· BigCodeBench Leaderboard data59%
8Claude 3.5 Sonnet· BigCodeBench Leaderboard data58.6%
9GPT-4 Turbo· BigCodeBench Leaderboard data58.2%
10Qwen 2.5 Coder 32B Instruct· BigCodeBench Leaderboard data58%
11Llama 3.3 70B Instruct· BigCodeBench Leaderboard data57.5%
12Claude 3 Opus· BigCodeBench Leaderboard data57.4%
13GPT-4o mini· BigCodeBench Leaderboard data57.4%
14GPT 4· BigCodeBench Leaderboard data57.2%
15Athene V2 Chat· BigCodeBench Leaderboard data56.8%
16qwen2.5-coder-14b-instruct· BigCodeBench Leaderboard data56.7%
17Qwen2.5 72B Instruct· BigCodeBench Leaderboard data55.9%
18Phi 4· BigCodeBench Leaderboard data55.4%
19DeepSeek R1 Distill Qwen 32B· BigCodeBench Leaderboard data54.9%
20Llama 3.1 70B Instruct· BigCodeBench Leaderboard data54.8%
21Llama 3 70B Instruct· BigCodeBench Leaderboard data54.5%
22QwQ 32B Preview· BigCodeBench Leaderboard data54.4%
23Claude 3 Sonnet· BigCodeBench Leaderboard data53.8%
24DeepSeek V2.5· BigCodeBench Leaderboard data53.2%
25Qwen2.5 32B Instruct· BigCodeBench Leaderboard data52.3%
26Qwen 2.5 14B Instruct· BigCodeBench Leaderboard data52.2%
27DeepSeek Coder 33B Instruct· BigCodeBench Leaderboard data51.1%
28GPT-3.5 Turbo· BigCodeBench Leaderboard data50.6%
29Mistral Small 24B Instruct2501· BigCodeBench Leaderboard data50.4%
30Mixtral 8x22B Instruct· BigCodeBench Leaderboard data50.2%
31Claude 3 Haiku· BigCodeBench Leaderboard data50.1%
32DeepSeek R1 Distill Llama 70b· BigCodeBench Leaderboard data49.9%
33Code Llama 70B Instruct· BigCodeBench Leaderboard data49.6%
34DeepSeek V2 Chat· BigCodeBench Leaderboard data49%
35Qwen2.5 Coder 7B Instruct· BigCodeBench Leaderboard data48.8%
36Phi-3 Medium 128K Instruct· BigCodeBench Leaderboard data48.7%
37DeepSeek R1 Distill Qwen 14B· BigCodeBench Leaderboard data48.4%
38Llama 3.1 Nemotron 70B Instruct· BigCodeBench Leaderboard data48.2%
39deepseek-coder-v2-lite-instruct· BigCodeBench Leaderboard data47.6%
40Athene 70B· BigCodeBench Leaderboard data47.5%
41yi-large· BigCodeBench Leaderboard data47.2%
42Qwen2.5 7B Instruct· BigCodeBench Leaderboard data46.1%
43C4AI Command R 08 2024· BigCodeBench Leaderboard data45.2%
44codestral-mamba· BigCodeBench Leaderboard data44.6%
45Qwen1.5-110B-Chat· BigCodeBench Leaderboard data44.4%
46deepseek-coder-6.7b-instruct· BigCodeBench Leaderboard data43.8%
47Yi 1.5 34B Chat· BigCodeBench Leaderboard data43.8%
48Llama 4 Scout· BigCodeBench Leaderboard data43.1%
49Qwen2 7B Instruct· BigCodeBench Leaderboard data42.1%
50Qwen1.5-32B-Chat· BigCodeBench Leaderboard data42%

Showing 1–50 of 77 results.