Skip to content

Benchmark

WildBench: WB_score.Coding & Debugging

Coding benchmarks compare models on programming tasks. Higher scores are better.

Models measured
41
Scores recorded
41
Sources
1
IQ weight
1.47

Score distribution

35.2
Mean
39.7
Median
18.5
Std dev
13.5
10th percentile
55.2
90th percentile
-17.7
Min
-17.760.5
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.47 for chat models, measured over 41 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 4oallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data60.5
2Athene 70Ballenai/WildBench, WB_score.Coding & Debugging· WildBench direct data59
3GPT-4o miniallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data57.2
4Claude 3.5 Sonnetallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data56.5
5Gemini 1.5 Proallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data55.2
6GPT-4 Turboallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data55.1
7DeepSeek V2 Chatallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data55
8Mistral Large 2allenai/WildBench, WB_score.Coding & Debugging· WildBench direct data53.8
9Claude 3 Opusallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data53.3
10GPT-4-0125-previewallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data52.9
11Gemma 2 9B IT SimPOallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data50.9
12Gemini 1.5 Flashallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data48.7
13yi-largeallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data47.7
14Nemotron-4 340B Instructallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data46.3
15Claude 3 Sonnetallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data46.1
16DeepSeek Coder V2 Instructallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data44.9
17Llama 3 70B Instructallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data44.7
18Yi 1.5 34B Chatallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data42.1
19Reka Coreallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data40.6
20Qwen2 72B Instructallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data39.8
21Mistral Nemoallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data39.7
22Claude 3 Haikuallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data37
23Gemma 2 9B ITallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data36.7
24GLM-4-9b-chatallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data35.4
25Mistral Largeallenai/WildBench, WB_score.Coding & Debugging, 2402· WildBench direct data33.7
26Command R Plusallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data28.4
27GPT-3.5 Turboallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data26.5
28Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data26
29Mixtral 8x7B Instruct v0.1allenai/WildBench, WB_score.Coding & Debugging· WildBench direct data25
30Starling LM 7B Betaallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data24.4
31Reka Flashallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data22.1
32Llama 3 8B Instructallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data22
33Tulu 2 DPO 70Ballenai/WildBench, WB_score.Coding & Debugging· WildBench direct data20.7
34Command R+allenai/WildBench, WB_score.Coding & Debugging· WildBench direct data19.3
35Mistral 7B Instruct v0.2allenai/WildBench, WB_score.Coding & Debugging· WildBench direct data18.4
36Gemma 2 2B ITallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data17.9
37Reka Edgeallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data13.5
38Llama-2-70b-chat-hfallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data9.3
39Gemma 7B ITallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data1.8
40Llama-2-7b-chat-hfallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data-6.8
41Gemma 2B ITallenai/WildBench, WB_score.Coding & Debugging· WildBench direct data-17.7

Data credits: WildBench, Allen Institute for AI