Skip to content

Benchmark

WildBench: llama_reward.Coding & Debugging.K=-1

Coding benchmarks compare models on programming tasks. Higher scores are better.

Models measured
32
Scores recorded
32
Sources
1
IQ weight
1.58

Score distribution

37.9
Mean
47.3
Median
34.3
Std dev
12.3
10th percentile
75.7
90th percentile
-62.6
Min
-62.679.4
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.58 for chat models, measured over 32 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude 3.5 Sonnetallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data79.4
2GPT-4 Turboallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data78.8
3Gemini 1.5 Proallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data77.5
4GPT 4oallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data75.7
5Gemini 1.5 Flashallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data70.6
6GPT-4-0125-previewallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data70.1
7Llama 3 70B Instructallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data67.2
8Claude 3 Opusallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data66.2
9Nemotron-4 340B Instructallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data64.2
10Reka Coreallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data60.5
11yi-largeallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data59.9
12Claude 3 Sonnetallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data58.8
13DeepSeek Coder V2 Instructallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data57.8
14Yi 1.5 34B Chatallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data49.7
15GLM-4-9b-chatallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data49.7
16Qwen2 72B Instructallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data47.3
17Claude 3 Haikuallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data43.1
18Reka Flashallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data37
19Mistral Largeallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf, 2402· WildBench direct data36.5
20Command R Plusallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data29.5
21Starling LM 7B Betaallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data26.7
22Mixtral 8x7B Instruct v0.1allenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data23.5
23Llama 3 8B Instructallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data22.1
24Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data17.4
25Tulu 2 DPO 70Ballenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data17.3
26GPT-3.5 Turboallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data17.3
27Mistral 7B Instruct v0.2allenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data15.7
28Reka Edgeallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data12.6
29Command R+allenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data12.3
30Gemma 7B ITallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data-26.2
31Llama-2-7b-chat-hfallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data-43.9
32Gemma 2B ITallenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf· WildBench direct data-62.6

Data credits: WildBench, Allen Institute for AI