Skip to content

Benchmark

WildBench: llama_reward.K=-1

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
32
Scores recorded
32
Sources
1
IQ weight
1.60

Score distribution

21.7
Mean
28.3
Median
26.4
Std dev
-4.3
10th percentile
50.1
90th percentile
-58.8
Min
-58.854.8
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.60 for chat models, measured over 32 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-4 Turboallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data54.8
2GPT 4oallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data52.5
3GPT-4-0125-previewallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data51.3
4Claude 3.5 Sonnetallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data50.1
5Gemini 1.5 Proallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data47.9
6Llama 3 70B Instructallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data45.8
7Nemotron-4 340B Instructallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data42.6
8Reka Coreallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data42
9Claude 3 Opusallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data41.3
10yi-largeallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data41.1
11Yi 1.5 34B Chatallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data40.6
12Gemini 1.5 Flashallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data40.1
13DeepSeek Coder V2 Instructallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data36.4
14Qwen2 72B Instructallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data31.8
15Claude 3 Sonnetallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data29.1
16GLM-4-9b-chatallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data28.3
17Command R Plusallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data26.1
18Reka Flashallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data19.2
19Mistral Largeallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf, 2402· WildBench direct data18.4
20Claude 3 Haikuallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data18.3
21Starling LM 7B Betaallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data15.9
22Llama 3 8B Instructallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data14.5
23Mixtral 8x7B Instruct v0.1allenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data9.7
24Command R+allenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data9.6
25Mistral 7B Instruct v0.2allenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data3.6
26Tulu 2 DPO 70Ballenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data3.2
27Reka Edgeallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data2.5
28Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data2.4
29GPT-3.5 Turboallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data-4.3
30Llama-2-7b-chat-hfallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data-25.2
31Gemma 7B ITallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data-35.9
32Gemma 2B ITallenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf· WildBench direct data-58.8

Data credits: WildBench, Allen Institute for AI