Skip to content

Benchmark

WildBench: llama_reward.task_macro.K=-1

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
32
Scores recorded
32
Sources
1
IQ weight
1.58

Score distribution

29.3
Mean
38.2
Median
30.4
Std dev
3.1
10th percentile
62.9
90th percentile
-60.9
Min
-60.967.2
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.58 for chat models, measured over 32 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 4oallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data67.2
2GPT-4 Turboallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data65.3
3Claude 3.5 Sonnetallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data65.2
4Gemini 1.5 Proallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data62.9
5GPT-4-0125-previewallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data59.7
6Llama 3 70B Instructallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data56.7
7Gemini 1.5 Flashallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data54.8
8Claude 3 Opusallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data52.8
9Nemotron-4 340B Instructallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data51.8
10Reka Coreallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data51.2
11yi-largeallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data50.9
12Yi 1.5 34B Chatallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data47.9
13DeepSeek Coder V2 Instructallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data45.3
14Claude 3 Sonnetallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data42.5
15Qwen2 72B Instructallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data40.8
16GLM-4-9b-chatallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data38.1
17Command R Plusallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data31.3
18Claude 3 Haikuallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data28.9
19Reka Flashallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data25.7
20Mistral Largeallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf, 2402· WildBench direct data24.8
21Llama 3 8B Instructallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data19.9
22Starling LM 7B Betaallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data19.5
23Mixtral 8x7B Instruct v0.1allenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data15.5
24Command R+allenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data12.5
25Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data8.7
26Tulu 2 DPO 70Ballenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data7.9
27Mistral 7B Instruct v0.2allenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data6.4
28Reka Edgeallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data5.4
29GPT-3.5 Turboallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data3.1
30Llama-2-7b-chat-hfallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data-31.5
31Gemma 7B ITallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data-33.2
32Gemma 2B ITallenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf· WildBench direct data-60.9

Data credits: WildBench, Allen Institute for AI