Skip to content

Benchmark

WildBench: task_macro_reward.K=-1

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
30
Scores recorded
30
Sources
1
IQ weight
1.37

Score distribution

-3.7
Mean
2.3
Median
29.3
Std dev
-32.6
10th percentile
30.2
90th percentile
-75.7
Min
-75.740.8
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.37 for chat models, measured over 30 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 4oallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data40.8
2Claude 3.5 Sonnetallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data37.6
3Gemini 1.5 Proallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data37.6
4GPT-4-0125-previewallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data30.2
5Gemini 1.5 Flashallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data26
6Llama 3 70B Instructallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data22.5
7Claude 3 Opusallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data22.4
8Nemotron-4 340B Instructallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data19.9
9yi-largeallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data17.8
10Reka Coreallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data17.1
11Yi 1.5 34B Chatallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data16.1
12DeepSeek Coder V2 Instructallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data11.4
13Claude 3 Sonnetallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data10.7
14Qwen2 72B Instructallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data5
15GLM-4-9b-chatallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data2.3
16Command R Plusallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-3.6
17Mistral Largeallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf, 2402· WildBench direct data-12.4
18Reka Flashallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-12.5
19Starling LM 7B Betaallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-15.1
20Llama 3 8B Instructallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-15.8
21Mixtral 8x7B Instruct v0.1allenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-19.7
22Command R+allenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-21
23Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-23.8
24Tulu 2 DPO 70Ballenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-27.1
25Reka Edgeallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-27.9
26Mistral 7B Instruct v0.2allenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-28.6
27GPT-3.5 Turboallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-32.6
28Llama-2-7b-chat-hfallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-53.6
29Gemma 7B ITallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-57.8
30Gemma 2B ITallenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-75.7

Data credits: WildBench, Allen Institute for AI