Skip to content

Benchmark

WildBench: mixture_of_rewards.Creative Tasks.K=-1

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
30
Scores recorded
30
Sources
1
IQ weight
1.32

Score distribution

-2
Mean
1.8
Median
24
Std dev
-29.7
10th percentile
21.1
90th percentile
-68.3
Min
-68.332.3
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.32 for chat models, measured over 30 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-4-0125-previewallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data32.3
2GPT 4oallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data28.3
3Gemini 1.5 Proallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data27.4
4Claude 3.5 Sonnetallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data21.1
5Yi 1.5 34B Chatallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data20.5
6Llama 3 70B Instructallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data19.6
7Reka Coreallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data19
8Gemini 1.5 Flashallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data17.5
9Nemotron-4 340B Instructallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data16.7
10yi-largeallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data16
11DeepSeek Coder V2 Instructallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data13.6
12Claude 3 Opusallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data12.2
13Command R Plusallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data11.7
14GLM-4-9b-chatallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data5.9
15Qwen2 72B Instructallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data1.8
16Claude 3 Sonnetallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-2.5
17Starling LM 7B Betaallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-4
18Mistral Largeallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf, 2402· WildBench direct data-5.5
19Command R+allenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-8.4
20Llama 3 8B Instructallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-10.2
21Reka Flashallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-10.4
22Mixtral 8x7B Instruct v0.1allenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-15.8
23Mistral 7B Instruct v0.2allenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-16.6
24Reka Edgeallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-17.8
25Tulu 2 DPO 70Ballenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-18
26Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-26.2
27GPT-3.5 Turboallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-29.7
28Llama-2-7b-chat-hfallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-37.9
29Gemma 7B ITallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-51.8
30Gemma 2B ITallenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-68.3

Data credits: WildBench, Allen Institute for AI