Skip to content

Benchmark

WildBench: llama_reward.Creative Tasks.K=-1

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
32
Scores recorded
32
Sources
1
IQ weight
1.46

Score distribution

16.3
Mean
21.4
Median
25.7
Std dev
-11
10th percentile
45.3
90th percentile
-60.6
Min
-60.650.3
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.46 for chat models, measured over 32 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-4-0125-previewallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data50.3
2GPT-4 Turboallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data48.9
3GPT 4oallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data45.7
4Gemini 1.5 Proallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data45.3
5Llama 3 70B Instructallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data41.6
6Reka Coreallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data39.5
7Yi 1.5 34B Chatallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data38.6
8Claude 3.5 Sonnetallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data37
9Nemotron-4 340B Instructallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data36.6
10Gemini 1.5 Flashallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data34.6
11yi-largeallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data33.7
12DeepSeek Coder V2 Instructallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data32.6
13Command R Plusallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data30.7
14Claude 3 Opusallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data29.4
15GLM-4-9b-chatallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data22.3
16Qwen2 72B Instructallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data21.4
17Claude 3 Sonnetallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data14.9
18Mistral Largeallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf, 2402· WildBench direct data13.7
19Starling LM 7B Betaallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data13.7
20Reka Flashallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data11
21Llama 3 8B Instructallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data8.7
22Command R+allenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data7.4
23Claude 3 Haikuallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data2.4
24Tulu 2 DPO 70Ballenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data2.2
25Mixtral 8x7B Instruct v0.1allenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data1.6
26Mistral 7B Instruct v0.2allenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data1.4
27Reka Edgeallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data-0.144
28GPT-3.5 Turboallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data-10.3
29Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data-11
30Llama-2-7b-chat-hfallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data-22.8
31Gemma 7B ITallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data-40.1
32Gemma 2B ITallenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf· WildBench direct data-60.6

Data credits: WildBench, Allen Institute for AI