Skip to content

Benchmark

WildBench: haiku_reward.Planning & Reasoning.K=-1

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
32
Scores recorded
32
Sources
1
IQ weight
1.40

Score distribution

3.8
Mean
10.4
Median
32.6
Std dev
-32.2
10th percentile
45.2
90th percentile
-75.7
Min
-75.753.1
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.40 for chat models, measured over 32 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 4oallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data53.1
2GPT-4 Turboallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data46.6
3Claude 3.5 Sonnetallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data46.4
4Gemini 1.5 Proallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data45.2
5GPT-4-0125-previewallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data40.3
6Claude 3 Opusallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data32.9
7Llama 3 70B Instructallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data32.8
8Gemini 1.5 Flashallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data32.3
9yi-largeallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data29.4
10Reka Coreallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data28.3
11Yi 1.5 34B Chatallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data27.7
12Nemotron-4 340B Instructallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data27.4
13Claude 3 Sonnetallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data22.5
14DeepSeek Coder V2 Instructallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data20.8
15Qwen2 72B Instructallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data13.2
16GLM-4-9b-chatallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data10.4
17Command R Plusallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data8.4
18Starling LM 7B Betaallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data-7.3
19Mistral Largeallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307, 2402· WildBench direct data-7.4
20Reka Flashallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data-7.8
21Llama 3 8B Instructallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data-9.7
22Command R+allenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data-13
23Mixtral 8x7B Instruct v0.1allenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data-17.9
24Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data-20.5
25Tulu 2 DPO 70Ballenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data-23.5
26Reka Edgeallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data-24.3
27Llama-2-70b-chat-hfallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data-25.4
28Mistral 7B Instruct v0.2allenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data-27.3
29GPT-3.5 Turboallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data-32.2
30Llama-2-7b-chat-hfallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data-47.3
31Gemma 7B ITallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data-57.5
32Gemma 2B ITallenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307· WildBench direct data-75.7

Data credits: WildBench, Allen Institute for AI