Skip to content

Benchmark

WildBench: WB_score.Planning & Reasoning

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
41
Scores recorded
41
Sources
1
IQ weight
1.74

Score distribution

42.2
Mean
46.8
Median
14
Std dev
26.8
10th percentile
56.2
90th percentile
-5.8
Min
-5.861
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.74 for chat models, measured over 41 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Athene 70Ballenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data61
2GPT 4oallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data60.2
3GPT-4o miniallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data58.2
4Mistral Large 2allenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data57.2
5GPT-4 Turboallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data56.2
6Gemma 2 9B IT SimPOallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data55.6
7Claude 3.5 Sonnetallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data55.6
8DeepSeek V2 Chatallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data54.8
9Gemini 1.5 Proallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data53.7
10GPT-4-0125-previewallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data53.5
11Claude 3 Opusallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data52.5
12yi-largeallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data51.3
13Gemini 1.5 Flashallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data50.8
14Llama 3 70B Instructallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data50.1
15DeepSeek Coder V2 Instructallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data49.2
16Nemotron-4 340B Instructallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data49.1
17Yi 1.5 34B Chatallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data48.1
18Reka Coreallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data48
19Claude 3 Sonnetallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data47.4
20Mistral Nemoallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data47.4
21Qwen2 72B Instructallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data46.8
22Gemma 2 9B ITallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data46.7
23GLM-4-9b-chatallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data42.5
24Command R Plusallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data41.9
25Mistral Largeallenai/WildBench, WB_score.Planning & Reasoning, 2402· WildBench direct data41.8
26Claude 3 Haikuallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data41.3
27Reka Flashallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data35
28Command R+allenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data34.6
29Mixtral 8x7B Instruct v0.1allenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data34.6
30Llama 3 8B Instructallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data34.4
31Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data34.2
32Starling LM 7B Betaallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data34.1
33Gemma 2 2B ITallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data33.8
34GPT-3.5 Turboallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data33.4
35Tulu 2 DPO 70Ballenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data32.3
36Mistral 7B Instruct v0.2allenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data30.1
37Llama-2-70b-chat-hfallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data26.8
38Reka Edgeallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data25
39Llama-2-7b-chat-hfallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data15.4
40Gemma 7B ITallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data10.2
41Gemma 2B ITallenai/WildBench, WB_score.Planning & Reasoning· WildBench direct data-5.8

Data credits: WildBench, Allen Institute for AI