Skip to content

Benchmark

WildBench: WB_score.Information/Advice seeking

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
41
Scores recorded
41
Sources
1
IQ weight
1.68

Score distribution

45.8
Mean
49
Median
11.8
Std dev
36.5
10th percentile
57.2
90th percentile
-2.1
Min
-2.160.8
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.68 for chat models, measured over 41 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Athene 70Ballenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data60.8
2GPT 4oallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data58.6
3GPT-4o miniallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data57.4
4Mistral Large 2allenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data57.4
5GPT-4 Turboallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data57.2
6Gemma 2 9B IT SimPOallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data56.5
7Claude 3.5 Sonnetallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data55.5
8GPT-4-0125-previewallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data54.4
9Claude 3 Opusallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data53.5
10Nemotron-4 340B Instructallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data53
11DeepSeek V2 Chatallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data52.7
12Llama 3 70B Instructallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data52.3
13Reka Coreallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data52.3
14Gemini 1.5 Proallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data52.2
15Mistral Nemoallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data51.9
16DeepSeek Coder V2 Instructallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data51.5
17yi-largeallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data51
18Yi 1.5 34B Chatallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data50.3
19Qwen2 72B Instructallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data49.5
20Command R Plusallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data49.2
21Gemma 2 9B ITallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data49
22Gemini 1.5 Flashallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data48.7
23Claude 3 Sonnetallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data47.1
24GLM-4-9b-chatallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data46.3
25Mistral Largeallenai/WildBench, WB_score.Information/Advice seeking, 2402· WildBench direct data46.1
26Claude 3 Haikuallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data45.3
27Command R+allenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data44.1
28Mixtral 8x7B Instruct v0.1allenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data41.9
29Starling LM 7B Betaallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data41.9
30Reka Flashallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data41.5
31Tulu 2 DPO 70Ballenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data40.7
32Mistral 7B Instruct v0.2allenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data40.1
33Gemma 2 2B ITallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data39.9
34Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data39.8
35Llama 3 8B Instructallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data39.3
36Llama-2-70b-chat-hfallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data38.3
37GPT-3.5 Turboallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data36.5
38Reka Edgeallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data34.4
39Llama-2-7b-chat-hfallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data27.7
40Gemma 7B ITallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data12.7
41Gemma 2B ITallenai/WildBench, WB_score.Information/Advice seeking· WildBench direct data-2.1

Data credits: WildBench, Allen Institute for AI