Skip to content

Benchmark

WildBench: WB_score.task_macro

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
41
Scores recorded
41
Sources
1
IQ weight
1.64

Score distribution

39.2
Mean
44.4
Median
15.2
Std dev
21.3
10th percentile
55.2
90th percentile
-9.7
Min
-9.759.5
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.64 for chat models, measured over 41 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Athene 70Ballenai/WildBench, WB_score.task_macro· WildBench direct data59.5
2GPT 4oallenai/WildBench, WB_score.task_macro· WildBench direct data59.3
3GPT-4o miniallenai/WildBench, WB_score.task_macro· WildBench direct data57.1
4Mistral Large 2allenai/WildBench, WB_score.task_macro· WildBench direct data55.6
5GPT-4 Turboallenai/WildBench, WB_score.task_macro· WildBench direct data55.2
6Claude 3.5 Sonnetallenai/WildBench, WB_score.task_macro· WildBench direct data54.7
7DeepSeek V2 Chatallenai/WildBench, WB_score.task_macro· WildBench direct data54
8Gemma 2 9B IT SimPOallenai/WildBench, WB_score.task_macro· WildBench direct data53.3
9Gemini 1.5 Proallenai/WildBench, WB_score.task_macro· WildBench direct data53
10GPT-4-0125-previewallenai/WildBench, WB_score.task_macro· WildBench direct data52.3
11Claude 3 Opusallenai/WildBench, WB_score.task_macro· WildBench direct data51.7
12yi-largeallenai/WildBench, WB_score.task_macro· WildBench direct data48.9
13Gemini 1.5 Flashallenai/WildBench, WB_score.task_macro· WildBench direct data48.9
14Llama 3 70B Instructallenai/WildBench, WB_score.task_macro· WildBench direct data47.8
15Nemotron-4 340B Instructallenai/WildBench, WB_score.task_macro· WildBench direct data47.7
16DeepSeek Coder V2 Instructallenai/WildBench, WB_score.task_macro· WildBench direct data47.4
17Reka Coreallenai/WildBench, WB_score.task_macro· WildBench direct data45.9
18Yi 1.5 34B Chatallenai/WildBench, WB_score.task_macro· WildBench direct data45.6
19Claude 3 Sonnetallenai/WildBench, WB_score.task_macro· WildBench direct data45.5
20Qwen2 72B Instructallenai/WildBench, WB_score.task_macro· WildBench direct data44.5
21Mistral Nemoallenai/WildBench, WB_score.task_macro· WildBench direct data44.4
22Gemma 2 9B ITallenai/WildBench, WB_score.task_macro· WildBench direct data42.7
23GLM-4-9b-chatallenai/WildBench, WB_score.task_macro· WildBench direct data39.1
24Mistral Largeallenai/WildBench, WB_score.task_macro, 2402· WildBench direct data38.9
25Claude 3 Haikuallenai/WildBench, WB_score.task_macro· WildBench direct data38.9
26Command R Plusallenai/WildBench, WB_score.task_macro· WildBench direct data36.8
27Mixtral 8x7B Instruct v0.1allenai/WildBench, WB_score.task_macro· WildBench direct data31.5
28Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, WB_score.task_macro· WildBench direct data30.7
29Reka Flashallenai/WildBench, WB_score.task_macro· WildBench direct data30.4
30Starling LM 7B Betaallenai/WildBench, WB_score.task_macro· WildBench direct data30.2
31GPT-3.5 Turboallenai/WildBench, WB_score.task_macro· WildBench direct data30
32Command R+allenai/WildBench, WB_score.task_macro· WildBench direct data29.5
33Llama 3 8B Instructallenai/WildBench, WB_score.task_macro· WildBench direct data29.2
34Tulu 2 DPO 70Ballenai/WildBench, WB_score.task_macro· WildBench direct data28
35Gemma 2 2B ITallenai/WildBench, WB_score.task_macro· WildBench direct data27.8
36Mistral 7B Instruct v0.2allenai/WildBench, WB_score.task_macro· WildBench direct data25.6
37Reka Edgeallenai/WildBench, WB_score.task_macro· WildBench direct data21.3
38Llama-2-70b-chat-hfallenai/WildBench, WB_score.task_macro· WildBench direct data20.7
39Llama-2-7b-chat-hfallenai/WildBench, WB_score.task_macro· WildBench direct data8.3
40Gemma 7B ITallenai/WildBench, WB_score.task_macro· WildBench direct data6.6
41Gemma 2B ITallenai/WildBench, WB_score.task_macro· WildBench direct data-9.7

Data credits: WildBench, Allen Institute for AI