Skip to content

Benchmark

WildBench: gpt4t_reward.Math & Data Analysis.K=-1

Math benchmarks measure how well models solve maths problems. Higher scores are better.

Models measured
32
Scores recorded
32
Sources
1
IQ weight
1.33

Score distribution

-46.1
Mean
-51.2
Median
26.2
Std dev
-73.3
10th percentile
-10.5
90th percentile
-92.7
Min
-92.76.3
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.33 for chat models, measured over 32 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 4oallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data6.3
2Claude 3.5 Sonnetallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-2.5
3Gemini 1.5 Proallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-3.2
4Gemini 1.5 Flashallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-10.5
5GPT-4-0125-previewallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-14.8
6Claude 3 Opusallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-19.1
7Llama 3 70B Instructallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-20.5
8yi-largeallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-23.5
9Reka Coreallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-27.5
10Nemotron-4 340B Instructallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-28.2
11Yi 1.5 34B Chatallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-28.6
12Claude 3 Sonnetallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-32.8
13DeepSeek Coder V2 Instructallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-35.2
14Qwen2 72B Instructallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-37.2
15GLM-4-9b-chatallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-48.5
16Claude 3 Haikuallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-51.2
17Command R Plusallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-52.9
18Reka Flashallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-57.6
19Mistral Largeallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, 2402· WildBench direct data-57.7
20Llama 3 8B Instructallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-58.6
21Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-61.1
22Mixtral 8x7B Instruct v0.1allenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-61.6
23Starling LM 7B Betaallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-64.5
24Command R+allenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-64.9
25Tulu 2 DPO 70Ballenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-70.1
26GPT-3.5 Turboallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-70.8
27Reka Edgeallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-71.2
28Mistral 7B Instruct v0.2allenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-73
29Llama-2-70b-chat-hfallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-73.3
30Llama-2-7b-chat-hfallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-83.1
31Gemma 7B ITallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-85.6
32Gemma 2B ITallenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09· WildBench direct data-92.7

Data credits: WildBench, Allen Institute for AI