Skip to content

Benchmark

WildBench: mixture_of_rewards.Math & Data Analysis.K=-1

Math benchmarks measure how well models solve maths problems. Higher scores are better.

Models measured
30
Scores recorded
30
Sources
1
IQ weight
1.28

Score distribution

-3.2
Mean
-2.8
Median
31.6
Std dev
-36.7
10th percentile
28.3
90th percentile
-74.4
Min
-74.447.6
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.28 for chat models, measured over 30 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 4oallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data47.6
2Claude 3.5 Sonnetallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data42.2
3Gemini 1.5 Proallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data37.4
4GPT-4-0125-previewallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data28.3
5Claude 3 Opusallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data28.2
6Llama 3 70B Instructallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data27.9
7Gemini 1.5 Flashallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data27.8
8yi-largeallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data24.2
9Nemotron-4 340B Instructallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data20.6
10Reka Coreallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data19.9
11Yi 1.5 34B Chatallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data17.8
12Claude 3 Sonnetallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data17.2
13DeepSeek Coder V2 Instructallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data13.2
14Qwen2 72B Instructallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data12.7
15GLM-4-9b-chatallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-2.8
16Command R Plusallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-8.9
17Mistral Largeallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf, 2402· WildBench direct data-10.6
18Reka Flashallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-11.9
19Llama 3 8B Instructallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-17.6
20Mixtral 8x7B Instruct v0.1allenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-19
21Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-19.5
22Starling LM 7B Betaallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-21.4
23Command R+allenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-27.8
24Tulu 2 DPO 70Ballenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-29.7
25GPT-3.5 Turboallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-31.3
26Reka Edgeallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-33.5
27Mistral 7B Instruct v0.2allenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-36.7
28Gemma 7B ITallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-57.7
29Llama-2-7b-chat-hfallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-59.6
30Gemma 2B ITallenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf· WildBench direct data-74.4

Data credits: WildBench, Allen Institute for AI