Skip to content

Benchmark

WildBench: llama_reward.Math & Data Analysis.K=-1

Math benchmarks measure how well models solve maths problems. Higher scores are better.

Models measured
32
Scores recorded
32
Sources
1
IQ weight
1.56

Score distribution

37.9
Mean
40.8
Median
31.6
Std dev
6.6
10th percentile
69.1
90th percentile
-52.4
Min
-52.478.6
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.56 for chat models, measured over 32 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 4oallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data78.6
2Claude 3.5 Sonnetallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data77.8
3GPT-4 Turboallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data72.9
4Gemini 1.5 Proallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data69.1
5Llama 3 70B Instructallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data67.1
6Claude 3 Opusallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data65.9
7GPT-4-0125-previewallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data63.9
8Gemini 1.5 Flashallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data62
9yi-largeallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data61.7
10Nemotron-4 340B Instructallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data59.5
11Reka Coreallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data59.2
12Claude 3 Sonnetallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data58.8
13Qwen2 72B Instructallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data56.8
14Yi 1.5 34B Chatallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data56.1
15DeepSeek Coder V2 Instructallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data54.6
16GLM-4-9b-chatallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data40.8
17Claude 3 Haikuallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data40.7
18Reka Flashallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data37.8
19Mistral Largeallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf, 2402· WildBench direct data36.9
20Command R Plusallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data36.4
21Llama 3 8B Instructallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data29.3
22Mixtral 8x7B Instruct v0.1allenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data26.6
23Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data23.8
24Starling LM 7B Betaallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data20.9
25Tulu 2 DPO 70Ballenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data16.4
26Command R+allenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data15.2
27GPT-3.5 Turboallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data13.3
28Reka Edgeallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data10.9
29Mistral 7B Instruct v0.2allenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data6.6
30Gemma 7B ITallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data-21.8
31Llama-2-7b-chat-hfallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data-33.2
32Gemma 2B ITallenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf· WildBench direct data-52.4

Data credits: WildBench, Allen Institute for AI