Benchmark
WildBench: haiku_reward.Math & Data Analysis.K=-1
Math benchmarks measure how well models solve maths problems. Higher scores are better.
- Models measured
- 32
- Scores recorded
- 32
- Sources
- 1
- IQ weight
- 1.15
Score distribution
- -1.3
- Mean
- -0.873
- Median
- 38.1
- Std dev
- -47.7
- 10th percentile
- 46.3
- 90th percentile
- -77.9
- Min
This benchmark has a weight of 1.15 for chat models, measured over 32 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | GPT 4oallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 58 |
| 2 | Claude 3.5 Sonnetallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 51.2 |
| 3 | Gemini 1.5 Proallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 46.4 |
| 4 | GPT-4 Turboallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 46.3 |
| 5 | Claude 3 Opusallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 37.8 |
| 6 | Llama 3 70B Instructallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 37 |
| 7 | GPT-4-0125-previewallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 35.8 |
| 8 | yi-largeallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 34.5 |
| 9 | Gemini 1.5 Flashallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 32.1 |
| 10 | Nemotron-4 340B Instructallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 30.5 |
| 11 | Reka Coreallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 27.9 |
| 12 | Yi 1.5 34B Chatallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 25.8 |
| 13 | Claude 3 Sonnetallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 25.5 |
| 14 | DeepSeek Coder V2 Instructallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 20.4 |
| 15 | Qwen2 72B Instructallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | 18.5 |
| 16 | GLM-4-9b-chatallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -0.873 |
| 17 | Command R Plusallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -10.1 |
| 18 | Mistral Largeallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307, 2402· WildBench direct data | -10.9 |
| 19 | Reka Flashallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -16.1 |
| 20 | Starling LM 7B Betaallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -20.6 |
| 21 | Nous Hermes 2 Mixtral 8x7B DPOallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -21.3 |
| 22 | Mixtral 8x7B Instruct v0.1allenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -22.1 |
| 23 | Llama 3 8B Instructallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -23.6 |
| 24 | Command R+allenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -33.6 |
| 25 | Tulu 2 DPO 70Ballenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -35.4 |
| 26 | GPT-3.5 Turboallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -36.4 |
| 27 | Reka Edgeallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -40.2 |
| 28 | Mistral 7B Instruct v0.2allenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -43.6 |
| 29 | Llama-2-70b-chat-hfallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -47.7 |
| 30 | Llama-2-7b-chat-hfallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -62.6 |
| 31 | Gemma 7B ITallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -65.6 |
| 32 | Gemma 2B ITallenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307· WildBench direct data | -77.9 |
Data credits: WildBench, Allen Institute for AI