Skip to content

Benchmark

MATH Level 5

Math benchmarks measure how well models solve maths problems. Scores are percentages. Higher is better.

Models measured
221
Scores recorded
238
Sources
2
IQ weight
1.72

Score distribution

32.6%
Mean
22%
Median
30.4%
Std dev
2.3%
10th percentile
87.1%
90th percentile
0%
Min
0%98.1%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.72 for chat models, measured over 221 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
151Granite 3.0 8B Instruct· Hugging Face Open LLM Leaderboard data14.2%
152Phi-3 Mini 128K Instruct· Hugging Face Open LLM Leaderboard data14%
153gemma-2-9b· Hugging Face Open LLM Leaderboard data13.4%
154MN 12b Mag Mell R1· Hugging Face Open LLM Leaderboard data13%
155Llama 3 70b Instruct Abliterated v3.5· Hugging Face Open LLM Leaderboard data12.8%
156Mistral Nemo· Hugging Face Open LLM Leaderboard data12.7%
157Nous Hermes 2 Mixtral 8x7B DPO· Hugging Face Open LLM Leaderboard data12.2%
158Claude 2· Epoch AI Benchmarking Hub data11.7%
159DBRX-instruct· Epoch AI Benchmarking Hub data
Values from 2 sources
11.7%
160GPT-3.5 Turbo· Epoch AI Benchmarking Hub data11.6%
161Phi-3 Mini 4K Instruct· Hugging Face Open LLM Leaderboard data11.6%
162Open Mistral Nemo· Epoch AI Benchmarking Hub data10.8%
163Mixtral 8X7b V0.1· Hugging Face Open LLM Leaderboard data10.2%
164open-mixtral-8x7b· Epoch AI Benchmarking Hub data10%
165granite-3-1-8b-base· Hugging Face Open LLM Leaderboard data9.4%
166DeepSeek LLM 67B Chat· Hugging Face Open LLM Leaderboard data
Values from 2 sources
9.3%
167L3 8B Stheno V3.2· Hugging Face Open LLM Leaderboard data9.3%
168Qwen1.5-7B· Hugging Face Open LLM Leaderboard data9.3%
169Mixtral 8x7B Instruct v0.1· Epoch AI Benchmarking Hub data
Values from 2 sources
9.3%
170Granite 3.0 2B Instruct· Hugging Face Open LLM Leaderboard data9.2%
171Qwen2.5 1.5B· Hugging Face Open LLM Leaderboard data9.1%
172L3 8B Lunaris V1v1· Hugging Face Open LLM Leaderboard data9.1%
173Neural Daredevil 8B abliterated· Hugging Face Open LLM Leaderboard data9.1%
174Llama 3 8B Instruct· Hugging Face Open LLM Leaderboard data
Values from 2 sources
8.7%
175Aya Expanse 8B· Hugging Face Open LLM Leaderboard data8.6%
176NousResearch: Hermes 2 Pro - Llama-3 8B· Hugging Face Open LLM Leaderboard data8.4%
177OpenChat 3.5· Hugging Face Open LLM Leaderboard data7.6%
178gemma-7b· Hugging Face Open LLM Leaderboard data7.4%
179Qwen2 1.5b Instruct· Hugging Face Open LLM Leaderboard data7.2%
180granite-3.0-3b-a800m-instruct· Hugging Face Open LLM Leaderboard data7%
181Llama 3.2 1B Instruct· Hugging Face Open LLM Leaderboard data7%
182Qwen 2 (1.5B)· Hugging Face Open LLM Leaderboard data7%
183fimbulvetr-11b-v2· Hugging Face Open LLM Leaderboard data6.8%
184Llama 3.1 8B· Hugging Face Open LLM Leaderboard data6.6%
185Qwen1.5-7B-Chat· Hugging Face Open LLM Leaderboard data6.3%
186Yi 34B Chat· Hugging Face Open LLM Leaderboard data
Values from 2 sources
6.3%
187mistral-nemo-base-2407· Hugging Face Open LLM Leaderboard data6%
188Starcoder2 15b· Hugging Face Open LLM Leaderboard data6%
189Yi-34B200k· Hugging Face Open LLM Leaderboard data5.7%
190SOLAR 10.7B Instruct v1.0· Hugging Face Open LLM Leaderboard data5.7%
191Qwen1.5-4B· Hugging Face Open LLM Leaderboard data5.3%
192Yi-34B· Hugging Face Open LLM Leaderboard data5.1%
193OpenHermes 2.5 Mistral 7B· Hugging Face Open LLM Leaderboard data5.1%
194Gemma 1.1 7B IT· Hugging Face Open LLM Leaderboard data4.9%
195Meta Llama 3 8b· Hugging Face Open LLM Leaderboard data4.5%
196OpenHermes-2-Mistral-7B· Hugging Face Open LLM Leaderboard data4.5%
197Mistral (7B) Instruct v0.3· Hugging Face Open LLM Leaderboard data
Values from 2 sources
3.9%
198open-mistral-7b· Epoch AI Benchmarking Hub data3.7%
199Mistral-7B-OpenOrca· Hugging Face Open LLM Leaderboard data3.5%
200Llama-2-70b-chat-hf· Epoch AI Benchmarking Hub data
Values from 2 sources
3.3%

Showing 151–200 of 238 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)