Skip to content

Benchmark

MATH-500

Math benchmarks measure how well models solve maths problems. Scores are percentages. Higher is better.

Models measured
27
Scores recorded
27
Sources
1
IQ weight
1.54

Score distribution

94.8%
Mean
96.2%
Median
3.5%
Std dev
90.2%
10th percentile
98.1%
90th percentile
83.9%
Min
83.9%99.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.54 for chat models, measured over 27 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1LongCat-Flash-Thinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim99.2%
2Sarvam 105B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim98.6%
3GLM 4.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim98.2%
4GLM 4.5 Air· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim98.1%
5Nemotron Nano 9b V2· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim97.8%
6Kimi K2 Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim97.4%
7Llama 3.1 Nemotron Ultra 253B v1· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim97%
8Sarvam-30B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim97%
9longcat-flash-lite· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim96.8%
10MiniMax M1 80K· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.96.8%
11Qwen3 14B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim96.8%
12Llama 3.3 Nemotron Super 49B v1· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim96.6%
13LongCat Flash Chat· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim96.4%
14Claude Sonnet 3.7· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim96.2%
15MiniMax M1 40K· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.96%
16DeepSeek R1 Zero· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim95.9%
17Aihubmix Phi 4 Mini (reasoning)· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim94.6%
18DeepSeek R1 Distill Llama 70b· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim94.5%
19DeepSeek R1 Distill Qwen 32B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim94.3%
20DeepSeek R1 Distill Qwen 14B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim93.9%
21DeepSeek R1 Distill Qwen 7B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim92.8%
22QwQ 32B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim90.6%
23QwQ 32B Preview· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim90.6%
24DeepSeek V3· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim90.2%
25O1 Mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim90%
26DeepSeek R1 Distill Llama 8B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim89.1%
27DeepSeek R1 Distill Qwen 1.5B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim83.9%

Data credits: Data by LLM Stats