Skip to content

Benchmark

MathArena HMMT Feb 2025

Math benchmarks measure how well models solve maths problems. Scores are percentages.

Models measured
41
Scores recorded
48
Sources
1
IQ weight
1.98

Score distribution

68.9%
Mean
76.7%
Median
27.6%
Std dev
28.3%
10th percentile
97.5%
90th percentile
1.7%
Min
1.7%100%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.98 for chat models, measured over 41 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
1GPT 5.2xhigh· MathArena data100%
2GPT 5.2high· MathArena data98.3%
3Step 3.5 Flash· MathArena data98.3%
4DeepSeek V3.2 Speciale· MathArena data97.5%
5Gemini 3 Flashhigh· MathArena data97.5%
6Gemini 3 Pro Previewpreview· MathArena data97.5%
7GLM 5· MathArena data97.5%
8Grok 4· MathArena data95%
9GLM 4.6· MathArena data93.3%
10GPT 5.1high· MathArena data93.3%
11Kimi K2 Thinking· MathArena data93.3%
12Kimi K2.5high· MathArena data93.3%
13DeepSeek V3.2think· MathArena data92.5%
14DeepSeek V3.2 Expthink· MathArena data90%
15GPT OSS 120Bhigh· MathArena data90%
16Grok 4.1 Fastreasoning· MathArena data90%
17GPT-5 minihigh· MathArena data89.2%
18GPT 5high· MathArena data88.3%
19DeepSeek V3.1think· MathArena data85.8%
20o4 Minihigh· MathArena data83.3%
21Gemini 2.5 Pro· MathArena data82.5%
22GLM 4.5· MathArena data77.5%
23OpenAI o3high· MathArena data77.5%
24DeepSeek R1· MathArena data76.7%
25GPT OSS 20bhigh· MathArena data76.7%
26GPT-5 nanohigh· MathArena data74.2%
27Grok 3 Minihigh· MathArena data74.2%
28GLM 4.5 Air· MathArena data69.2%
29Claude Sonnet 4.5think· MathArena data67.5%
30o3 Minihigh· MathArena data67.5%
31o4 Minimedium· MathArena data67.5%
32Gemini 2.5 Flashthinking· MathArena data64.2%
33Qwen 3 235B A22B· MathArena data62.5%
34Claude Opus 4think· MathArena data60%
35o3 Minimedium· MathArena data53.3%
36Grok 3 Minilow· MathArena data50.8%
37Qwen3 30B A3B· MathArena data50.8%
38o1medium· MathArena data48.3%
39o4 Minilow· MathArena data47.5%
40QwQ 32B· MathArena data47.5%
41Gemini 2.0 Flashthinking· MathArena data35.8%
42Claude Sonnet 3.7think· MathArena data31.7%
43o3 Minilow· MathArena data28.3%
44QwQ 32B Preview· MathArena data18.3%
45DeepSeek V3· MathArena data13.3%
46Gemini 2.0 Flash· MathArena data13.3%
47GPT 4o· MathArena data5.8%
48Claude 3.5 Sonnet· MathArena data1.7%

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: MathArena (ETH SRI / INSAIT), MIT — cite dekoninck2026matharena (licence)