Skip to content

Benchmark

MathArena BRUMO 2025

Math benchmarks measure how well models solve maths problems. Scores are percentages.

Models measured
33
Scores recorded
36
Sources
1
IQ weight
1.36

Score distribution

89.6%
Mean
92.5%
Median
9.1%
Std dev
80.8%
10th percentile
99.2%
90th percentile
65.8%
Min
65.8%100%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.36 for chat models, measured over 33 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
1Gemini 3 Flashhigh· MathArena data100%
2Step 3.5 Flash· MathArena data100%
3DeepSeek V3.2 Speciale· MathArena data99.2%
4GLM 5· MathArena data99.2%
5Gemini 3 Pro Previewpreview· MathArena data98.3%
6GPT 5.2high· MathArena data98.3%
7Kimi K2.5high· MathArena data98.3%
8Grok 4.1 Fastreasoning· MathArena data97.5%
9DeepSeek V3.2think· MathArena data96.7%
10DeepSeek V3.2 Expthink· MathArena data95.8%
11OpenAI o3high· MathArena data95.8%
12Grok 4· MathArena data95%
13GLM 4.6· MathArena data94.2%
14GPT 5.1high· MathArena data93.3%
15Kimi K2 Thinking· MathArena data93.3%
16DeepSeek R1· MathArena data92.5%
17GLM 4.5· MathArena data92.5%
18GPT OSS 120Bhigh· MathArena data92.5%
19GPT 5high· MathArena data91.7%
20Claude Sonnet 4.5think· MathArena data90.8%
21DeepSeek V3.1think· MathArena data90%
22Gemini 2.5 Pro· MathArena data90%
23GLM 4.5 Air· MathArena data90%
24GPT-5 minihigh· MathArena data90%
25GPT OSS 20bhigh· MathArena data86.7%
26o4 Minihigh· MathArena data86.7%
27Qwen 3 235B A22B· MathArena data86.7%
28Grok 3 Minihigh· MathArena data85%
29o4 Minimedium· MathArena data84.2%
30Gemini 2.5 Flashthinking· MathArena data83.3%
31Claude Opus 4think· MathArena data81.7%
32GPT-5 nanohigh· MathArena data80.8%
33Qwen3 30B A3B· MathArena data77.5%
34o4 Minilow· MathArena data66.7%
35Claude Sonnet 3.7think· MathArena data65.8%
36Grok 3 Minilow· MathArena data65.8%

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: MathArena (ETH SRI / INSAIT), MIT — cite dekoninck2026matharena (licence)