Skip to content

Benchmark

MathArena SMT 2025

Math benchmarks measure how well models solve maths problems. Scores are percentages.

Models measured
32
Scores recorded
36
Sources
1
IQ weight
2.01

Score distribution

84.1%
Mean
85.8%
Median
8.7%
Std dev
75.5%
10th percentile
92%
90th percentile
56.6%
Min
56.6%95.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.01 for chat models, measured over 32 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
1GPT 5.2xhigh· MathArena data95.9%
2Gemini 3 Pro Previewpreview· MathArena data93.4%
3Gemini 3 Flashhigh· MathArena data92.9%
4GPT 5high· MathArena data92%
5GPT 5.2high· MathArena data92%
6Step 3.5 Flash· MathArena data91.5%
7GLM 5· MathArena data91%
8GPT 5.1high· MathArena data91%
9Kimi K2 Thinking· MathArena data91%
10GLM 4.6· MathArena data90.6%
11Kimi K2.5high· MathArena data90.6%
12DeepSeek V3.2 Speciale· MathArena data89.2%
13GPT-5 minihigh· MathArena data89.2%
14o4 Minihigh· MathArena data88.7%
15DeepSeek V3.2think· MathArena data87.7%
16GPT OSS 120Bhigh· MathArena data87.7%
17OpenAI o3high· MathArena data87.7%
18Grok 4· MathArena data85.8%
19GPT-5 nanohigh· MathArena data85.4%
20DeepSeek V3.2 Expthink· MathArena data84.9%
21Gemini 2.5 Pro· MathArena data84.9%
22Grok 4.1 Fastreasoning· MathArena data84.6%
23Claude Sonnet 4.5think· MathArena data84%
24DeepSeek V3.1think· MathArena data84%
25DeepSeek R1· MathArena data83%
26GPT OSS 20bhigh· MathArena data83%
27GLM 4.5· MathArena data82.1%
28o4 Minimedium· MathArena data79.7%
29Grok 3 Minihigh· MathArena data78.8%
30GLM 4.5 Air· MathArena data77.4%
31Qwen 3 235B A22B· MathArena data76.9%
32Gemini 2.5 Flashthinking· MathArena data75.5%
33o4 Minilow· MathArena data68.9%
34Qwen3 30B A3B· MathArena data67.9%
35Grok 3 Minilow· MathArena data63.7%
36Claude Sonnet 3.7think· MathArena data56.6%

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: MathArena (ETH SRI / INSAIT), MIT — cite dekoninck2026matharena (licence)