Skip to content

Benchmark

Vals MATH 500

Math benchmarks measure how well models solve maths problems. Scores are percentages. Higher is better.

Models measured
49
Scores recorded
54
Sources
1
IQ weight
2.52

Score distribution

83.5%
Mean
89%
Median
13.8%
Std dev
70.2%
10th percentile
94.8%
90th percentile
25.4%
Min
25.4%96.4%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.52 for chat models, measured over 49 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 3 Pro Preview· Vals AI96.4%
2Grok 4· Vals AI96.2%
3GPT 5· Vals AI96%
4Claude Opus 4.1· Vals AI95.4%
5Gemini 2.5 Pro· Vals AI95.2%
6GPT OSS 120B· Vals AI94.8%
7GPT-5 mini· Vals AI94.8%
8OpenAI o3· Vals AI94.6%
9Qwen 3 235B A22B· Vals AI94.6%
10GPT OSS 20b· Vals AI94.2%
11Grok 3 Mini Fast Betahigh, reasoning· Vals AI94.2%
12Kimi K2 Instruct· Vals AI94.2%
13o4 Mini· Vals AI94.2%
14GLM 4.5· Vals AI94%
15Claude Sonnet 4thinking· Vals AI93.8%
16GPT-5 nano· Vals AI93.8%
17DeepSeek R1· Vals AI92.2%
18gemini-2.5-flash-preview-04-17thinking· Vals AI91.8%
19o3 Mini· Vals AI91.8%
20Claude Sonnet 3.7thinking· Vals AI91.6%
21gemini-2.5-flash-preview-04-17· Vals AI91.6%
22Claude Opus 4· Vals AI90.4%
23o1· Vals AI90.4%
24Claude Sonnet 4· Vals AI90.3%
25Grok 3· Vals AI89.8%
26Gemini 2.0 Flash Experimental· Vals AI89%
27MiniMax M2.1· Vals AI89%
28DeepSeek V3· Vals AI88.6%
29Gemini 2.0 Flash· Vals AI88%
30GPT-4.1 mini· Vals AI88%
31GPT 4.1· Vals AI87.2%
32Mistral Medium 3· Vals AI87%
33Llama 4 Maverick· Vals AI85.2%
34Gemini 2.0 Flash Experimentalthinking· Vals AI84.6%
35gemini-1.5-pro-002· Vals AI82.8%
36GPT-4.1 nano· Vals AI80.2%
37Llama 4 Scout· Vals AI79.2%
38gemini-1.5-flash-002· Vals AI78.8%
39Grok 2· Vals AI78.4%
40Claude Sonnet 3.7· Vals AI76.8%
41Command A 03 2025· Vals AI76.2%
42GPT 4o· Vals AI75.2%
43Mistral Large2411· Vals AI74.4%
44Llama 3.3 70B Instruct Turbo· Vals AI73.4%
45GPT-4o mini· Vals AI72.6%
46Claude 3.5 Sonnet· Vals AI72.4%
47Llama 3.1 405B Instruct Turbo· Vals AI71.4%
48Mistral Small· Vals AI70.6%
49Grok 3 Mini Fast Betalow, reasoning· Vals AI70.2%
50Meta Llama 3.1 70B Instruct Turbo· Vals AI65%

Showing 1–50 of 54 results.