Skip to content

Benchmark

Vals MGSM

Math benchmarks measure how well models solve maths problems. Scores are percentages.

Models measured
62
Scores recorded
67
Sources
1
IQ weight
2.55

Score distribution

88.8%
Mean
90.8%
Median
7.4%
Std dev
85.7%
10th percentile
93.3%
90th percentile
41.7%
Min
41.7%95.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.55 for chat models, measured over 62 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.5· Vals AI95.2%
2Claude Opus 4.1· Vals AI94.4%
3Claude Sonnet 4.5· Vals AI94.3%
4GPT 5.2· Vals AI94%
5Gemini 3 Pro Preview· Vals AI93.9%
6Claude Opus 4· Vals AI93.8%
7o4 Mini· Vals AI93.4%
8Gemini 3 Flash· Vals AI93.3%
9Claude Sonnet 4· Vals AI93%
10Claude Sonnet 3.7thinking· Vals AI93%
11GPT 5.1· Vals AI93%
12GPT 5· Vals AI92.8%
13Claude 3.5 Sonnet· Vals AI92.6%
14GPT-5 mini· Vals AI92.6%
15Qwen 3 235B A22B· Vals AI92.5%
16Llama 4 Maverick· Vals AI92.4%
17Claude Sonnet 3.7· Vals AI92.4%
18DeepSeek R1· Vals AI92.3%
19Claude Haiku 4.5thinking· Vals AI92.1%
20DeepSeek V3· Vals AI92.1%
21Qwen3 Max Preview· Vals AI92.1%
22GPT OSS 120B· Vals AI92%
23Qwen3 Max· Vals AI91.8%
24OpenAI o3· Vals AI91.7%
25Grok 3· Vals AI91.3%
26o3 Mini· Vals AI91.3%
27Llama 3.3 70B Instruct Turbo· Vals AI91.1%
28Kimi K2 Instruct· Vals AI90.9%
29Grok 4· Vals AI90.9%
30Claude Sonnet 4thinking· Vals AI90.9%
31DeepSeek V3.2· Vals AI90.9%
32Grok 4 Fast· Vals AI90.9%
33Mistral Medium 3· Vals AI90.9%
34GLM 4.5· Vals AI90.8%
35GPT 4o· Vals AI90.7%
36Grok 3 Mini Fast Betahigh, reasoning· Vals AI90.4%
37Grok 3 Mini Fast Betalow, reasoning· Vals AI90.4%
38Kimi K2 Thinking· Vals AI90.1%
39gemini-2.5-flash-preview-09-2025· Vals AI89.9%
40gemini-2.5-flash-preview-09-2025thinking· Vals AI89.8%
41GLM 4.6· Vals AI89.7%
42Gemini 2.5 Flash Lite Preview 09-2025· Vals AI89.5%
43Grok 4.1 Fast· Vals AI89.5%
44GPT-5 nano· Vals AI89.3%
45o1· Vals AI89.3%
46gemini-1.5-pro-002· Vals AI89.2%
47Gemini 2.0 Flash· Vals AI89%
48GPT OSS 20b· Vals AI89%
49Gemini 2.5 Flash Lite Preview 09-2025thinking· Vals AI88.4%
50GLM 4.7· Vals AI88.2%

Showing 1–50 of 67 results.