Skip to content

Benchmark

HELM Capabilities: Omni-MATH - Acc

Math benchmarks measure how well models solve maths problems. Scores are percentages. Higher is better.

Models measured
57
Scores recorded
59
Sources
1
IQ weight
1.57

Score distribution

40.5%
Mean
39.1%
Median
16.8%
Std dev
21%
10th percentile
64.7%
90th percentile
7.2%
Min
7.2%72.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.57 for chat models, measured over 57 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-5 mini· Stanford HELM specialized suites data docs72.2%
2o4 Mini· Stanford HELM specialized suites data docs72%
3Qwen3 235B A22B Instruct 2507· Stanford HELM specialized suites data docs71.8%
4OpenAI o3· Stanford HELM specialized suites data docs71.4%
5GPT OSS 120B· Stanford HELM specialized suites data docs68.8%
6Kimi K2 Instruct· Stanford HELM specialized suites data docs65.4%
7GPT 5· Stanford HELM specialized suites data docs64.7%
8Claude Opus 4thinking, 10k· Stanford HELM specialized suites data docs61.6%
9Grok 4· Stanford HELM specialized suites data docs60.3%
10Claude Sonnet 4thinking, 10k· Stanford HELM specialized suites data docs60.2%
11GPT OSS 20b· Stanford HELM specialized suites data docs56.5%
12Claude Haiku 4.5· Stanford HELM specialized suites data docs56.1%
13Gemini 3 Pro Preview· Stanford HELM specialized suites data docs55.6%
14Claude Sonnet 4.5· Stanford HELM specialized suites data docs55.3%
15GPT-5 nano· Stanford HELM specialized suites data docs54.7%
16Claude Sonnet 4setting not named by the source· Stanford HELM specialized suites data docs51.3%
17Claude Opus 4setting not named by the source· Stanford HELM specialized suites data docs51.1%
18GPT-4.1 mini· Stanford HELM specialized suites data docs49.1%
19Gemini 2.5 Flash-Lite· Stanford HELM specialized suites data docs48%
20GPT 4.1· Stanford HELM specialized suites data docs47.1%
21Qwen3 Next 80B A3Bthinking· Stanford HELM specialized suites data docs46.7%
22Grok 3· Stanford HELM specialized suites data docs46.4%
23GPT 5.1· Stanford HELM specialized suites data docs46.4%
24Gemini 2.0 Flash· Stanford HELM specialized suites data docs45.9%
25DeepSeek R1· Stanford HELM specialized suites data docs42.4%
26Llama 4 Maverick· Stanford HELM specialized suites data docs42.2%
27Gemini 2.5 Pro Preview 03-25· Stanford HELM specialized suites data docs41.6%
28Palmyra X5· Stanford HELM specialized suites data docs41.5%
29DeepSeek V3· Stanford HELM specialized suites data docs40.3%
30GLM 4.5 Air· Stanford HELM specialized suites data docs39.1%
31gemini-2.5-flash-preview-04-17· Stanford HELM specialized suites data docs38.4%
32Gemini 2.0 Flash-Lite Preview· Stanford HELM specialized suites data docs37.4%
33Llama 4 Scout· Stanford HELM specialized suites data docs37.3%
34GPT-4.1 nano· Stanford HELM specialized suites data docs36.7%
35gemini-1.5-pro-002· Stanford HELM specialized suites data docs36.4%
36Nova Premier 1.0· Stanford HELM specialized suites data docs35%
37Claude Sonnet 3.7· Stanford HELM specialized suites data docs33%
38Qwen2.5 72B Instruct Turbo· Stanford HELM specialized suites data docs33%
39Grok 3 Mini· Stanford HELM specialized suites data docs31.8%
40gemini-1.5-flash-002· Stanford HELM specialized suites data docs30.5%
41Granite 4.0 H Small· Stanford HELM specialized suites data docs29.6%
42Qwen2.5 7B Instruct Turbo· Stanford HELM specialized suites data docs29.4%
43GPT 4o· Stanford HELM specialized suites data docs29.3%
44Mistral Large2411· Stanford HELM specialized suites data docs28.1%
45GPT-4o mini· Stanford HELM specialized suites data docs28%
46Claude 3.5 Sonnet· Stanford HELM specialized suites data docs27.6%
47Llama 3.1 405B Instruct Turbo· Stanford HELM specialized suites data docs24.9%
48Mistral Small· Stanford HELM specialized suites data docs24.8%
49Nova Prov1· Stanford HELM specialized suites data docs24.2%
50Nova Litev1· Stanford HELM specialized suites data docs23.3%

Showing 1–50 of 59 results.

Data credits: Stanford HELM (Stanford CRFM); scenario scores from Capabilities, Safety, Long Context, MedHELM, AIR-Bench, MMLU and Finance