Skip to content

Benchmark

AIME 2024

Math benchmarks measure how well models solve maths problems. Scores are percentages. Higher is better.

Models measured
50
Scores recorded
60
Sources
2
IQ weight
0.61

Score distribution

75.7%
Mean
80.4%
Median
18.4%
Std dev
48.1%
10th percentile
93.3%
90th percentile
13.1%
Min
13.1%96.6%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.61 for chat models, measured over 48 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT OSS 120Bofficial model card Table 3 · reasoning high · with tools; Table 3 does not name the exact tool · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.96.6%
2GPT OSS 20bofficial model card Table 3 · reasoning high · with tools; Table 3 does not name the exact tool · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.96%
3GPT OSS 120Bofficial model card Table 3 · reasoning high · no tools · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.95.8%
4Grok 3 Mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim95.8%
5o4 Mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim93.4%
6Grok 3· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim93.3%
7LongCat-Flash-Thinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim93.3%
8GPT OSS 20bofficial model card Table 3 · reasoning high · no tools · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.92.1%
9Gemini 2.5 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim92%
10OpenAI o3· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim91.6%
11DeepSeek R1· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim91.4%
12GLM 4.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim91%
13Ministral 14Blatest· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim89.8%
14GLM 4.5 Air· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim89.4%
15Gemini 2.5 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim88%
16GPT OSS 120Bofficial model card Table 3 · reasoning medium · with tools; Table 3 does not name the exact tool · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.87.9%
17o3 Mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim87.3%
18DeepSeek R1 Distill Llama 70b· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim86.7%
19DeepSeek R1 Zero· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim86.7%
20GPT OSS 20bofficial model card Table 3 · reasoning medium · with tools; Table 3 does not name the exact tool · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.86%
21MiniMax M1 80K· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.86%
22Ministral 8Blatest· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim86%
23O1 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim86%
24Qwen 3 235B A22B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim85.7%
25DeepSeek R1 Distill Qwen 32B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim83.3%
26DeepSeek R1 Distill Qwen 7B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim83.3%
27MiniMax M1 40K· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.83.3%
28Qwen3 32B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim81.4%
29Phi 4 Reasoning Plus· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim81.3%
30GPT OSS 120Bofficial model card Table 3 · reasoning medium · no tools · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.80.4%
31Qwen3 30B A3B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim80.4%
32Claude Sonnet 3.7· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim80%
33DeepSeek R1 Distill Llama 8B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim80%
34DeepSeek R1 Distill Qwen 14B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim80%
35GPT OSS 20bofficial model card Table 3 · reasoning medium · no tools · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.80%
36QwQ 32B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim79.5%
37Qwen3 14B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim79.3%
38Ministral 3Blatest· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim77.5%
39GPT OSS 120Bofficial model card Table 3 · reasoning low · with tools; Table 3 does not name the exact tool · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.75.4%
40Phi-4-reasoning· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim75.3%
41o1· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim74.3%
42Magistral Medium· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim73.6%
43Gemini 2.0 Flashthinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim73.3%
44longcat-flash-lite· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim72.2%
45Kimi K2 0905· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim72%
46Magistral Small 25092506· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim70.7%
47Kimi K2 Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim69.6%
48DeepSeek V3.1· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim66.3%
49GPT OSS 20bofficial model card Table 3 · reasoning low · with tools; Table 3 does not name the exact tool · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.61.2%
50GPT OSS 120Bofficial model card Table 3 · reasoning low · no tools · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.56.3%

Showing 1–50 of 60 results.

Data credits: Data by LLM Stats · BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.