Skip to content

Benchmark

Omni-MATH

Math benchmarks measure how well models solve maths problems. Scores are percentages. Higher is better.

Models measured
10
Scores recorded
98
Sources
2
IQ weight
1.43

Score distribution

35.4%
Mean
30.3%
Median
18.9%
Std dev
15.7%
10th percentile
65.4%
90th percentile
8.9%
Min
8.9%82.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.43 for chat models, measured over 10 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1O1 MiniDifficulty: 1-3· Omni-MATH82.2%
2Phi 4 Reasoning Plus· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim81.9%
3O1 PreviewDifficulty: 1-3· Omni-MATH80.1%
4Phi-4-reasoning· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim76.6%
5Qwen2.5 Math 72B InstructDifficulty: 1-3· Omni-MATH71%
6GPT 4oDifficulty: 1-3· Omni-MATH68.4%
7O1 MiniPrecalculus· Omni-MATH68.2%
8O1 MiniAlgebra· Omni-MATH67.8%
9Claude 3.5 SonnetDifficulty: 1-3· Omni-MATH66.2%
10Qwen2.5 Math 7B InstructDifficulty: 1-3· Omni-MATH66.2%
11DeepSeek Coder V2 InstructDifficulty: 1-3· Omni-MATH65.4%
12O1 MiniDifficulty: 3-5· Omni-MATH63.1%
13O1 MiniNumber Theory· Omni-MATH61.7%
14O1 MiniCalculus· Omni-MATH60.9%
15O1 Mini· Omni-MATH60.5%
16O1 MiniApplied Mathematics· Omni-MATH60.5%
17O1 PreviewAlgebra· Omni-MATH57.7%
18O1 PreviewPrecalculus· Omni-MATH57.5%
19deepseek-coder-v2-lite-instructDifficulty: 1-3· Omni-MATH55.9%
20O1 PreviewCalculus· Omni-MATH53.9%
21O1 PreviewApplied Mathematics· Omni-MATH53.4%
22O1 Preview· Omni-MATH52.6%
23O1 MiniGeometry· Omni-MATH51.5%
24O1 PreviewDifficulty: 3-5· Omni-MATH50.8%
25O1 PreviewNumber Theory· Omni-MATH49.7%
26O1 MiniDifficulty: 5-8· Omni-MATH49.1%
27Qwen2.5 Math 72B InstructAlgebra· Omni-MATH43.3%
28O1 PreviewGeometry· Omni-MATH43.1%
29O1 MiniDifficulty: 8-10· Omni-MATH42.7%
30Qwen2.5 Math 72B InstructPrecalculus· Omni-MATH42.5%
31O1 PreviewDifficulty: 5-8· Omni-MATH42.3%
32Qwen2.5 Math 72B InstructCalculus· Omni-MATH39.8%
33GPT 4oPrecalculus· Omni-MATH39.8%
34Qwen2.5 Math 7B InstructAlgebra· Omni-MATH39.4%
35O1 PreviewDifficulty: 8-10· Omni-MATH37.7%
36O1 MiniDiscrete Mathematics· Omni-MATH37.7%
37Qwen2.5 Math 7B InstructPrecalculus· Omni-MATH37.5%
38Qwen2.5 Math 72B Instruct· Omni-MATH36.2%
39GPT 4oAlgebra· Omni-MATH36.1%
40DeepSeek Coder V2 InstructPrecalculus· Omni-MATH35.2%
41Qwen2.5 Math 72B InstructNumber Theory· Omni-MATH34.3%
42Qwen2.5 Math 72B InstructApplied Mathematics· Omni-MATH33.4%
43Qwen2.5 Math 7B Instruct· Omni-MATH33.2%
44Qwen2.5 Math 72B InstructDifficulty: 3-5· Omni-MATH31.4%
45O1 PreviewDiscrete Mathematics· Omni-MATH31.3%
46Qwen2.5 Math 7B InstructCalculus· Omni-MATH31.3%
47GPT 4o· Omni-MATH30.5%
48Qwen2.5 Math 7B InstructApplied Mathematics· Omni-MATH30.4%
49Claude 3.5 SonnetAlgebra· Omni-MATH30.3%
50DeepSeek Coder V2 InstructAlgebra· Omni-MATH30.2%

Showing 1–50 of 98 results.

Data credits: Data by LLM Stats