Skip to content

Benchmark

Beyond AIME

Math benchmarks measure how well models solve maths problems. Scores are percentages.

Models measured
7
Scores recorded
7
Sources
1
IQ weight
1.05

Score distribution

73.6%
Mean
69.1%
Median
10.1%
Std dev
66.9%
10th percentile
87%
90th percentile
58.3%
Min
58.3%88%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.05 for chat models, measured over 7 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Doubao Seed 2.1 Turbo· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim88%
2Doubao Seed 2.1 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim87%
3Doubao Seed 1.8· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim77%
4Sarvam 105B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim69.1%
5Seed 2.0 Mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim69%
6Mistral Medium 3.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim66.9%
7Sarvam-30B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim58.3%

Data credits: Data by LLM Stats