Skip to content

Benchmark

OTIS Mock AIME

Math benchmarks measure how well models solve maths problems. Scores are percentages.

Models measured
196
Scores recorded
302
Sources
1
IQ weight
1.62

Score distribution

62.7%
Mean
71.1%
Median
32.9%
Std dev
6.5%
10th percentile
98.3%
90th percentile
0%
Min
0%100%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.62 for chat models, measured over 196 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
201o1low· Epoch AI Benchmarking Hub data53.3%
202Qwen3 4B Instruct 2507· Epoch AI Benchmarking Hub data52.2%
203DeepSeek R1 Distill Llama 70b· Epoch AI Benchmarking Hub data51.4%
204Gemini 3.5 Flash Liteminimal· Epoch AI Benchmarking Hub data51.1%
205GPT OSS 20bhigh· Epoch AI Benchmarking Hub data50.8%
206DeepSeek R1 Distill Qwen 14B· Epoch AI Benchmarking Hub data50.6%
207DeepSeek Chat· Epoch AI Benchmarking Hub data48.9%
208Claude Opus 4.5· Epoch AI Benchmarking Hub data48.1%
209O1 Minihigh· Epoch AI Benchmarking Hub data46.9%
210Claude Sonnet 3.716k· Epoch AI Benchmarking Hub data46.7%
211DeepSeek V4 Pronone· Epoch AI Benchmarking Hub data46.7%
212GPT 5minimal· Epoch AI Benchmarking Hub data46.7%
213GPT 5.4 Nanonone· Epoch AI Benchmarking Hub data46.7%
214GPT-5 nanolow· Epoch AI Benchmarking Hub data46.7%
215GPT-4.1 mini· Epoch AI Benchmarking Hub data44.7%
216O1 Minimedium· Epoch AI Benchmarking Hub data44.7%
217Gemini 3.1 Flash Litelow· Epoch AI Benchmarking Hub data44.4%
218o3 Minilow· Epoch AI Benchmarking Hub data44.4%
219DeepSeek R1 0528 Qwen3 8B· Epoch AI Benchmarking Hub data43.9%
220Qwen3.5 9B· Epoch AI Benchmarking Hub data42.8%
221Claude Opus 4· Epoch AI Benchmarking Hub data42.2%
222GPT OSS 20blow· Epoch AI Benchmarking Hub data40.3%
223Claude Opus 4.1· Epoch AI Benchmarking Hub data40%
224GPT-5.6 Lunanone· Epoch AI Benchmarking Hub data40%
225GPT 4.1· Epoch AI Benchmarking Hub data38.3%
226DeepSeek V3· Epoch AI Benchmarking Hub data37.8%
227Gemini 3.1 Flash Liteminimal· Epoch AI Benchmarking Hub data37.8%
228GPT 5.1none· Epoch AI Benchmarking Hub data37.8%
229GPT-4.5 Preview· Epoch AI Benchmarking Hub data37.8%
230Claude Haiku 4.5· Epoch AI Benchmarking Hub data35.8%
231Claude Sonnet 4.5· Epoch AI Benchmarking Hub data35.6%
232GPT-5 nanominimal· Epoch AI Benchmarking Hub data35.6%
233Mistral Medium 3· Epoch AI Benchmarking Hub data32.2%
234Gemini 2.0 Flash· Epoch AI Benchmarking Hub data31.1%
235O1 Preview· Epoch AI Benchmarking Hub data31.1%
236Mistral Small 3.2· Epoch AI Benchmarking Hub data30.3%
237Magistral Small 25092506· Epoch AI Benchmarking Hub data30%
238Claude Sonnet 4· Epoch AI Benchmarking Hub data28.9%
239GLM 5.2none· Epoch AI Benchmarking Hub data28.9%
240GPT-4.1 nano· Epoch AI Benchmarking Hub data28.9%
241Magistral Small 2509· Epoch AI Benchmarking Hub data28.1%
242GPT-5.4 mininone· Epoch AI Benchmarking Hub data26.7%
243MiniMax M3none· Epoch AI Benchmarking Hub data26.7%
244Qwen3 14Bnone· Epoch AI Benchmarking Hub data25.8%
245Qwen3 30B A3Bnone· Epoch AI Benchmarking Hub data25.6%
246GLM 4.7 Flashnone· Epoch AI Benchmarking Hub data25%
247gemini-1.5-pro-002· Epoch AI Benchmarking Hub data23.1%
248Qwen3 32Bnone· Epoch AI Benchmarking Hub data23.1%
249Gemma 3 27B IT· Epoch AI Benchmarking Hub data22.5%
250Qwen3 8Bnone· Epoch AI Benchmarking Hub data22.2%

Showing 201–250 of 302 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)