Skip to content

Benchmark

OTIS Mock AIME

Math benchmarks measure how well models solve maths problems. Scores are percentages.

Models measured
196
Scores recorded
302
Sources
1
IQ weight
1.62

Score distribution

62.7%
Mean
71.1%
Median
32.9%
Std dev
6.5%
10th percentile
98.3%
90th percentile
0%
Min
0%100%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.62 for chat models, measured over 196 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
251Claude Sonnet 3.7· Epoch AI Benchmarking Hub data21.9%
252DeepSeek R1 Distill Qwen 1.5B· Epoch AI Benchmarking Hub data21.4%
253Llama 4 Maverick· Epoch AI Benchmarking Hub data20.6%
254Qwen Plus· Epoch AI Benchmarking Hub data17.8%
255Gemma 3 12B IT· Epoch AI Benchmarking Hub data16.7%
256gemini-1.5-flash-002· Epoch AI Benchmarking Hub data16.3%
257Qwen Max· Epoch AI Benchmarking Hub data16.1%
258Phi 4· Epoch AI Benchmarking Hub data13.8%
259Grok 2· Epoch AI Benchmarking Hub data11.5%
260Llama 3.1 405B Instruct· Epoch AI Benchmarking Hub data9.7%
261Mistral Large2407· Epoch AI Benchmarking Hub data8.5%
262Qwen2.5 72B Instruct· Epoch AI Benchmarking Hub data8.1%
263Qwen3 1.7Bnone· Epoch AI Benchmarking Hub data8.1%
264Llama 4 Scout· Epoch AI Benchmarking Hub data7.8%
265Mistral Large2411· Epoch AI Benchmarking Hub data7.8%
266Gemma 3 4B IT· Epoch AI Benchmarking Hub data7.5%
267Qwen2.5 32B Instruct· Epoch AI Benchmarking Hub data7.4%
268GPT-4o mini· Epoch AI Benchmarking Hub data6.9%
269gemini-1.5-pro-001· Epoch AI Benchmarking Hub data6.8%
270GPT-4 Turbo· Epoch AI Benchmarking Hub data6.7%
271Mistral-Small-3· Epoch AI Benchmarking Hub data6.7%
272Claude 3.5 Sonnet· Epoch AI Benchmarking Hub data6.5%
273GPT 4o· Epoch AI Benchmarking Hub data6.4%
274Qwen Turbo· Epoch AI Benchmarking Hub data6.1%
275Mistral Small2503· Epoch AI Benchmarking Hub data5.8%
276Mistral Small· Epoch AI Benchmarking Hub data5.3%
277Llama 3.3 70B Instruct· Epoch AI Benchmarking Hub data5.1%
278Claude 3 Opus· Epoch AI Benchmarking Hub data4.7%
279gemini-1.5-flash-8b-001· Epoch AI Benchmarking Hub data4.6%
280Claude Haiku 3.5· Epoch AI Benchmarking Hub data4.3%
281Llama 3 70B Instruct· Epoch AI Benchmarking Hub data4.3%
282gemini-1.5-flash-001· Epoch AI Benchmarking Hub data3.9%
283Mistral Small 3.1· Epoch AI Benchmarking Hub data3.9%
284Llama 3.1 70B Instruct· Epoch AI Benchmarking Hub data3.6%
285Llama 3.2 90B Vision Instruct· Epoch AI Benchmarking Hub data2.6%
286Claude 2· Epoch AI Benchmarking Hub data2.5%
287Claude 3 Sonnet· Epoch AI Benchmarking Hub data2.5%
288Qwen2.5 7B Instruct· Epoch AI Benchmarking Hub data2.5%
289GPT-3.5 Turbo· Epoch AI Benchmarking Hub data2.2%
290Claude 2.1· Epoch AI Benchmarking Hub data1.9%
291Llama 3 8B Instruct· Epoch AI Benchmarking Hub data1.9%
292Mistral Large2402· Epoch AI Benchmarking Hub data1.9%
293Claude 3 Haiku· Epoch AI Benchmarking Hub data1.8%
294Llama 3.1 8B Instruct· Epoch AI Benchmarking Hub data1.7%
295Gemma 2 27B IT· Epoch AI Benchmarking Hub data1.4%
296Gemma 3 1b it· Epoch AI Benchmarking Hub data1.1%
297GPT 4· Epoch AI Benchmarking Hub data1.1%
298DeepSeek LLM 67B Chat· Epoch AI Benchmarking Hub data0.833%
299Gemma 2 9B IT· Epoch AI Benchmarking Hub data0.556%
300Llama 3.2 1B Instruct· Epoch AI Benchmarking Hub data0.556%

Showing 251–300 of 302 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)