Skip to content

Benchmark

Arena ELO - Math

Math benchmarks measure how well models solve maths problems. Higher scores are better.

Models measured
350
Scores recorded
405
Sources
1
IQ weight
1.24

Score distribution

1,316
Mean
1,367.4
Median
152.6
Std dev
1,089
10th percentile
1,474.8
90th percentile
837.8
Min
837.81,547.6
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.24 for chat models, measured over 350 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
351Starling LM 7B Beta· LMArena data1,124.5
352Granite 3.0 2B Instruct· LMArena data1,116.9
353WizardLM 70B· LMArena data1,116
354Yi 34B Chat· LMArena data1,113.7
355Phi-3 Mini 4K Instruct· LMArena data1,111.4
356Snowflake Arctic Instruct· LMArena data1,108.7
357DeepSeek LLM 67B Chat· LMArena data1,107.7
358Tulu 2 DPO 70B· LMArena data1,107.3
359Gemma 1.1 7B IT· LMArena data1,106.9
360OpenChat 3.50106· LMArena data1,106.7
361SmolLM2 1.7B Instruct· LMArena data1,104.8
362OpenHermes 2.5 Mistral 7B· LMArena data1,097.6
363Llama 2 70B Chat· LMArena data1,091.3
364Phi-3 Mini 128K Instruct· LMArena data1,089
365Llama 3.2 1B Instruct· LMArena data1,085.7
366Mistral 7B Instruct v0.2· LMArena data1,085.4
367Starling LM 7B Alpha· LMArena data1,081.5
368Qwen1.5-7B-Chat· LMArena data1,079.7
369Dolphin 2.2.1 Mistral 7B· LMArena data1,076.6
370Llama 2 70B SteerLM Chat· LMArena data1,071.8
371OpenChat 3.5· LMArena data1,070.7
372Vicuna 33B· LMArena data1,070.7
373Qwen-14B-Chat· LMArena data1,067.7
374Gemma 7B IT· LMArena data1,066.4
375Llama 2 13B Chat· LMArena data1,064.9
376SOLAR 10.7B Instruct v1.0· LMArena data1,063.9
377Nous Hermes 2 Mixtral 8x7B DPO· LMArena data1,060.6
378Code Llama 34B Instruct· LMArena data1,056.4
379PaLM 2· LMArena data1,048.9
380Gemma 1.1 2B IT· LMArena data1,047.2
381MPT 30B Chat· LMArena data1,046.6
382Llama 2 7B Chat· LMArena data1,041.9
383Zephyr 7B Beta· LMArena data1,040.9
384StripedHyena Nous 7B· LMArena data1,033.1
385Guanaco 33B· LMArena data1,032.6
386Vicuna 13B· LMArena data1,030.4
387Mistral 7B Instruct· LMArena data1,027.5
388Qwen1.5-4B-Chat· LMArena data1,025.9
389OLMo 7B Instruct· LMArena data1,017.8
390WizardLM 13B· LMArena data1,017.1
391Gemma 2B IT· LMArena data1,008.7
392Vicuna 7B· LMArena data993.8
393ChatGLM3 6B· LMArena data989.1
394GPT4All 13B Snoozy· LMArena data940.9
395Koala 13B· LMArena data932.4
396ChatGLM 6B· LMArena data925.7
397RWKV-4 Raven 14B· LMArena data921.9
398MPT 7B Chat· LMArena data919.2
399ChatGLM2 6B· LMArena data915.2
400Alpaca 13B· LMArena data908.2

Showing 351–400 of 405 results.

Data credits: LMArena, CC BY 4.0 (licence)