Skip to content

Benchmark

Arena ELO - Industry Mathematical

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
348
Scores recorded
402
Sources
1
IQ weight
1.25

Score distribution

1,321.4
Mean
1,367.7
Median
154.1
Std dev
1,094.5
10th percentile
1,481.4
90th percentile
833.6
Min
833.61,566.4
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.25 for chat models, measured over 348 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
351Gemini ProDev API· LMArena data1,123.9
352SmolLM2 1.7B Instruct· LMArena data1,122.2
353Yi 34B Chat· LMArena data1,116
354Phi-3 Mini 4K Instruct· LMArena data1,109.4
355Qwen1.5-7B-Chat· LMArena data1,107.6
356WizardLM 70B· LMArena data1,107.6
357Tulu 2 DPO 70B· LMArena data1,099.1
358Llama 3.2 1B Instruct· LMArena data1,098.2
359Gemma 1.1 7B IT· LMArena data1,098
360Snowflake Arctic Instruct· LMArena data1,097.5
361OpenHermes 2.5 Mistral 7B· LMArena data1,097.2
362OpenChat 3.50106· LMArena data1,094.5
363DeepSeek LLM 67B Chat· LMArena data1,093.2
364Llama 2 70B Chat· LMArena data1,089.6
365Mistral 7B Instruct v0.2· LMArena data1,085.6
366Phi-3 Mini 128K Instruct· LMArena data1,083
367Starling LM 7B Alpha· LMArena data1,082.3
368Vicuna 33B· LMArena data1,076.9
369Qwen-14B-Chat· LMArena data1,073.4
370OpenChat 3.5· LMArena data1,069.4
371Llama 2 13B Chat· LMArena data1,069
372Llama 2 70B SteerLM Chat· LMArena data1,065.2
373Gemma 7B IT· LMArena data1,061.5
374SOLAR 10.7B Instruct v1.0· LMArena data1,056.7
375Dolphin 2.2.1 Mistral 7B· LMArena data1,055.5
376Code Llama 34B Instruct· LMArena data1,055.2
377MPT 30B Chat· LMArena data1,053.4
378Nous Hermes 2 Mixtral 8x7B DPO· LMArena data1,050.1
379Zephyr 7B Beta· LMArena data1,044.6
380PaLM 2· LMArena data1,043.3
381Gemma 1.1 2B IT· LMArena data1,043
382Llama 2 7B Chat· LMArena data1,038.9
383Vicuna 13B· LMArena data1,022
384StripedHyena Nous 7B· LMArena data1,021.3
385Guanaco 33B· LMArena data1,020.2
386OLMo 7B Instruct· LMArena data1,018.2
387WizardLM 13B· LMArena data1,015.3
388Mistral 7B Instruct· LMArena data1,009.4
389Gemma 2B IT· LMArena data1,004.4
390Qwen1.5-4B-Chat· LMArena data998.5
391Vicuna 7B· LMArena data980.2
392ChatGLM3 6B· LMArena data964.4
393Koala 13B· LMArena data928.8
394RWKV-4 Raven 14B· LMArena data919.2
395ChatGLM 6B· LMArena data912.2
396MPT 7B Chat· LMArena data907.1
397OpenAssistant Pythia 12B· LMArena data890.9
398Alpaca 13B· LMArena data879.5
399FastChat-T5 3B· LMArena data851.5
400Dolly 2.0 12B· LMArena data840.1

Showing 351–400 of 402 results.

Data credits: LMArena, CC BY 4.0 (licence)