Skip to content

Benchmark

Arena ELO - Industry Life and Physical and Social Science

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
364
Scores recorded
421
Sources
1
IQ weight
1.24

Score distribution

1,316.4
Mean
1,364.4
Median
158.9
Std dev
1,071.5
10th percentile
1,475.1
90th percentile
800.9
Min
800.91,549.3
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.24 for chat models, measured over 364 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
351Starling LM 7B Beta· LMArena data1,141.9
352Phi-3 Medium 4K Instruct· LMArena data1,137.9
353Granite 3.1 8B Instruct· LMArena data1,136.3
354Yi 34B Chat· LMArena data1,133.9
355Qwen1.5-32B-Chat· LMArena data1,132.2
356Llama 2 70B Chat· LMArena data1,119.8
357Gemini Pro· LMArena data1,118.5
358Mixtral 8x7B Instruct v0.1· LMArena data1,117.1
359Starling LM 7B Alpha· LMArena data1,109.7
360WizardLM 70B· LMArena data1,109.4
361Granite 3.1 2B Instruct· LMArena data1,108.1
362Phi-3 Small 8K Instruct· LMArena data1,107
363Llama 3.2 3B· LMArena data1,105.9
364Tulu 2 DPO 70B· LMArena data1,104.9
365Nous Hermes 2 Mixtral 8x7B DPO· LMArena data1,103.8
366Qwen1.5-14B-Chat· LMArena data1,102.5
367DBRX Instruct Preview· LMArena data1,099.9
368Vicuna 33B· LMArena data1,099.7
369Granite 3.0 8B Instruct· LMArena data1,099
370Llama 2 70B SteerLM Chat· LMArena data1,094.3
371GPT-3.5 Turbo0125· LMArena data1,087
372Llama 2 13B Chat· LMArena data1,081.8
373Snowflake Arctic Instruct· LMArena data1,081.8
374OpenChat 3.50106· LMArena data1,078.7
375Mistral 7B Instruct v0.2· LMArena data1,078.4
376Gemma 1.1 7B IT· LMArena data1,077.8
377Qwen1.5-7B-Chat· LMArena data1,076.2
378SOLAR 10.7B Instruct v1.0· LMArena data1,072.1
379Granite 3.0 2B Instruct· LMArena data1,071.5
380WizardLM 13B· LMArena data1,068.9
381Phi-3 Mini 4K InstructJune 2024· LMArena data1,068.1
382OpenChat 3.5· LMArena data1,067.3
383Code Llama 34B Instruct· LMArena data1,066.5
384DeepSeek LLM 67B Chat· LMArena data1,065.8
385Dolphin 2.2.1 Mistral 7B· LMArena data1,065.4
386OpenHermes 2.5 Mistral 7B· LMArena data1,065.3
387Phi-3 Mini 4K Instruct· LMArena data1,061.9
388Zephyr 7B Beta· LMArena data1,060
389Guanaco 33B· LMArena data1,059.4
390Llama 3.2 1B Instruct· LMArena data1,054.6
391GPT-3.5 Turbo1106· LMArena data1,051.2
392Llama 2 7B Chat· LMArena data1,046.9
393SmolLM2 1.7B Instruct· LMArena data1,046.2
394MPT 30B Chat· LMArena data1,038.3
395Vicuna 13B· LMArena data1,034.5
396Zephyr 7B Alpha· LMArena data1,028
397Phi-3 Mini 128K Instruct· LMArena data1,022.8
398Gemma 7B IT· LMArena data1,020.3
399Falcon 180B Chat· LMArena data1,018.6
400StripedHyena Nous 7B· LMArena data1,011.9

Showing 351–400 of 421 results.

Data credits: LMArena, CC BY 4.0 (licence)