Skip to content

Benchmark

Arena ELO - Exclude Ties

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
366
Scores recorded
423
Sources
1
IQ weight
1.28

Score distribution

1,247.7
Mean
1,316.3
Median
217.2
Std dev
908.5
10th percentile
1,465.2
90th percentile
533.7
Min
533.71,565.6
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.28 for chat models, measured over 366 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
351Zephyr ORPO 141B A35B v0.1· LMArena data998.4
352Granite 3.1 8B Instruct· LMArena data992.2
353Qwen1.5-32B-Chat· LMArena data988.8
354Starling LM 7B Beta· LMArena data985.7
355Phi-3 Medium 4K Instruct· LMArena data978.5
356Mixtral 8x7B Instruct v0.1· LMArena data978.4
357Gemini Pro· LMArena data977.6
358Yi 34B Chat· LMArena data976.3
359Qwen1.5-14B-Chat· LMArena data975.9
360GPT-3.5 Turbo0125· LMArena data967.3
361WizardLM 70B· LMArena data966.4
362Tulu 2 DPO 70B· LMArena data962.8
363DBRX Instruct Preview· LMArena data958.8
364Llama 2 70B Chat· LMArena data955.2
365Nous Hermes 2 Mixtral 8x7B DPO· LMArena data951.1
366Granite 3.1 2B Instruct· LMArena data945.9
367Starling LM 7B Alpha· LMArena data944.9
368Vicuna 33B· LMArena data944.7
369OpenChat 3.50106· LMArena data943.9
370DeepSeek LLM 67B Chat· LMArena data942.4
371Phi-3 Small 8K Instruct· LMArena data936
372Llama 3.2 3B· LMArena data933.2
373Llama 2 70B SteerLM Chat· LMArena data930.4
374OpenChat 3.5· LMArena data929.3
375Snowflake Arctic Instruct· LMArena data927.3
376OpenHermes 2.5 Mistral 7B· LMArena data923.4
377GPT-3.5 Turbo1106· LMArena data923
378Gemma 1.1 7B IT· LMArena data918
379Mistral 7B Instruct v0.2· LMArena data914.9
380Granite 3.0 8B Instruct· LMArena data910
381Llama 2 13B Chat· LMArena data908.5
382SOLAR 10.7B Instruct v1.0· LMArena data907.9
383Qwen1.5-7B-Chat· LMArena data907.4
384Dolphin 2.2.1 Mistral 7B· LMArena data906.1
385WizardLM 13B· LMArena data904.1
386MPT 30B Chat· LMArena data895.2
387Zephyr 7B Beta· LMArena data888.3
388Code Llama 34B Instruct· LMArena data882.7
389Zephyr 7B Alpha· LMArena data875.3
390Phi-3 Mini 4K InstructJune 2024· LMArena data874.1
391Granite 3.0 2B Instruct· LMArena data872.9
392Code Llama 70B Instruct· LMArena data872.3
393Phi-3 Mini 4K Instruct· LMArena data872.1
394Guanaco 33B· LMArena data872
395Vicuna 13B· LMArena data868.3
396Falcon 180B Chat· LMArena data867.9
397Qwen-14B-Chat· LMArena data865.2
398Gemma 7B IT· LMArena data863.8
399Llama 2 7B Chat· LMArena data861.2
400Phi-3 Mini 128K Instruct· LMArena data844.6

Showing 351–400 of 423 results.

Data credits: LMArena, CC BY 4.0 (licence)