Skip to content

Benchmark

Arena ELO Style-Controlled

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
366
Scores recorded
423
Sources
1
IQ weight
1.10

Score distribution

1,341.3
Mean
1,363.7
Median
121
Std dev
1,152.6
10th percentile
1,474.5
90th percentile
953.4
Min
953.41,527
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.10 for chat models, measured over 366 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
351Granite 3.1 8B Instruct· LMArena data1,208.7
352GPT-3.5 Turbo1106· LMArena data1,204.4
353Qwen1.5-32B-Chat· LMArena data1,204.2
354Gemma 2 2B IT· LMArena data1,200.2
355Phi-3 Medium 4K Instruct· LMArena data1,198.1
356Mixtral 8x7B Instruct v0.1· LMArena data1,197.3
357DBRX Instruct Preview· LMArena data1,195.7
358Qwen1.5-14B-Chat· LMArena data1,191.2
359InternLM2.5 20B Chat· LMArena data1,190.9
360DeepSeek LLM 67B Chat· LMArena data1,185.1
361WizardLM 70B· LMArena data1,184.9
362Yi 34B Chat· LMArena data1,183.9
363Granite 3.0 8B Instruct· LMArena data1,183.3
364OpenChat 3.5· LMArena data1,183.2
365OpenChat 3.50106· LMArena data1,182.9
366Gemma 1.1 7B IT· LMArena data1,182.7
367Snowflake Arctic Instruct· LMArena data1,180.2
368Granite 3.1 2B Instruct· LMArena data1,179
369Tulu 2 DPO 70B· LMArena data1,177.8
370OpenHermes 2.5 Mistral 7B· LMArena data1,176.1
371Vicuna 33B· LMArena data1,173.1
372Phi-3 Small 8K Instruct· LMArena data1,171.2
373Starling LM 7B Beta· LMArena data1,171.1
374Llama 2 70B Chat· LMArena data1,170.8
375Starling LM 7B Alpha· LMArena data1,167.6
376Llama 3.2 3B· LMArena data1,166.9
377Nous Hermes 2 Mixtral 8x7B DPO· LMArena data1,164.5
378Granite 3.0 2B Instruct· LMArena data1,156.8
379Llama 2 70B SteerLM Chat· LMArena data1,154.9
380QwQ 32B Preview· LMArena data1,154.3
381SOLAR 10.7B Instruct v1.0· LMArena data1,152.6
382Dolphin 2.2.1 Mistral 7B· LMArena data1,152.4
383MPT 30B Chat· LMArena data1,150.9
384WizardLM 13B· LMArena data1,149.7
385Mistral 7B Instruct v0.2· LMArena data1,149.5
386Falcon 180B Chat· LMArena data1,148.3
387Qwen1.5-7B-Chat· LMArena data1,144
388Phi-3 Mini 4K InstructJune 2024· LMArena data1,143.3
389Vicuna 13B· LMArena data1,141.7
390Llama 2 13B Chat· LMArena data1,141.6
391Qwen-14B-Chat· LMArena data1,139.5
392PaLM 2· LMArena data1,139.4
393Gemma 7B IT· LMArena data1,137.9
394Code Llama 34B Instruct· LMArena data1,137
395Zephyr 7B Beta· LMArena data1,131
396Phi-3 Mini 128K Instruct· LMArena data1,130
397Phi-3 Mini 4K Instruct· LMArena data1,128.2
398Guanaco 33B· LMArena data1,127.7
399Zephyr 7B Alpha· LMArena data1,127.1
400StripedHyena Nous 7B· LMArena data1,121.8

Showing 351–400 of 423 results.

Data credits: LMArena, CC BY 4.0 (licence)