Skip to content

Benchmark

Arena ELO - Longer Query

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
345
Scores recorded
401
Sources
1
IQ weight
1.34

Score distribution

1,323.7
Mean
1,357.1
Median
137.1
Std dev
1,102.6
10th percentile
1,475.3
90th percentile
951.6
Min
951.61,549
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.34 for chat models, measured over 345 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
351Gemma 2 2B IT· LMArena data1,129.7
352Llama 3 8B Instruct· LMArena data1,127.4
353Granite 3.0 8B Instruct· LMArena data1,121.8
354GPT-3.5 Turbo0125· LMArena data1,120.6
355Phi-3 Medium 4K Instruct· LMArena data1,119.9
356Qwen1.5-14B-Chat· LMArena data1,112.5
357DBRX Instruct Preview· LMArena data1,111.4
358Tulu 2 DPO 70B· LMArena data1,105.7
359Zephyr ORPO 141B A35B v0.1· LMArena data1,104.6
360Starling LM 7B Beta· LMArena data1,103.9
361Mixtral 8x7B Instruct v0.1· LMArena data1,102.6
362Llama 3.2 3B· LMArena data1,100.2
363WizardLM 70B· LMArena data1,096.8
364Yi 34B Chat· LMArena data1,093.5
365OpenChat 3.5· LMArena data1,092.7
366DeepSeek LLM 67B Chat· LMArena data1,091.7
367Qwen1.5-7B-Chat· LMArena data1,089.9
368Phi-3 Small 8K Instruct· LMArena data1,087.4
369OpenChat 3.50106· LMArena data1,085.4
370Starling LM 7B Alpha· LMArena data1,080.8
371Granite 3.0 2B Instruct· LMArena data1,070.1
372GPT-3.5 Turbo1106· LMArena data1,069.4
373Llama 2 13B Chat· LMArena data1,064
374Llama 2 70B Chat· LMArena data1,061.8
375Mistral 7B Instruct v0.2· LMArena data1,060
376SmolLM2 1.7B Instruct· LMArena data1,056.1
377Gemma 1.1 7B IT· LMArena data1,055.4
378OpenHermes 2.5 Mistral 7B· LMArena data1,054
379WizardLM 13B· LMArena data1,053.3
380Llama 3.2 1B Instruct· LMArena data1,049.5
381Vicuna 13B· LMArena data1,047.1
382Nous Hermes 2 Mixtral 8x7B DPO· LMArena data1,045.5
383Phi-3 Mini 4K Instruct· LMArena data1,043.8
384Zephyr 7B Beta· LMArena data1,041.4
385Phi-3 Mini 4K InstructJune 2024· LMArena data1,041.3
386Vicuna 7B· LMArena data1,035.3
387Vicuna 33B· LMArena data1,028.4
388Qwen-14B-Chat· LMArena data1,028.3
389Gemma 7B IT· LMArena data1,021.4
390Code Llama 34B Instruct· LMArena data1,013.2
391PaLM 2· LMArena data1,009.5
392Gemma 1.1 2B IT· LMArena data1,002.4
393Mistral 7B Instruct· LMArena data1,000.4
394Snowflake Arctic Instruct· LMArena data999
395Llama 2 7B Chat· LMArena data998.8
396Llama 2 70B SteerLM Chat· LMArena data997.5
397Phi-3 Mini 128K Instruct· LMArena data995.5
398Qwen1.5-4B-Chat· LMArena data987.4
399StripedHyena Nous 7B· LMArena data982.3
400Gemma 2B IT· LMArena data980.4

Showing 351–400 of 401 results.

Data credits: LMArena, CC BY 4.0 (licence)