Skip to content

Benchmark

Arena ELO - Non English

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
366
Scores recorded
423
Sources
1
IQ weight
1.20

Score distribution

1,288.7
Mean
1,334.4
Median
160.8
Std dev
1,032.5
10th percentile
1,449.6
90th percentile
781.3
Min
781.31,534.5
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.20 for chat models, measured over 366 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
351InternLM2.5 20B Chat· LMArena data1,104.3
352Granite 3.1 8B Instruct· LMArena data1,099.2
353Zephyr ORPO 141B A35B v0.1· LMArena data1,098.1
354Llama 3 8B Instruct· LMArena data1,097.6
355Starling LM 7B Beta· LMArena data1,097
356Qwen1.5-14B-Chat· LMArena data1,094.6
357Phi-3 Medium 4K Instruct· LMArena data1,093.9
358Gemini Pro· LMArena data1,093.8
359Yi 34B Chat· LMArena data1,079.2
360WizardLM 70B· LMArena data1,077.5
361Mixtral 8x7B Instruct v0.1· LMArena data1,076.9
362Snowflake Arctic Instruct· LMArena data1,074.2
363GPT-3.5 Turbo1106· LMArena data1,074.2
364DeepSeek LLM 67B Chat· LMArena data1,072.9
365OpenChat 3.50106· LMArena data1,072.1
366DBRX Instruct Preview· LMArena data1,071.2
367Granite 3.1 2B Instruct· LMArena data1,068.2
368Tulu 2 DPO 70B· LMArena data1,067.9
369OpenChat 3.5· LMArena data1,066.8
370Llama 2 70B SteerLM Chat· LMArena data1,063.5
371Qwen1.5-7B-Chat· LMArena data1,058.4
372Phi-3 Small 8K Instruct· LMArena data1,057.9
373Starling LM 7B Alpha· LMArena data1,055.6
374Vicuna 33B· LMArena data1,054.9
375Gemma 1.1 7B IT· LMArena data1,051.5
376Nous Hermes 2 Mixtral 8x7B DPO· LMArena data1,048.9
377Llama 2 70B Chat· LMArena data1,045.2
378Qwen-14B-Chat· LMArena data1,041.5
379Granite 3.0 8B Instruct· LMArena data1,036.7
380WizardLM 13B· LMArena data1,034.3
381Granite 3.0 2B Instruct· LMArena data1,032.5
382Vicuna 13B· LMArena data1,027.2
383OpenHermes 2.5 Mistral 7B· LMArena data1,024.3
384Llama 2 13B Chat· LMArena data1,023.9
385Phi-3 Mini 4K Instruct· LMArena data1,020.6
386Llama 3.2 3B· LMArena data1,018.8
387SOLAR 10.7B Instruct v1.0· LMArena data1,014
388Phi-3 Mini 4K InstructJune 2024· LMArena data1,012.6
389Dolphin 2.2.1 Mistral 7B· LMArena data1,012.4
390Mistral 7B Instruct v0.2· LMArena data1,011.9
391Code Llama 34B Instruct· LMArena data1,010.6
392Vicuna 7B· LMArena data1,002.9
393Phi-3 Mini 128K Instruct· LMArena data1,000.4
394Falcon 180B Chat· LMArena data999.7
395Gemma 7B IT· LMArena data998.6
396Code Llama 70B Instruct· LMArena data992.3
397MPT 30B Chat· LMArena data990.5
398Gemma 1.1 2B IT· LMArena data988.4
399Zephyr 7B Alpha· LMArena data987.8
400Zephyr 7B Beta· LMArena data987.6

Showing 351–400 of 423 results.

Data credits: LMArena, CC BY 4.0 (licence)