Skip to content

Benchmark

Arena ELO - Industry Writing and Literature and Language

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
365
Scores recorded
422
Sources
1
IQ weight
1.17

Score distribution

1,297.8
Mean
1,328.7
Median
142.8
Std dev
1,086
10th percentile
1,453.9
90th percentile
809.1
Min
809.11,540.8
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.17 for chat models, measured over 365 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
351GPT-3.5 Turbo0125· LMArena data1,139
352Granite 3.1 8B Instruct· LMArena data1,138.6
353InternLM2.5 20B Chat· LMArena data1,136
354Gemini Pro· LMArena data1,126.8
355Phi-3 Medium 4K Instruct· LMArena data1,126.4
356Tulu 2 DPO 70B· LMArena data1,123.4
357Mixtral 8x7B Instruct v0.1· LMArena data1,122.9
358DBRX Instruct Preview· LMArena data1,121.5
359Vicuna 33B· LMArena data1,118.8
360Qwen1.5-32B-Chat· LMArena data1,118.2
361OpenChat 3.5· LMArena data1,116.8
362Yi 34B Chat· LMArena data1,116.6
363Qwen1.5-14B-Chat· LMArena data1,112.4
364Starling LM 7B Beta· LMArena data1,111.2
365DeepSeek LLM 67B Chat· LMArena data1,110.7
366OpenChat 3.50106· LMArena data1,105.2
367Falcon 180B Chat· LMArena data1,105.1
368Nous Hermes 2 Mixtral 8x7B DPO· LMArena data1,105.1
369Snowflake Arctic Instruct· LMArena data1,100.9
370Starling LM 7B Alpha· LMArena data1,098.3
371WizardLM 13B· LMArena data1,098.2
372OpenHermes 2.5 Mistral 7B· LMArena data1,096
373Llama 3.2 3B· LMArena data1,095.6
374Llama 2 70B SteerLM Chat· LMArena data1,094.4
375Phi-3 Small 8K Instruct· LMArena data1,093.7
376Llama 2 70B Chat· LMArena data1,092.7
377Granite 3.1 2B Instruct· LMArena data1,092.1
378GPT-3.5 Turbo1106· LMArena data1,091
379SOLAR 10.7B Instruct v1.0· LMArena data1,087.9
380Dolphin 2.2.1 Mistral 7B· LMArena data1,086
381Zephyr 7B Beta· LMArena data1,085.7
382Zephyr 7B Alpha· LMArena data1,081.6
383MPT 30B Chat· LMArena data1,079.3
384Vicuna 13B· LMArena data1,078.9
385Granite 3.0 8B Instruct· LMArena data1,078.4
386Gemma 1.1 7B IT· LMArena data1,073.1
387Mistral 7B Instruct v0.2· LMArena data1,073
388Guanaco 33B· LMArena data1,066.3
389Qwen1.5-7B-Chat· LMArena data1,064.4
390Llama 2 13B Chat· LMArena data1,061.5
391Phi-3 Mini 4K InstructJune 2024· LMArena data1,056.7
392Phi-3 Mini 4K Instruct· LMArena data1,054.4
393Granite 3.0 2B Instruct· LMArena data1,050.6
394Qwen-14B-Chat· LMArena data1,049.9
395Vicuna 7B· LMArena data1,048.6
396Code Llama 34B Instruct· LMArena data1,048.3
397Phi-3 Mini 128K Instruct· LMArena data1,040.3
398Llama 2 7B Chat· LMArena data1,037.1
399StripedHyena Nous 7B· LMArena data1,034.7
400Gemma 7B IT· LMArena data1,033.8

Showing 351–400 of 422 results.

Data credits: LMArena, CC BY 4.0 (licence)