Skip to content

Benchmark

Arena ELO - Industry Legal and Government

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
338
Scores recorded
394
Sources
1
IQ weight
1.34

Score distribution

1,329.7
Mean
1,368.1
Median
139.4
Std dev
1,104.4
10th percentile
1,475.2
90th percentile
895.4
Min
895.41,543.1
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.34 for chat models, measured over 338 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
351Llama 2 70B Chat· LMArena data1,113.9
352OpenChat 3.5· LMArena data1,112.9
353Gemini Pro· LMArena data1,107.8
354OpenChat 3.50106· LMArena data1,105.2
355DBRX Instruct Preview· LMArena data1,104.4
356Qwen1.5-7B-Chat· LMArena data1,103
357Phi-3 Small 8K Instruct· LMArena data1,102.5
358WizardLM 13B· LMArena data1,099.7
359OpenHermes 2.5 Mistral 7B· LMArena data1,099.6
360Vicuna 33B· LMArena data1,098.7
361Mistral 7B Instruct v0.2· LMArena data1,098.5
362Zephyr 7B Beta· LMArena data1,095
363Gemma 1.1 7B IT· LMArena data1,094
364Snowflake Arctic Instruct· LMArena data1,091.2
365DeepSeek LLM 67B Chat· LMArena data1,089.2
366Granite 3.0 8B Instruct· LMArena data1,083.9
367Granite 3.0 2B Instruct· LMArena data1,083.8
368SOLAR 10.7B Instruct v1.0· LMArena data1,082.6
369Llama 2 13B Chat· LMArena data1,079.8
370Phi-3 Mini 4K Instruct· LMArena data1,070.9
371Llama 3.2 1B Instruct· LMArena data1,070.6
372Phi-3 Mini 4K InstructJune 2024· LMArena data1,068.4
373Llama 2 7B Chat· LMArena data1,064.8
374Code Llama 34B Instruct· LMArena data1,064.2
375Vicuna 13B· LMArena data1,056.9
376GPT-3.5 Turbo1106· LMArena data1,056.2
377Qwen-14B-Chat· LMArena data1,050.6
378OLMo 7B Instruct· LMArena data1,046.6
379Gemma 7B IT· LMArena data1,044.3
380Vicuna 7B· LMArena data1,031.9
381Mistral 7B Instruct· LMArena data1,024.4
382Gemma 2B IT· LMArena data1,019.5
383Qwen1.5-4B-Chat· LMArena data1,010.2
384Phi-3 Mini 128K Instruct· LMArena data1,006.1
385PaLM 2· LMArena data1,005.5
386StripedHyena Nous 7B· LMArena data1,000.2
387Gemma 1.1 2B IT· LMArena data995.2
388RWKV-4 Raven 14B· LMArena data975.7
389ChatGLM3 6B· LMArena data973.4
390Koala 13B· LMArena data963.7
391OpenAssistant Pythia 12B· LMArena data922.3
392Alpaca 13B· LMArena data910.9
393FastChat-T5 3B· LMArena data898.8
394ChatGLM 6B· LMArena data895.4

Showing 351–394 of 394 results.

Data credits: LMArena, CC BY 4.0 (licence)