Skip to content

Benchmark

Arena ELO - Industry Writing and Literature and Language

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
365
Scores recorded
422
Sources
1
IQ weight
1.17

Score distribution

1,297.8
Mean
1,328.7
Median
142.8
Std dev
1,086
10th percentile
1,453.9
90th percentile
809.1
Min
809.11,540.8
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.17 for chat models, measured over 365 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
301Command R+08-2024· LMArena data1,241.1
302Athene 70B0725· LMArena data1,240.8
303GPT OSS 20b· LMArena data1,238.3
304Llama 3.1 70B Instruct· LMArena data1,235.7
305Mercury· LMArena data1,229.2
306Jamba 1.5 Large· LMArena data1,227.7
307Granite 4.0 H Small· LMArena data1,227
308Gemini 1.5 Flash-8B001· LMArena data1,226.8
309Gemma 2 9B IT SimPO· LMArena data1,225.9
310Nemotron-4 340B Instruct· LMArena data1,223.4
311Aya Expanse 32B· LMArena data1,222.2
312Llama 3.1 Nemotron 51B Instruct· LMArena data1,219.9
313Gemma 2 9B IT· LMArena data1,219
314GPT 40314· LMArena data1,218.7
315Command R+· LMArena data1,218.7
316Claude 3 Sonnet20240229· LMArena data1,217.7
317Reka Flash20240904· LMArena data1,217.1
318GPT 40613· LMArena data1,214
319Mistral Small 24B Instruct2501· LMArena data1,213.9
320Granite 4.2 3B· LMArena data1,213.6
321Nova Lite 1.0· LMArena data1,211.2
322GLM 40520· LMArena data1,210.6
323Llama 3 70B Instruct· LMArena data1,206.1
324OLMo 2 32B Instruct0325· LMArena data1,203.6
325Qwen2 72B Instruct· LMArena data1,201.7
326Phi 4· LMArena data1,197.9
327Claude 3 Haiku20240307· LMArena data1,193.9
328Nova Micro 1.0· LMArena data1,192.3
329Qwen 2.5 Coder 32B Instruct· LMArena data1,191.1
330Aya Expanse 8B· LMArena data1,185.5
331Ministral 8B2410· LMArena data1,180.6
332Mistral Large2402· LMArena data1,179.8
333Llama 3.1 Tulu 3 8B· LMArena data1,177.6
334Hunyuan Standard 256K· LMArena data1,173.4
335Mistral Medium· LMArena data1,172.1
336Qwen1.5-110B-Chat· LMArena data1,170.5
337DeepSeek Coder V2 Instruct· LMArena data1,170.3
338Jamba 1.5 Mini· LMArena data1,165.4
339Llama 3.1 8B Instruct· LMArena data1,162.1
340Mixtral 8x22B Instruct v0.1· LMArena data1,161.1
341Reka Flash 21BOnline, 20240226· LMArena data1,160.3
342Qwen1.5-72B-Chat· LMArena data1,159.1
343Gemma 2 2B IT· LMArena data1,157.1
344Reka Flash 21B20240226· LMArena data1,152.2
345Llama 3 8B Instruct· LMArena data1,147.8
346WizardLM 70B· LMArena data1,144.6
347Yi 1.5 34B Chat· LMArena data1,144.1
348Gemini ProDev API· LMArena data1,144
349QwQ 32B Preview· LMArena data1,139.4
350Zephyr ORPO 141B A35B v0.1· LMArena data1,139.1

Showing 301–350 of 422 results.

Data credits: LMArena, CC BY 4.0 (licence)