Skip to content

Benchmark

Arena ELO - Expert

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
318
Scores recorded
371
Sources
1
IQ weight
1.30

Score distribution

1,333.9
Mean
1,371.6
Median
146.5
Std dev
1,113
10th percentile
1,491.8
90th percentile
953.7
Min
953.71,581.3
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.30 for chat models, measured over 318 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
301Aya Expanse 32B· LMArena data1,181.8
302Reka Flash20240904· LMArena data1,181.7
303DeepSeek Coder V2 Instruct· LMArena data1,181.2
304GLM 40520· LMArena data1,178.8
305Command R+08-2024· LMArena data1,173.3
306Claude 3 Sonnet20240229· LMArena data1,172.2
307Gemma 2 27B IT· LMArena data1,171.3
308Nemotron-4 340B Instruct· LMArena data1,171
309Qwen2 72B Instruct· LMArena data1,171
310Ministral 8B2410· LMArena data1,169.6
311Llama 3.1 Nemotron 51B Instruct· LMArena data1,166.6
312Gemma 2 9B IT SimPO· LMArena data1,154.3
313Command R+· LMArena data1,153.2
314Aya Expanse 8B· LMArena data1,152.4
315InternLM2.5 20B Chat· LMArena data1,150.4
316Llama 3 70B Instruct· LMArena data1,148.9
317GPT 40314· LMArena data1,148.6
318Claude 3 Haiku20240307· LMArena data1,147.6
319Gemma 2 9B IT· LMArena data1,147
320Qwen1.5-110B-Chat· LMArena data1,144.2
321Yi 1.5 34B Chat· LMArena data1,143.8
322Llama 3.1 8B Instruct· LMArena data1,143.8
323Granite 3.1 8B Instruct· LMArena data1,141.4
324Qwen1.5-72B-Chat· LMArena data1,135.8
325Jamba 1.5 Mini· LMArena data1,135.4
326QwQ 32B Preview· LMArena data1,134.3
327Granite 3.1 2B Instruct· LMArena data1,130.9
328GPT 40613· LMArena data1,127.7
329Qwen1.5-32B-Chat· LMArena data1,126.1
330Mistral Medium· LMArena data1,125.1
331Mistral Large2402· LMArena data1,122.9
332Reka Flash 21BOnline, 20240226· LMArena data1,116.2
333Llama 3 8B Instruct· LMArena data1,113
334Mixtral 8x22B Instruct v0.1· LMArena data1,113
335Phi-3 Medium 4K Instruct· LMArena data1,107.9
336Reka Flash 21B20240226· LMArena data1,107
337Gemma 2 2B IT· LMArena data1,095.9
338Qwen1.5-14B-Chat· LMArena data1,093.8
339Llama 3.2 3B· LMArena data1,089.9
340Mixtral 8x7B Instruct v0.1· LMArena data1,087.5
341Granite 3.0 8B Instruct· LMArena data1,087.2
342Starling LM 7B Beta· LMArena data1,083.7
343DBRX Instruct Preview· LMArena data1,075.9
344GPT-3.5 Turbo1106· LMArena data1,069.4
345Phi-3 Small 8K Instruct· LMArena data1,066.9
346GPT-3.5 Turbo0125· LMArena data1,065.3
347Granite 3.0 2B Instruct· LMArena data1,064.2
348Zephyr ORPO 141B A35B v0.1· LMArena data1,061.6
349Yi 34B Chat· LMArena data1,061.1
350Gemini ProDev API· LMArena data1,058.8

Showing 301–350 of 371 results.

Data credits: LMArena, CC BY 4.0 (licence)