Skip to content

Benchmark

Arena ELO

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
366
Scores recorded
423
Sources
1
IQ weight
1.19

Score distribution

1,312
Mean
1,354.8
Median
146.6
Std dev
1,084.5
10th percentile
1,462.5
90th percentile
833.5
Min
833.51,539.3
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.19 for chat models, measured over 366 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
301Llama 3.1 Tulu 3 70B· LMArena data1,255.8
302Claude Haiku 3.520241022· LMArena data1,255.4
303Magistral Medium2506· LMArena data1,254.9
304Reka Core20240904· LMArena data1,248.4
305Granite 4.0 H Small· LMArena data1,241.5
306Gemini 1.5 Flash001· LMArena data1,239.3
307Jamba 1.5 Large· LMArena data1,237.1
308Mistral Small 24B Instruct2501· LMArena data1,233.6
309Gemma 2 27B IT· LMArena data1,231.4
310Qwen 2.5 Coder 32B Instruct· LMArena data1,230.1
311Command R+08-2024· LMArena data1,228.9
312Nova Lite 1.0· LMArena data1,228.5
313Llama 3.1 Nemotron 51B Instruct· LMArena data1,228
314Gemma 2 9B IT SimPO· LMArena data1,227.1
315GLM 40520· LMArena data1,225.9
316Gemini 1.5 Flash-8B001· LMArena data1,225.9
317Nemotron-4 340B Instruct· LMArena data1,224.8
318Aya Expanse 32B· LMArena data1,224
319Llama 3 70B Instruct· LMArena data1,220.8
320Claude 3 Sonnet20240229· LMArena data1,218.1
321Reka Flash20240904· LMArena data1,218
322OLMo 2 32B Instruct0325· LMArena data1,217.9
323Phi 4· LMArena data1,216.7
324Nova Micro 1.0· LMArena data1,208.5
325Gemma 2 9B IT· LMArena data1,207.4
326GPT 40314· LMArena data1,206
327Command R+· LMArena data1,203.9
328Qwen2 72B Instruct· LMArena data1,203.3
329Hunyuan Standard 256K· LMArena data1,202
330Claude 3 Haiku20240307· LMArena data1,194.7
331Llama 3.1 Tulu 3 8B· LMArena data1,193.3
332DeepSeek Coder V2 Instruct· LMArena data1,191.2
333Ministral 8B2410· LMArena data1,191
334Jamba 1.5 Mini· LMArena data1,186.7
335Llama 3.1 8B Instruct· LMArena data1,186.6
336GPT 40613· LMArena data1,186
337Aya Expanse 8B· LMArena data1,185.1
338Mistral Large2402· LMArena data1,176.5
339Qwen1.5-110B-Chat· LMArena data1,174.6
340Yi 1.5 34B Chat· LMArena data1,173
341Reka Flash 21BOnline, 20240226· LMArena data1,170.5
342Qwen1.5-72B-Chat· LMArena data1,166.2
343Llama 3 8B Instruct· LMArena data1,165.6
344Mistral Medium· LMArena data1,165.2
345Reka Flash 21B20240226· LMArena data1,165
346Mixtral 8x22B Instruct v0.1· LMArena data1,162
347QwQ 32B Preview· LMArena data1,161.8
348InternLM2.5 20B Chat· LMArena data1,158.5
349Gemma 2 2B IT· LMArena data1,155.8
350Granite 3.1 8B Instruct· LMArena data1,149.6

Showing 301–350 of 423 results.

Data credits: LMArena, CC BY 4.0 (licence)