Skip to content

Benchmark

Arena ELO - Industry Life and Physical and Social Science

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
364
Scores recorded
421
Sources
1
IQ weight
1.24

Score distribution

1,316.4
Mean
1,364.4
Median
158.9
Std dev
1,071.5
10th percentile
1,475.1
90th percentile
800.9
Min
800.91,549.3
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.24 for chat models, measured over 364 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
301Granite 4.0 H Small· LMArena data1,253.4
302Command R+08-2024· LMArena data1,251.8
303GPT-4 Turbo Preview0125· LMArena data1,249.9
304Jamba 1.5 Large· LMArena data1,249.4
305Claude Haiku 3.520241022· LMArena data1,247.2
306OLMo 2 32B Instruct0325· LMArena data1,239.5
307Mistral Small 24B Instruct2501· LMArena data1,234.9
308Magistral Medium2506· LMArena data1,233.6
309Aya Expanse 32B· LMArena data1,232.5
310Nova Lite 1.0· LMArena data1,232.4
311Gemini 1.5 Flash-8B001· LMArena data1,232.2
312Gemini 1.5 Flash001· LMArena data1,231.8
313Gemma 2 9B IT SimPO· LMArena data1,230.9
314Nemotron-4 340B Instruct· LMArena data1,228.4
315Gemma 2 27B IT· LMArena data1,228.1
316GLM 40520· LMArena data1,226.2
317Llama 3.1 Nemotron 51B Instruct· LMArena data1,225.8
318Llama 3 70B Instruct· LMArena data1,225.8
319Reka Flash20240904· LMArena data1,223
320Qwen 2.5 Coder 32B Instruct· LMArena data1,221.4
321Command R+· LMArena data1,216.3
322Nova Micro 1.0· LMArena data1,210.3
323Claude 3 Sonnet20240229· LMArena data1,209.2
324Phi 4· LMArena data1,206.5
325Aya Expanse 8B· LMArena data1,203.7
326Gemma 2 9B IT· LMArena data1,203.6
327Hunyuan Standard 256K· LMArena data1,200.6
328Qwen2 72B Instruct· LMArena data1,195.2
329Llama 3.1 Tulu 3 8B· LMArena data1,193.4
330Llama 3.1 8B Instruct· LMArena data1,193.3
331Jamba 1.5 Mini· LMArena data1,190.2
332Ministral 8B2410· LMArena data1,189
333Claude 3 Haiku20240307· LMArena data1,178.4
334GPT 40314· LMArena data1,176.9
335Yi 1.5 34B Chat· LMArena data1,173.1
336Llama 3 8B Instruct· LMArena data1,170.1
337Reka Flash 21BOnline, 20240226· LMArena data1,168.2
338Qwen1.5-110B-Chat· LMArena data1,166
339InternLM2.5 20B Chat· LMArena data1,164.9
340DeepSeek Coder V2 Instruct· LMArena data1,162.7
341Reka Flash 21B20240226· LMArena data1,159.7
342Gemma 2 2B IT· LMArena data1,159
343Mistral Medium· LMArena data1,158.3
344Qwen1.5-72B-Chat· LMArena data1,157.8
345QwQ 32B Preview· LMArena data1,153.9
346Mistral Large2402· LMArena data1,153.8
347GPT 40613· LMArena data1,153.4
348Mixtral 8x22B Instruct v0.1· LMArena data1,151
349Gemini ProDev API· LMArena data1,143.9
350Zephyr ORPO 141B A35B v0.1· LMArena data1,142.2

Showing 301–350 of 421 results.

Data credits: LMArena, CC BY 4.0 (licence)