Skip to content

Benchmark

Arena ELO - Industry Legal and Government

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
338
Scores recorded
394
Sources
1
IQ weight
1.34

Score distribution

1,329.7
Mean
1,368.1
Median
139.4
Std dev
1,104.4
10th percentile
1,475.2
90th percentile
895.4
Min
895.41,543.1
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.34 for chat models, measured over 338 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
301Hunyuan Large Vision· LMArena data1,245.5
302Command R+· LMArena data1,244.8
303GLM 40520· LMArena data1,244
304Gemma 2 27B IT· LMArena data1,241.1
305Phi 4· LMArena data1,241
306Nova Lite 1.0· LMArena data1,238.2
307Magistral Medium2506· LMArena data1,231.6
308OLMo 2 32B Instruct0325· LMArena data1,228.3
309Llama 3 70B Instruct· LMArena data1,227.8
310Granite 4.0 H Small· LMArena data1,225.6
311Aya Expanse 8B· LMArena data1,224.8
312Jamba 1.5 Mini· LMArena data1,223.6
313Nova Micro 1.0· LMArena data1,221.1
314Mistral Small 24B Instruct2501· LMArena data1,218.9
315Reka Flash20240904· LMArena data1,217.2
316Gemma 2 9B IT· LMArena data1,215
317Claude 3 Sonnet20240229· LMArena data1,213.7
318Ministral 8B2410· LMArena data1,211
319Llama 3.1 8B Instruct· LMArena data1,206.2
320Yi 1.5 34B Chat· LMArena data1,196.3
321Reka Flash 21BOnline, 20240226· LMArena data1,194.2
322Qwen2 72B Instruct· LMArena data1,193.1
323GPT 40314· LMArena data1,191.5
324Claude 3 Haiku20240307· LMArena data1,191.2
325Qwen1.5-110B-Chat· LMArena data1,188.2
326Mistral Large2402· LMArena data1,180.8
327Reka Flash 21B20240226· LMArena data1,180.5
328Qwen1.5-72B-Chat· LMArena data1,173.4
329Granite 3.1 8B Instruct· LMArena data1,170.5
330Mistral Medium· LMArena data1,169.8
331Gemma 2 2B IT· LMArena data1,169.5
332Llama 3 8B Instruct· LMArena data1,167.6
333Gemini ProDev API· LMArena data1,166.2
334Mixtral 8x22B Instruct v0.1· LMArena data1,165
335DeepSeek Coder V2 Instruct· LMArena data1,163
336GPT 40613· LMArena data1,154.5
337InternLM2.5 20B Chat· LMArena data1,151.3
338Starling LM 7B Beta· LMArena data1,151.1
339QwQ 32B Preview· LMArena data1,143
340Qwen1.5-32B-Chat· LMArena data1,140.4
341Yi 34B Chat· LMArena data1,137.6
342Tulu 2 DPO 70B· LMArena data1,135.4
343Zephyr ORPO 141B A35B v0.1· LMArena data1,135.3
344Mixtral 8x7B Instruct v0.1· LMArena data1,132
345Phi-3 Medium 4K Instruct· LMArena data1,130.7
346WizardLM 70B· LMArena data1,129.2
347Starling LM 7B Alpha· LMArena data1,122.1
348Qwen1.5-14B-Chat· LMArena data1,119.1
349GPT-3.5 Turbo0125· LMArena data1,117.9
350Llama 3.2 3B· LMArena data1,115.2

Showing 301–350 of 394 results.

Data credits: LMArena, CC BY 4.0 (licence)