Skip to content

Benchmark

Arena ELO - Longer Query

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
345
Scores recorded
401
Sources
1
IQ weight
1.34

Score distribution

1,323.7
Mean
1,357.1
Median
137.1
Std dev
1,102.6
10th percentile
1,475.3
90th percentile
951.6
Min
951.61,549
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.34 for chat models, measured over 345 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
301Granite 4.0 H Small· LMArena data1,242.8
302Llama 3.1 70B Instruct· LMArena data1,240.7
303Llama 3.1 Nemotron 70B Instruct· LMArena data1,238.2
304GPT-4 Turbo Preview1106· LMArena data1,235.8
305Nova Lite 1.0· LMArena data1,233.5
306Athene 70B0725· LMArena data1,231.9
307Gemma 2 27B IT· LMArena data1,230.3
308Command R+08-2024· LMArena data1,230.1
309Aya Expanse 32B· LMArena data1,227.3
310Gemma 2 9B IT SimPO· LMArena data1,226
311Nemotron-4 340B Instruct· LMArena data1,224.3
312Hunyuan Standard 256K· LMArena data1,224.3
313Llama 3.1 Tulu 3 70B· LMArena data1,224
314Reka Core20240904· LMArena data1,220.1
315DeepSeek Coder V2 Instruct· LMArena data1,218.8
316Gemini 1.5 Flash-8B001· LMArena data1,218.4
317Phi 4· LMArena data1,217.1
318Ministral 8B2410· LMArena data1,211.7
319Claude 3 Sonnet20240229· LMArena data1,210.8
320GLM 40520· LMArena data1,206.5
321Jamba 1.5 Large· LMArena data1,206.1
322Nova Micro 1.0· LMArena data1,205.3
323Llama 3.1 Nemotron 51B Instruct· LMArena data1,205
324Command R+· LMArena data1,201.3
325Gemma 2 9B IT· LMArena data1,196.4
326OLMo 2 32B Instruct0325· LMArena data1,193.9
327Qwen2 72B Instruct· LMArena data1,192.1
328Reka Flash20240904· LMArena data1,191.5
329Aya Expanse 8B· LMArena data1,191.2
330Claude 3 Haiku20240307· LMArena data1,190.1
331GPT 40314· LMArena data1,190
332GPT 40613· LMArena data1,187.6
333Llama 3.1 8B Instruct· LMArena data1,181.3
334Llama 3.1 Tulu 3 8B· LMArena data1,180.5
335Llama 3 70B Instruct· LMArena data1,173.7
336Mistral Large2402· LMArena data1,172.9
337QwQ 32B Preview· LMArena data1,166.5
338Granite 3.1 8B Instruct· LMArena data1,161.6
339Qwen1.5-110B-Chat· LMArena data1,156.8
340Qwen1.5-72B-Chat· LMArena data1,156.8
341Jamba 1.5 Mini· LMArena data1,156.4
342Granite 3.1 2B Instruct· LMArena data1,154.7
343Mistral Medium· LMArena data1,153.2
344InternLM2.5 20B Chat· LMArena data1,149
345Qwen1.5-32B-Chat· LMArena data1,145.7
346Mixtral 8x22B Instruct v0.1· LMArena data1,143.5
347Yi 1.5 34B Chat· LMArena data1,143.2
348Reka Flash 21B20240226· LMArena data1,140.6
349Reka Flash 21BOnline, 20240226· LMArena data1,139.1
350Gemini ProDev API· LMArena data1,131.4

Showing 301–350 of 401 results.

Data credits: LMArena, CC BY 4.0 (licence)