Skip to content

Benchmark

Arena ELO - Non English

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
366
Scores recorded
423
Sources
1
IQ weight
1.20

Score distribution

1,288.7
Mean
1,334.4
Median
160.8
Std dev
1,032.5
10th percentile
1,449.6
90th percentile
781.3
Min
781.31,534.5
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.20 for chat models, measured over 366 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
301Nova Pro 1.0· LMArena data1,233.5
302Magistral Medium2506· LMArena data1,232
303Gemini 1.5 Flash001· LMArena data1,230.5
304Llama 3.1 70B Instruct· LMArena data1,219.2
305Gemma 2 27B IT· LMArena data1,217
306Command R+08-2024· LMArena data1,216.4
307Granite 4.0 H Small· LMArena data1,216.3
308Gemini 1.5 Flash-8B001· LMArena data1,214.6
309Aya Expanse 32B· LMArena data1,213.4
310Olmo 3.1 32B Think· LMArena data1,209
311Nova Lite 1.0· LMArena data1,208.1
312Reka Flash20240904· LMArena data1,207.1
313Gemma 2 9B IT SimPO· LMArena data1,206.3
314Nemotron-4 340B Instruct· LMArena data1,206
315Claude 3 Sonnet20240229· LMArena data1,205.1
316Qwen 2.5 Coder 32B Instruct· LMArena data1,204.6
317GLM 40520· LMArena data1,199.6
318Mistral Small 24B Instruct2501· LMArena data1,198.1
319Phi 4· LMArena data1,196.7
320Jamba 1.5 Large· LMArena data1,196.3
321GPT 40314· LMArena data1,194.2
322Command R+· LMArena data1,189.5
323Gemma 2 9B IT· LMArena data1,187.8
324Nova Micro 1.0· LMArena data1,186.3
325DeepSeek Coder V2 Instruct· LMArena data1,182
326Llama 3.1 Nemotron 51B Instruct· LMArena data1,180.5
327Aya Expanse 8B· LMArena data1,180
328Hunyuan Standard 256K· LMArena data1,178.5
329Claude 3 Haiku20240307· LMArena data1,177.5
330Qwen2 72B Instruct· LMArena data1,175.9
331Llama 3.1 Tulu 3 8B· LMArena data1,169
332Ministral 8B2410· LMArena data1,165
333GPT 40613· LMArena data1,160.5
334OLMo 2 32B Instruct0325· LMArena data1,160.2
335Reka Flash 21BOnline, 20240226· LMArena data1,155
336Llama 3.1 8B Instruct· LMArena data1,148
337Reka Flash 21B20240226· LMArena data1,145.8
338Mistral Large2402· LMArena data1,142.1
339Qwen1.5-110B-Chat· LMArena data1,141.9
340Llama 3 70B Instruct· LMArena data1,141.7
341Gemini ProDev API· LMArena data1,138
342Qwen1.5-72B-Chat· LMArena data1,135
343Jamba 1.5 Mini· LMArena data1,134.7
344QwQ 32B Preview· LMArena data1,129.6
345Mistral Medium· LMArena data1,128.6
346Mixtral 8x22B Instruct v0.1· LMArena data1,128.2
347Yi 1.5 34B Chat· LMArena data1,121
348Gemma 2 2B IT· LMArena data1,120.7
349GPT-3.5 Turbo0125· LMArena data1,107.7
350Qwen1.5-32B-Chat· LMArena data1,106.4

Showing 301–350 of 423 results.

Data credits: LMArena, CC BY 4.0 (licence)