Skip to content

Benchmark

Arena ELO - Non English

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
366
Scores recorded
423
Sources
1
IQ weight
1.20

Score distribution

1,288.7
Mean
1,334.4
Median
160.8
Std dev
1,032.5
10th percentile
1,449.6
90th percentile
781.3
Min
781.31,534.5
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.20 for chat models, measured over 366 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
251Hunyuan Turbo0110· LMArena data1,285
252Claude 3.5 Sonnet20241022· LMArena data1,283.1
253GPT 4o2024-05-13· LMArena data1,282.8
254Llama 3.1 Nemotron Ultra 253B v1· LMArena data1,282.2
255Grok 22024-08-13· LMArena data1,282
256Gemini 1.5 Flash002· LMArena data1,278.3
257Olmo 3.1 32B Instruct· LMArena data1,275.5
258GLM 4 Plus· LMArena data1,275.3
259Gemini Advanced0514· LMArena data1,274
260Molmo 2 8B· LMArena data1,273.8
261DeepSeek V2.51210· LMArena data1,273.3
262Gemma 3 4B IT· LMArena data1,273.2
263Claude 3.5 Sonnet20240620· LMArena data1,270
264Llama 4 Maverick· LMArena data1,269.1
265Yi Lightning· LMArena data1,269
266GPT OSS 20b· LMArena data1,268.4
267Athene V2 Chat· LMArena data1,268.2
268Granite 4.2 3B· LMArena data1,267.9
269GPT-4o mini2024-07-18· LMArena data1,266.2
270Hunyuan Large2025-02-10· LMArena data1,265.2
271Qwen2.5-Plus1127· LMArena data1,264.8
272Gemini 1.5 Pro001· LMArena data1,264.4
273Qwen Max0919· LMArena data1,262.9
274Granite 4.1 8B· LMArena data1,261.7
275GPT 4o2024-08-06· LMArena data1,261.6
276Hunyuan Standard2025-02-10· LMArena data1,261.5
277GPT-4.1 nano2025-04-14· LMArena data1,260
278Mercury· LMArena data1,259
279Claude 3 Opus20240229· LMArena data1,257.7
280Grok 2 mini2024-08-13· LMArena data1,257
281Mistral Small 3.1 24B Instruct2503· LMArena data1,254.6
282Llama 3.3 Nemotron Super 49B v1· LMArena data1,253.4
283DeepSeek V2.5· LMArena data1,253.2
284Olmo 3 32B Think· LMArena data1,253.2
285Llama 4 Scout· LMArena data1,252.1
286Qwen2.5 72B Instruct· LMArena data1,251.8
287Llama 3.1 405B InstructFP8· LMArena data1,247.8
288Llama 3.1 405B InstructBF16· LMArena data1,246.8
289GPT-4 Turbo Preview1106· LMArena data1,245.7
290GPT-4 Turbo2024-04-09· LMArena data1,245.1
291Llama 3.1 Nemotron 70B Instruct· LMArena data1,244.9
292GPT-4 Turbo Preview0125· LMArena data1,240.8
293Claude Haiku 3.520241022· LMArena data1,237.8
294Reka Core20240904· LMArena data1,236.5
295Mistral Large 22407· LMArena data1,236.5
296Llama 3.3 70B Instruct· LMArena data1,236.3
297Hunyuan Large Vision· LMArena data1,235.7
298Llama 3.1 Tulu 3 70B· LMArena data1,235.6
299Mistral Large2411· LMArena data1,235.2
300Athene 70B0725· LMArena data1,234.7

Showing 251–300 of 423 results.

Data credits: LMArena, CC BY 4.0 (licence)