Skip to content

Benchmark

Arena ELO - Longer Query

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
345
Scores recorded
401
Sources
1
IQ weight
1.34

Score distribution

1,323.7
Mean
1,357.1
Median
137.1
Std dev
1,102.6
10th percentile
1,475.3
90th percentile
951.6
Min
951.61,549
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.34 for chat models, measured over 345 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
251Step-2 16K Exp202412· LMArena data1,306.3
252GLM 4.5V· LMArena data1,304.6
253Hunyuan Standard2025-02-10· LMArena data1,300.6
254DeepSeek V2.51210· LMArena data1,300.5
255Nemotron 3 Nano 30B A3BBF16· LMArena data1,300.1
256Llama 3.3 Nemotron Super 49B v1· LMArena data1,299.2
257Llama 3.1 Nemotron Ultra 253B v1· LMArena data1,298.6
258Mistral Small 3.1 24B Instruct2503· LMArena data1,298.5
259Yi Lightning· LMArena data1,296.6
260Olmo 3 32B Think· LMArena data1,294.3
261Magistral Medium2506· LMArena data1,294.3
262Qwen2.5-Plus1127· LMArena data1,291.6
263Gemini 1.5 Pro001· LMArena data1,290.5
264Granite 4.2 3B· LMArena data1,289.9
265Athene V2 Chat· LMArena data1,289.2
266GPT-4o mini2024-07-18· LMArena data1,288.7
267GPT 4o2024-05-13· LMArena data1,288.3
268Qwen Max0919· LMArena data1,288.2
269GLM 4 Plus· LMArena data1,285.7
270Gemini 1.5 Flash002· LMArena data1,283.6
271GPT 4o2024-08-06· LMArena data1,283.2
272GPT-4.1 nano2025-04-14· LMArena data1,282.5
273Qwen2.5 72B Instruct· LMArena data1,281.4
274Hunyuan Large Vision· LMArena data1,279.5
275Llama 4 Maverick· LMArena data1,279.4
276Grok 22024-08-13· LMArena data1,276
277Gemma 3n E4B IT· LMArena data1,275.8
278Granite 4.1 8B· LMArena data1,275.5
279Claude 3.5 Sonnet20240620· LMArena data1,274.7
280Gemma 3 4B IT· LMArena data1,273.3
281DeepSeek V2.5· LMArena data1,272.8
282Olmo 3.1 32B Think· LMArena data1,270.8
283Grok 2 mini2024-08-13· LMArena data1,266.1
284Llama 3.1 405B InstructBF16· LMArena data1,265.5
285Llama 4 Scout· LMArena data1,264.5
286Mercury· LMArena data1,263.2
287Mistral Large2411· LMArena data1,261.3
288Claude Haiku 3.520241022· LMArena data1,260.4
289Llama 3.1 405B InstructFP8· LMArena data1,259.3
290Claude 3 Opus20240229· LMArena data1,258.8
291Mistral Large 22407· LMArena data1,257.2
292Llama 3.3 70B Instruct· LMArena data1,255.3
293Nova Pro 1.0· LMArena data1,254.6
294GPT-4 Turbo2024-04-09· LMArena data1,253.6
295Gemini Advanced0514· LMArena data1,253.2
296Gemini 1.5 Flash001· LMArena data1,251.4
297GPT OSS 20b· LMArena data1,250.8
298Qwen 2.5 Coder 32B Instruct· LMArena data1,250.5
299Mistral Small 24B Instruct2501· LMArena data1,245.6
300GPT-4 Turbo Preview0125· LMArena data1,243.6

Showing 251–300 of 401 results.

Data credits: LMArena, CC BY 4.0 (licence)