Skip to content

Benchmark

Arena ELO Style-Controlled

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
366
Scores recorded
423
Sources
1
IQ weight
1.10

Score distribution

1,341.3
Mean
1,363.7
Median
121
Std dev
1,152.6
10th percentile
1,474.5
90th percentile
953.4
Min
953.41,527
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.10 for chat models, measured over 366 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
301Granite 4.2 8B· LMArena data1,289.5
302Jamba 1.5 Large· LMArena data1,289.5
303Granite 4.2 3B· LMArena data1,289.3
304Reka Core20240904· LMArena data1,288.1
305GPT 40314· LMArena data1,288
306Granite 4.0 H Small· LMArena data1,287.5
307Llama 3.1 Nemotron 51B Instruct· LMArena data1,286.8
308Gemini 1.5 Flash001· LMArena data1,286.6
309Llama 3.1 Tulu 3 70B· LMArena data1,286.2
310Olmo 3.1 32B Think· LMArena data1,286
311Claude 3 Sonnet20240229· LMArena data1,281.5
312Gemma 2 9B IT SimPO· LMArena data1,280.5
313Nemotron-4 340B Instruct· LMArena data1,277.3
314Llama 3 70B Instruct· LMArena data1,276.7
315Command R+08-2024· LMArena data1,276.4
316GPT 40613· LMArena data1,276.4
317Mistral Small 24B Instruct2501· LMArena data1,274.4
318GLM 40520· LMArena data1,273.5
319Reka Flash20240904· LMArena data1,272.6
320Qwen 2.5 Coder 32B Instruct· LMArena data1,270.7
321Aya Expanse 32B· LMArena data1,267.4
322Gemma 2 9B IT· LMArena data1,267.1
323DeepSeek Coder V2 Instruct· LMArena data1,265.5
324Qwen2 72B Instruct· LMArena data1,262
325Command R+· LMArena data1,261.9
326Claude 3 Haiku20240307· LMArena data1,261.8
327Nova Lite 1.0· LMArena data1,260.5
328Gemini 1.5 Flash-8B001· LMArena data1,258.9
329Phi 4· LMArena data1,256.3
330OLMo 2 32B Instruct0325· LMArena data1,251.7
331Mistral Large2402· LMArena data1,242.7
332Nova Micro 1.0· LMArena data1,240.8
333Jamba 1.5 Mini· LMArena data1,239.9
334Ministral 8B2410· LMArena data1,237.7
335Gemini ProDev API· LMArena data1,237
336Qwen1.5-110B-Chat· LMArena data1,234.3
337Reka Flash 21BOnline, 20240226· LMArena data1,233.7
338Qwen1.5-72B-Chat· LMArena data1,233.7
339Hunyuan Standard 256K· LMArena data1,233.2
340Mixtral 8x22B Instruct v0.1· LMArena data1,229.8
341Reka Flash 21B20240226· LMArena data1,227
342GPT-3.5 Turbo0125· LMArena data1,225.8
343Gemini Pro· LMArena data1,223.8
344Llama 3 8B Instruct· LMArena data1,223.8
345Aya Expanse 8B· LMArena data1,223.2
346Mistral Medium· LMArena data1,222.9
347Llama 3.1 Tulu 3 8B· LMArena data1,220.6
348Zephyr ORPO 141B A35B v0.1· LMArena data1,213.2
349Yi 1.5 34B Chat· LMArena data1,213
350Llama 3.1 8B Instruct· LMArena data1,211.3

Showing 301–350 of 423 results.

Data credits: LMArena, CC BY 4.0 (licence)