Skip to content

Benchmark

Arena ELO - German

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
273
Scores recorded
319
Sources
1
IQ weight
1.52

Score distribution

1,328.6
Mean
1,372.1
Median
138.3
Std dev
1,110.8
10th percentile
1,469.8
90th percentile
901.9
Min
901.91,585.2
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.52 for chat models, measured over 273 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
251DeepSeek V2.5· LMArena data1,226.4
252Phi 4· LMArena data1,222.4
253Llama 3.1 70B Instruct· LMArena data1,221.6
254Gemma 2 9B IT SimPO· LMArena data1,218.4
255Gemini 1.5 Flash001· LMArena data1,216.4
256Command R+08-2024· LMArena data1,215.9
257Mistral Small 24B Instruct2501· LMArena data1,211.3
258Gemma 2 27B IT· LMArena data1,208.9
259Gemini 1.5 Flash-8B001· LMArena data1,205.9
260Claude 3 Sonnet20240229· LMArena data1,203.7
261Jamba 1.5 Large· LMArena data1,199.8
262Aya Expanse 32B· LMArena data1,198.6
263GPT 40314· LMArena data1,198.2
264Command R+· LMArena data1,195.4
265Nova Micro 1.0· LMArena data1,192.2
266Nemotron-4 340B Instruct· LMArena data1,191.9
267Aya Expanse 8B· LMArena data1,187.6
268Gemma 2 9B IT· LMArena data1,186.1
269GLM 40520· LMArena data1,183.8
270Reka Flash20240904· LMArena data1,178.3
271Mistral Large2402· LMArena data1,177.9
272Claude 3 Haiku20240307· LMArena data1,173.6
273Llama 3 70B Instruct· LMArena data1,169
274DeepSeek Coder V2 Instruct· LMArena data1,164.3
275Jamba 1.5 Mini· LMArena data1,161.9
276GPT 40613· LMArena data1,161.5
277Mistral Medium· LMArena data1,154.6
278Reka Flash 21B20240226· LMArena data1,152.6
279Reka Flash 21BOnline, 20240226· LMArena data1,150.8
280Qwen2 72B Instruct· LMArena data1,150.7
281Llama 3.1 8B Instruct· LMArena data1,144.2
282Mixtral 8x22B Instruct v0.1· LMArena data1,141.2
283Gemini ProDev API· LMArena data1,124.5
284Qwen1.5-110B-Chat· LMArena data1,122.6
285Gemma 2 2B IT· LMArena data1,114.1
286Mixtral 8x7B Instruct v0.1· LMArena data1,113.9
287Yi 1.5 34B Chat· LMArena data1,110.8
288Llama 3 8B Instruct· LMArena data1,104.4
289Phi-3 Medium 4K Instruct· LMArena data1,100.7
290GPT-3.5 Turbo0125· LMArena data1,090.4
291Qwen1.5-72B-Chat· LMArena data1,083.7
292WizardLM 70B· LMArena data1,082.6
293Starling LM 7B Beta· LMArena data1,081.1
294Phi-3 Small 8K Instruct· LMArena data1,079.6
295Snowflake Arctic Instruct· LMArena data1,073.6
296InternLM2.5 20B Chat· LMArena data1,062.7
297OpenChat 3.50106· LMArena data1,059.8
298Vicuna 33B· LMArena data1,059.1
299Qwen1.5-32B-Chat· LMArena data1,058.2
300DBRX Instruct Preview· LMArena data1,057.3

Showing 251–300 of 319 results.

Data credits: LMArena, CC BY 4.0 (licence)