Skip to content

Benchmark

Arena ELO - Industry Medicine and Healthcare

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
336
Scores recorded
391
Sources
1
IQ weight
1.38

Score distribution

1,319.8
Mean
1,368.4
Median
152.6
Std dev
1,071.7
10th percentile
1,469.7
90th percentile
819.1
Min
819.11,538.2
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.38 for chat models, measured over 336 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
301Nova Micro 1.0· LMArena data1,207.5
302Command R+· LMArena data1,203.4
303GLM 40520· LMArena data1,203.1
304Aya Expanse 8B· LMArena data1,203.1
305Phi 4· LMArena data1,201.8
306Reka Flash20240904· LMArena data1,200.2
307Gemma 2 27B IT· LMArena data1,197.7
308Jamba 1.5 Large· LMArena data1,196.5
309Nemotron-4 340B Instruct· LMArena data1,194.6
310Claude 3 Sonnet20240229· LMArena data1,188.3
311Llama 3.1 Nemotron 51B Instruct· LMArena data1,186.6
312Qwen 2.5 Coder 32B Instruct· LMArena data1,186.5
313Gemini 1.5 Flash001· LMArena data1,186.4
314OLMo 2 32B Instruct0325· LMArena data1,181.9
315Llama 3.1 8B Instruct· LMArena data1,181.6
316Jamba 1.5 Mini· LMArena data1,181
317Ministral 8B2410· LMArena data1,180.2
318Qwen2 72B Instruct· LMArena data1,175.7
319Gemma 2 9B IT· LMArena data1,165.5
320Claude 3 Haiku20240307· LMArena data1,165.2
321Yi 1.5 34B Chat· LMArena data1,159.4
322InternLM2.5 20B Chat· LMArena data1,151.7
323Llama 3 8B Instruct· LMArena data1,148.1
324Reka Flash 21B20240226· LMArena data1,147.3
325Qwen1.5-110B-Chat· LMArena data1,146
326GPT 40314· LMArena data1,145.2
327Gemma 2 2B IT· LMArena data1,143.7
328Reka Flash 21BOnline, 20240226· LMArena data1,142.8
329Qwen1.5-72B-Chat· LMArena data1,136.4
330Gemini ProDev API· LMArena data1,134.8
331DeepSeek Coder V2 Instruct· LMArena data1,133.3
332Mistral Large2402· LMArena data1,133.3
333Mistral Medium· LMArena data1,132.2
334Starling LM 7B Beta· LMArena data1,129.4
335GPT 40613· LMArena data1,123.3
336Mixtral 8x22B Instruct v0.1· LMArena data1,119.9
337Phi-3 Medium 4K Instruct· LMArena data1,118.8
338Gemini Pro· LMArena data1,116.5
339Starling LM 7B Alpha· LMArena data1,111.5
340Qwen1.5-32B-Chat· LMArena data1,111
341Zephyr ORPO 141B A35B v0.1· LMArena data1,106.4
342Qwen1.5-14B-Chat· LMArena data1,104.9
343Llama 3.2 3B· LMArena data1,104.3
344Yi 34B Chat· LMArena data1,102.6
345Phi-3 Small 8K Instruct· LMArena data1,097.2
346WizardLM 70B· LMArena data1,094.7
347Mixtral 8x7B Instruct v0.1· LMArena data1,093.6
348Llama 2 70B Chat· LMArena data1,093.2
349GPT-3.5 Turbo0125· LMArena data1,074.2
350OpenChat 3.50106· LMArena data1,073.3

Showing 301–350 of 391 results.

Data credits: LMArena, CC BY 4.0 (licence)