Skip to content

Benchmark

Arena ELO - Instruction Following

Instruction following benchmarks measure how closely models follow instructions. Higher scores are better.

Models measured
366
Scores recorded
423
Sources
1
IQ weight
1.18

Score distribution

1,294.5
Mean
1,332.7
Median
155.5
Std dev
1,056.7
10th percentile
1,465
90th percentile
780.7
Min
780.71,539.1
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.18 for chat models, measured over 366 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
301Llama 3.1 Tulu 3 70B· LMArena data1,232.5
302Llama 3.1 70B Instruct· LMArena data1,231.1
303Athene 70B0725· LMArena data1,226.7
304Qwen 2.5 Coder 32B Instruct· LMArena data1,222.7
305Granite 4.0 H Small· LMArena data1,222.7
306Mistral Small 24B Instruct2501· LMArena data1,213.8
307Jamba 1.5 Large· LMArena data1,212.5
308Gemini 1.5 Flash001· LMArena data1,211.8
309Reka Core20240904· LMArena data1,211.8
310Gemma 2 27B IT· LMArena data1,205.9
311Nova Lite 1.0· LMArena data1,204.9
312Nemotron-4 340B Instruct· LMArena data1,203
313GLM 40520· LMArena data1,202.6
314Hunyuan Standard 256K· LMArena data1,201.7
315Phi 4· LMArena data1,201.1
316GPT 40314· LMArena data1,200.9
317Llama 3.1 Nemotron 51B Instruct· LMArena data1,200.6
318Claude 3 Sonnet20240229· LMArena data1,199.3
319Gemini 1.5 Flash-8B001· LMArena data1,199
320Command R+08-2024· LMArena data1,196.6
321Aya Expanse 32B· LMArena data1,195.9
322Llama 3 70B Instruct· LMArena data1,194.1
323Gemma 2 9B IT SimPO· LMArena data1,190.9
324GPT 40613· LMArena data1,188.5
325OLMo 2 32B Instruct0325· LMArena data1,185.8
326Reka Flash20240904· LMArena data1,185.2
327Qwen2 72B Instruct· LMArena data1,181
328DeepSeek Coder V2 Instruct· LMArena data1,180.2
329Command R+· LMArena data1,179.6
330Gemma 2 9B IT· LMArena data1,178.1
331Llama 3.1 Tulu 3 8B· LMArena data1,174.1
332Nova Micro 1.0· LMArena data1,174
333Claude 3 Haiku20240307· LMArena data1,173.2
334Mistral Large2402· LMArena data1,169.1
335Ministral 8B2410· LMArena data1,160.5
336Llama 3.1 8B Instruct· LMArena data1,158.8
337Qwen1.5-110B-Chat· LMArena data1,157.7
338QwQ 32B Preview· LMArena data1,155.8
339Aya Expanse 8B· LMArena data1,154.7
340Mistral Medium· LMArena data1,151.7
341Jamba 1.5 Mini· LMArena data1,147.9
342Mixtral 8x22B Instruct v0.1· LMArena data1,147
343Qwen1.5-72B-Chat· LMArena data1,141.2
344Yi 1.5 34B Chat· LMArena data1,139.1
345InternLM2.5 20B Chat· LMArena data1,135
346Granite 3.1 8B Instruct· LMArena data1,130.6
347Reka Flash 21BOnline, 20240226· LMArena data1,129.3
348Llama 3 8B Instruct· LMArena data1,127.1
349Reka Flash 21B20240226· LMArena data1,121.3
350GPT-3.5 Turbo0125· LMArena data1,118.5

Showing 301–350 of 423 results.

Data credits: LMArena, CC BY 4.0 (licence)