Skip to content

Benchmark

Arena ELO - Instruction Following

Instruction following benchmarks measure how closely models follow instructions. Higher scores are better.

Models measured
366
Scores recorded
423
Sources
1
IQ weight
1.18

Score distribution

1,294.5
Mean
1,332.7
Median
155.5
Std dev
1,056.7
10th percentile
1,465
90th percentile
780.7
Min
780.71,539.1
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.18 for chat models, measured over 366 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
351Zephyr ORPO 141B A35B v0.1· LMArena data1,117.6
352Gemma 2 2B IT· LMArena data1,117.5
353Granite 3.1 2B Instruct· LMArena data1,115.9
354Qwen1.5-32B-Chat· LMArena data1,115.5
355Gemini ProDev API· LMArena data1,114
356Phi-3 Medium 4K Instruct· LMArena data1,113.3
357DBRX Instruct Preview· LMArena data1,111.5
358Tulu 2 DPO 70B· LMArena data1,110.2
359Gemini Pro· LMArena data1,109.9
360Mixtral 8x7B Instruct v0.1· LMArena data1,109.3
361Qwen1.5-14B-Chat· LMArena data1,101.6
362Starling LM 7B Beta· LMArena data1,095.7
363WizardLM 70B· LMArena data1,092.9
364GPT-3.5 Turbo1106· LMArena data1,092.5
365Yi 34B Chat· LMArena data1,090.8
366Llama 3.2 3B· LMArena data1,088.8
367Granite 3.0 8B Instruct· LMArena data1,087.9
368Phi-3 Small 8K Instruct· LMArena data1,086.5
369DeepSeek LLM 67B Chat· LMArena data1,079
370OpenChat 3.50106· LMArena data1,076.3
371Llama 2 70B Chat· LMArena data1,070.7
372OpenHermes 2.5 Mistral 7B· LMArena data1,070.1
373OpenChat 3.5· LMArena data1,070.1
374Starling LM 7B Alpha· LMArena data1,069.8
375Snowflake Arctic Instruct· LMArena data1,066
376Vicuna 33B· LMArena data1,064.9
377Llama 2 70B SteerLM Chat· LMArena data1,060.3
378Mistral 7B Instruct v0.2· LMArena data1,059.9
379Qwen1.5-7B-Chat· LMArena data1,058.2
380Phi-3 Mini 4K InstructJune 2024· LMArena data1,057.6
381Gemma 1.1 7B IT· LMArena data1,056.7
382Nous Hermes 2 Mixtral 8x7B DPO· LMArena data1,055.9
383Granite 3.0 2B Instruct· LMArena data1,055.5
384MPT 30B Chat· LMArena data1,053.8
385Dolphin 2.2.1 Mistral 7B· LMArena data1,053.3
386Phi-3 Mini 4K Instruct· LMArena data1,053.2
387WizardLM 13B· LMArena data1,048.3
388Falcon 180B Chat· LMArena data1,046.5
389SOLAR 10.7B Instruct v1.0· LMArena data1,046.4
390Llama 2 13B Chat· LMArena data1,045.2
391Vicuna 13B· LMArena data1,035
392Zephyr 7B Alpha· LMArena data1,034.3
393Zephyr 7B Beta· LMArena data1,031.4
394Qwen-14B-Chat· LMArena data1,031.2
395Llama 3.2 1B Instruct· LMArena data1,031
396SmolLM2 1.7B Instruct· LMArena data1,029.1
397Code Llama 34B Instruct· LMArena data1,027.8
398Code Llama 70B Instruct· LMArena data1,023.6
399Phi-3 Mini 128K Instruct· LMArena data1,022.7
400Gemma 7B IT· LMArena data1,016.9

Showing 351–400 of 423 results.

Data credits: LMArena, CC BY 4.0 (licence)