Skip to content

Benchmark

Arena ELO - Coding

Coding benchmarks compare models on programming tasks. Higher scores are better.

Models measured
361
Scores recorded
418
Sources
1
IQ weight
1.28

Score distribution

1,325
Mean
1,374.5
Median
162.1
Std dev
1,081.6
10th percentile
1,488
90th percentile
682.8
Min
682.81,564.3
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.28 for chat models, measured over 361 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
301DeepSeek Coder V2 Instruct· LMArena data1,250.9
302GPT-4 Turbo Preview0125· LMArena data1,249.1
303Granite 4.0 H Small· LMArena data1,249
304Mistral Small 24B Instruct2501· LMArena data1,245.6
305Nova Lite 1.0· LMArena data1,238.7
306Gemini 1.5 Flash001· LMArena data1,235.6
307Llama 3.1 Tulu 3 70B· LMArena data1,235.3
308Phi 4· LMArena data1,231.2
309Hunyuan Standard 256K· LMArena data1,230.7
310Gemma 3 4B IT· LMArena data1,230.1
311Reka Core20240904· LMArena data1,229.9
312GLM 40520· LMArena data1,227.5
313Jamba 1.5 Large· LMArena data1,227.4
314Llama 3.1 Nemotron 51B Instruct· LMArena data1,222.9
315Claude 3 Sonnet20240229· LMArena data1,222.6
316Nova Micro 1.0· LMArena data1,217.6
317Gemini 1.5 Flash-8B001· LMArena data1,217.6
318Gemma 2 27B IT· LMArena data1,210.8
319OLMo 2 32B Instruct0325· LMArena data1,210
320Nemotron-4 340B Instruct· LMArena data1,209.2
321GPT 40314· LMArena data1,208.6
322Llama 3 70B Instruct· LMArena data1,206
323Ministral 8B2410· LMArena data1,201.8
324Claude 3 Haiku20240307· LMArena data1,198.9
325Aya Expanse 32B· LMArena data1,196.5
326Qwen2 72B Instruct· LMArena data1,195.7
327Llama 3.1 8B Instruct· LMArena data1,194.7
328Gemma 2 9B IT SimPO· LMArena data1,190.8
329Reka Flash20240904· LMArena data1,189.9
330GPT 40613· LMArena data1,187.3
331Command R+08-2024· LMArena data1,187
332Granite 3.1 8B Instruct· LMArena data1,186.3
333Qwen1.5-110B-Chat· LMArena data1,183.8
334Llama 3.1 Tulu 3 8B· LMArena data1,183.3
335Mistral Large2402· LMArena data1,182.9
336Jamba 1.5 Mini· LMArena data1,178.2
337Gemma 2 9B IT· LMArena data1,173.4
338Command R+· LMArena data1,171.7
339Yi 1.5 34B Chat· LMArena data1,169.2
340Mixtral 8x22B Instruct v0.1· LMArena data1,165.5
341Qwen1.5-72B-Chat· LMArena data1,165.4
342Reka Flash 21BOnline, 20240226· LMArena data1,164.4
343Mistral Medium· LMArena data1,162
344Aya Expanse 8B· LMArena data1,160.4
345InternLM2.5 20B Chat· LMArena data1,158.9
346QwQ 32B Preview· LMArena data1,155.1
347Qwen1.5-32B-Chat· LMArena data1,154.6
348Reka Flash 21B20240226· LMArena data1,153.6
349Llama 3 8B Instruct· LMArena data1,151.5
350Granite 3.1 2B Instruct· LMArena data1,149.2

Showing 301–350 of 418 results.

Data credits: LMArena, CC BY 4.0 (licence)