Skip to content

Benchmark

Arena ELO - Coding

Coding benchmarks compare models on programming tasks. Higher scores are better.

Models measured
361
Scores recorded
418
Sources
1
IQ weight
1.28

Score distribution

1,325
Mean
1,374.5
Median
162.1
Std dev
1,081.6
10th percentile
1,488
90th percentile
682.8
Min
682.81,564.3
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.28 for chat models, measured over 361 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
251Magistral Medium2506· LMArena data1,319.1
252Olmo 3 32B Think· LMArena data1,317.6
253Step-2 16K Exp202412· LMArena data1,316.1
254Qwen2.5-Plus1127· LMArena data1,314.1
255Granite 4.1 8B· LMArena data1,313
256Yi Lightning· LMArena data1,312.3
257Athene V2 Chat· LMArena data1,312.1
258Llama 3.1 Nemotron Ultra 253B v1· LMArena data1,311.7
259DeepSeek V2.51210· LMArena data1,309.3
260Mistral Small 3.1 24B Instruct2503· LMArena data1,308.9
261Hunyuan Large Vision· LMArena data1,308.2
262Hunyuan Large2025-02-10· LMArena data1,306.6
263Claude 3.5 Sonnet20240620· LMArena data1,306.4
264GPT OSS 20b· LMArena data1,306.4
265GPT-4.1 nano2025-04-14· LMArena data1,305.5
266Llama 4 Maverick· LMArena data1,301.3
267DeepSeek V2.5· LMArena data1,301.2
268GPT 4o2024-05-13· LMArena data1,297.1
269Llama 3.3 Nemotron Super 49B v1· LMArena data1,295.5
270Gemini 1.5 Pro002· LMArena data1,294.3
271Qwen2.5 72B Instruct· LMArena data1,292.4
272Llama 3.1 405B InstructBF16· LMArena data1,291.2
273GLM 4 Plus· LMArena data1,290.4
274Olmo 3.1 32B Think· LMArena data1,290.3
275GPT-4o mini2024-07-18· LMArena data1,289.6
276Qwen Max0919· LMArena data1,288
277Grok 22024-08-13· LMArena data1,287.4
278GLM 4 Plus0111· LMArena data1,286.6
279Claude Haiku 3.520241022· LMArena data1,286.3
280Llama 4 Scout· LMArena data1,285.6
281GPT 4o2024-08-06· LMArena data1,283.4
282Llama 3.1 405B InstructFP8· LMArena data1,283.4
283Gemma 3 12B IT· LMArena data1,281
284Mistral Large 22407· LMArena data1,276.6
285Qwen 2.5 Coder 32B Instruct· LMArena data1,275.6
286Mistral Large2411· LMArena data1,275.4
287Llama 3.1 Nemotron 70B Instruct· LMArena data1,271.7
288Nova Pro 1.0· LMArena data1,269.8
289Hunyuan Standard2025-02-10· LMArena data1,269.6
290Grok 2 mini2024-08-13· LMArena data1,268.5
291GPT-4 Turbo2024-04-09· LMArena data1,268.3
292Gemma 3n E4B IT· LMArena data1,268.3
293Llama 3.3 70B Instruct· LMArena data1,268
294Athene 70B0725· LMArena data1,266.9
295Gemini 1.5 Pro001· LMArena data1,266.2
296Claude 3 Opus20240229· LMArena data1,264.1
297Gemini 1.5 Flash002· LMArena data1,261.4
298Llama 3.1 70B Instruct· LMArena data1,259.6
299Gemini Advanced0514· LMArena data1,254.7
300GPT-4 Turbo Preview1106· LMArena data1,253.9

Showing 251–300 of 418 results.

Data credits: LMArena, CC BY 4.0 (licence)