Skip to content

Benchmark

LiveBench Coding

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
42
Scores recorded
46
Sources
1
IQ weight
2.03

Score distribution

49.4%
Mean
48.1%
Median
17.8%
Std dev
27.5%
10th percentile
72.5%
90th percentile
15.5%
Min
15.5%85.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.03 for chat models, measured over 41 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 2.5 Pro· Epoch AI Benchmarking Hub data85.9%
2o3 Minihigh· Epoch AI Benchmarking Hub data82.7%
3GPT-4.5 Preview· Epoch AI Benchmarking Hub data75.2%
4Claude Sonnet 3.7· Epoch AI Benchmarking Hub data74.5%
5GPT 5.1high· Epoch AI Benchmarking Hub data72.5%
6QwQ 32B· Epoch AI Benchmarking Hub data72.2%
7DeepSeek V3· Epoch AI Benchmarking Hub data70.9%
8o1high· Epoch AI Benchmarking Hub data69.7%
9Claude 3.5 Sonnet· Epoch AI Benchmarking Hub data67.1%
10DeepSeek R1· Epoch AI Benchmarking Hub data66.7%
11o3 Minimedium· Epoch AI Benchmarking Hub data65.4%
12Qwen2.5-Max· Epoch AI Benchmarking Hub data64.4%
13gemini-2.0-pro-exp-02-05· Epoch AI Benchmarking Hub data63.5%
14gemini-exp· Epoch AI Benchmarking Hub data63.4%
15o3 Minilow· Epoch AI Benchmarking Hub data61.5%
16Qwen 2.5 Coder 32B Instruct· Epoch AI Benchmarking Hub data56.9%
17Gemini 2.0 Flash Experimental· Epoch AI Benchmarking Hub data54.4%
18Gemini 2.0 Flash· Epoch AI Benchmarking Hub data53.9%
19Gemini 2.0 Flash Experimentalthinking· Epoch AI Benchmarking Hub data53.5%
20DeepSeek R1 Distill Llama 70b· Epoch AI Benchmarking Hub data51.6%
21GPT 4o· Epoch AI Benchmarking Hub data51.4%
22Claude Haiku 3.5· Epoch AI Benchmarking Hub data51.4%
23O1 Minimedium· Epoch AI Benchmarking Hub data48.1%
24Gemini 2.0 Flash Lite· Epoch AI Benchmarking Hub data47.1%
25Mistral Large2411· Epoch AI Benchmarking Hub data47.1%
26Learnlm 1.5 Pro Experimental· Epoch AI Benchmarking Hub data46.9%
27Grok 2· Epoch AI Benchmarking Hub data46.4%
28Gemini 2.0 Flash-Lite Preview· Epoch AI Benchmarking Hub data43.8%
29GPT-4o mini· Epoch AI Benchmarking Hub data43.2%
30Gemma 3 27B IT· Epoch AI Benchmarking Hub data39.9%
31Claude 3 Opus· Epoch AI Benchmarking Hub data38.6%
32Nova Prov1· Epoch AI Benchmarking Hub data38.2%
33QwQ 32B Preview· Epoch AI Benchmarking Hub data37.2%
34Llama 3.3 70B Instruct· Epoch AI Benchmarking Hub data36.6%
35Mistral Small2503· Epoch AI Benchmarking Hub data36.2%
36Gemma 2 27B IT· Epoch AI Benchmarking Hub data36%
37Mistral Small· Epoch AI Benchmarking Hub data35.3%
38Perplexity Sonar· Epoch AI Benchmarking Hub data35.2%
39DeepSeek R1 Distill Qwen 32B· Epoch AI Benchmarking Hub data33.7%
40Phi 4· Epoch AI Benchmarking Hub data30.7%
41Nova Litev1· Epoch AI Benchmarking Hub data27.5%
42Gemma 2 9B IT· Epoch AI Benchmarking Hub data22.5%
43Phi-3 Small 8K Instruct· Epoch AI Benchmarking Hub data20.3%
44Nova Microv1· Epoch AI Benchmarking Hub data20.2%
45C4AI Command R 08 2024· Epoch AI Benchmarking Hub data17.9%
46Phi-3 Mini 4K Instruct· Epoch AI Benchmarking Hub data15.5%

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)