Skip to content

Benchmark

LiveCodeBench Hard

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
16
Scores recorded
22
Sources
1
IQ weight
0.98

Score distribution

38.1%
Mean
46.9%
Median
21.9%
Std dev
4.3%
10th percentile
63.6%
90th percentile
1.9%
Min
1.9%71.1%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.98 for chat models, measured over 16 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1o4 Minidifficulty:hard, high· LiveCodeBench Leaderboard data71.1%
2OpenAI o3difficulty:hard, high· LiveCodeBench Leaderboard data66%
3DeepSeek R1difficulty:hard· LiveCodeBench Leaderboard data63.6%
4o4 Minidifficulty:hard, medium· LiveCodeBench Leaderboard data62.9%
5Gemini 2.5 Prodifficulty:hard· LiveCodeBench Leaderboard data62%
6Qwen 3 235B A22Bdifficulty:hard· LiveCodeBench Leaderboard data54%
7Grok 3 Minidifficulty:hard, high· LiveCodeBench Leaderboard data51.1%
8o4 Minidifficulty:hard, low· LiveCodeBench Leaderboard data48.9%
9Gemini 2.5 Flashdifficulty:hard· LiveCodeBench Leaderboard data48.3%
10o3 Minidifficulty:hard, high· LiveCodeBench Leaderboard data47.4%
11o3 Minidifficulty:hard, Med· LiveCodeBench Leaderboard data43.1%
12o3 Minidifficulty:hard, low· LiveCodeBench Leaderboard data38%
13Claude Opus 4difficulty:hard, thinking· LiveCodeBench Leaderboard data33.1%
14Claude Sonnet 4difficulty:hard, thinking· LiveCodeBench Leaderboard data32.6%
15Claude Opus 4difficulty:hard· LiveCodeBench Leaderboard data23.7%
16Claude Sonnet 4difficulty:hard· LiveCodeBench Leaderboard data20.6%
17DeepSeek V3difficulty:hard· LiveCodeBench Leaderboard data14.2%
18Claude 3.5 Sonnetdifficulty:hard· LiveCodeBench Leaderboard data12.8%
19GPT 4odifficulty:hard· LiveCodeBench Leaderboard data4.5%
20GPT-4o minidifficulty:hard· LiveCodeBench Leaderboard data4.3%
21GPT-4 Turbodifficulty:hard· LiveCodeBench Leaderboard data4%
22Claude 3 Haikudifficulty:hard· LiveCodeBench Leaderboard data1.9%