Skip to content

Benchmark

LiveCodeBench Medium

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
16
Scores recorded
22
Sources
1
IQ weight
2.89

Score distribution

71.2%
Mean
82.5%
Median
24.4%
Std dev
30.7%
10th percentile
92.2%
90th percentile
9%
Min
9%92.7%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.89 for chat models, measured over 16 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1o4 Minidifficulty:medium, high· LiveCodeBench Leaderboard data92.7%
2Gemini 2.5 Prodifficulty:medium· LiveCodeBench Leaderboard data92.2%
3o4 Minidifficulty:medium, medium· LiveCodeBench Leaderboard data92.2%
4DeepSeek R1difficulty:medium· LiveCodeBench Leaderboard data90.9%
5OpenAI o3difficulty:medium, high· LiveCodeBench Leaderboard data89.8%
6Qwen 3 235B A22Bdifficulty:medium· LiveCodeBench Leaderboard data88.8%
7o3 Minidifficulty:medium, high· LiveCodeBench Leaderboard data87.5%
8o4 Minidifficulty:medium, low· LiveCodeBench Leaderboard data86.2%
9Grok 3 Minidifficulty:medium, high· LiveCodeBench Leaderboard data85.9%
10o3 Minidifficulty:medium, Med· LiveCodeBench Leaderboard data85.1%
11Gemini 2.5 Flashdifficulty:medium· LiveCodeBench Leaderboard data82.5%
12Claude Opus 4difficulty:medium, thinking· LiveCodeBench Leaderboard data80.4%
13o3 Minidifficulty:medium, low· LiveCodeBench Leaderboard data77.6%
14Claude Sonnet 4difficulty:medium, thinking· LiveCodeBench Leaderboard data76%
15Claude Opus 4difficulty:medium· LiveCodeBench Leaderboard data69.2%
16Claude Sonnet 4difficulty:medium· LiveCodeBench Leaderboard data64%
17DeepSeek V3difficulty:medium· LiveCodeBench Leaderboard data53.5%
18Claude 3.5 Sonnetdifficulty:medium· LiveCodeBench Leaderboard data44.7%
19GPT 4odifficulty:medium· LiveCodeBench Leaderboard data32.1%
20GPT-4 Turbodifficulty:medium· LiveCodeBench Leaderboard data30.7%
21GPT-4o minidifficulty:medium· LiveCodeBench Leaderboard data25.2%
22Claude 3 Haikudifficulty:medium· LiveCodeBench Leaderboard data9%