Skip to content

Benchmark

LiveCodeBench AtCoder

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
16
Scores recorded
22
Sources
1
IQ weight
1.41

Score distribution

61.9%
Mean
67.9%
Median
17%
Std dev
35.1%
10th percentile
79.2%
90th percentile
24%
Min
24%83.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.41 for chat models, measured over 16 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1o4 Miniplatform:atcoder, high· LiveCodeBench Leaderboard data83.2%
2DeepSeek R1platform:atcoder· LiveCodeBench Leaderboard data80.5%
3Gemini 2.5 Proplatform:atcoder· LiveCodeBench Leaderboard data79.2%
4o4 Miniplatform:atcoder, medium· LiveCodeBench Leaderboard data78.7%
5OpenAI o3platform:atcoder, high· LiveCodeBench Leaderboard data78.7%
6Qwen 3 235B A22Bplatform:atcoder· LiveCodeBench Leaderboard data74.9%
7o4 Miniplatform:atcoder, low· LiveCodeBench Leaderboard data72.8%
8Grok 3 Miniplatform:atcoder, high· LiveCodeBench Leaderboard data72.3%
9Gemini 2.5 Flashplatform:atcoder· LiveCodeBench Leaderboard data70.9%
10o3 Miniplatform:atcoder, high· LiveCodeBench Leaderboard data67.9%
11o3 Miniplatform:atcoder, Med· LiveCodeBench Leaderboard data66.8%
12o3 Miniplatform:atcoder, low· LiveCodeBench Leaderboard data65.5%
13Claude Opus 4platform:atcoder, thinking· LiveCodeBench Leaderboard data62.5%
14Claude Sonnet 4platform:atcoder, thinking· LiveCodeBench Leaderboard data61.8%
15Claude Opus 4platform:atcoder· LiveCodeBench Leaderboard data53.2%
16Claude Sonnet 4platform:atcoder· LiveCodeBench Leaderboard data52.5%
17DeepSeek V3platform:atcoder· LiveCodeBench Leaderboard data44.3%
18Claude 3.5 Sonnetplatform:atcoder· LiveCodeBench Leaderboard data43.4%
19GPT-4 Turboplatform:atcoder· LiveCodeBench Leaderboard data36.5%
20GPT-4o miniplatform:atcoder· LiveCodeBench Leaderboard data35.1%
21GPT 4oplatform:atcoder· LiveCodeBench Leaderboard data33.6%
22Claude 3 Haikuplatform:atcoder· LiveCodeBench Leaderboard data24%