Skip to content

Benchmark

CyBench Guided

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
9
Scores recorded
9
Sources
2
IQ weight
0.43

Score distribution

11.6%
Mean
12.5%
Median
4.4%
Std dev
5%
10th percentile
17.5%
90th percentile
5%
Min
5%17.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.43 for chat models, measured over 9 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 4o· CyBench official data
Values from 2 sources
17.5%
2Claude 3.5 Sonnet· CyBench official data
Values from 2 sources
15%
3Llama 3.1 405B Instruct· CyBench official data
Values from 2 sources
15%
4Claude 3 Opus· CyBench official data
Values from 2 sources
12.5%
5O1 Preview· CyBench official data
Values from 2 sources
10%
6Llama 3 70B Instruct· Epoch AI Benchmarking Hub data7.5%
7Gemini 1.5 Pro· CyBench official data5%
8Mixtral 8x22B Instruct· CyBench official data5%
9Mixtral 8x22B Instruct v0.1· Epoch AI Benchmarking Hub data5%

Data credits: CyBench — cite Zhang et al. 2025 · Epoch AI Benchmarking Hub, CC BY 4.0 (licence)