Skip to content

Benchmark

Dtbench EDT answer preference

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
170
Scores recorded
177
Sources
1
IQ weight
0.99

Score distribution

67.8%
Mean
69%
Median
11.3%
Std dev
53%
10th percentile
81.6%
90th percentile
43.3%
Min
43.3%96.1%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.99 for chat models, measured over 170 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
151gemini-1.5-flash-002· Epoch AI Benchmarking Hub data54.9%
152Llama 4 Scout· Epoch AI Benchmarking Hub data54.7%
153Mistral Nemo· Epoch AI Benchmarking Hub data54.5%
154Gemini 2.0 Flash Lite· Epoch AI Benchmarking Hub data54.4%
155Llama 3 8B Instruct· Epoch AI Benchmarking Hub data54.1%
156Claude 2· Epoch AI Benchmarking Hub data53.9%
157Mistral Large2402· Epoch AI Benchmarking Hub data53.9%
158C4AI Command A 03 2025· Epoch AI Benchmarking Hub data53.3%
159Muse Spark 1.3· Epoch AI Benchmarking Hub data53.3%
160GPT-5 nanohigh· Epoch AI Benchmarking Hub data53%
161Llama 3.3 70B Instruct· Epoch AI Benchmarking Hub data52.8%
162Llama 3.1 70B Instruct· Epoch AI Benchmarking Hub data52.7%
163open-mixtral-8x7b· Epoch AI Benchmarking Hub data51.1%
164Qwen2.5 7B Instruct· Epoch AI Benchmarking Hub data51%
165gemini-1.5-pro-001· Epoch AI Benchmarking Hub data51%
166Llama 2 70B Chat· Epoch AI Benchmarking Hub data51%
167Mixtral 8x22B Instruct v0.1· Epoch AI Benchmarking Hub data50.3%
168GPT OSS 20bhigh· Epoch AI Benchmarking Hub data48.7%
169Llama 3.1 8B Instruct· Epoch AI Benchmarking Hub data48.6%
170gemini-1.5-flash-8b-001· Epoch AI Benchmarking Hub data48.3%
171gemini-1.5-flash-001· Epoch AI Benchmarking Hub data48.2%
172GPT 5.4 Nanoxhigh· Epoch AI Benchmarking Hub data48%
173Ministral 8B· Epoch AI Benchmarking Hub data47.8%
174Gemini 2.0 Flash· Epoch AI Benchmarking Hub data47.6%
175open-mistral-7b· Epoch AI Benchmarking Hub data45.5%
176GPT-5.4 minixhigh· Epoch AI Benchmarking Hub data45.3%
177Llama 2 13B Chat· Epoch AI Benchmarking Hub data43.3%

Showing 151–177 of 177 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)