Skip to content

Benchmark

Dtbench Accuracy

Scores are percentages. Higher is better.

Models measured
170
Scores recorded
239
Sources
1
IQ weight
1.22

Score distribution

79.8%
Mean
84.5%
Median
16.6%
Std dev
52.5%
10th percentile
97.1%
90th percentile
41.7%
Min
41.7%99.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.22 for chat models, measured over 170 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
201Mistral Small2506· Epoch AI Benchmarking Hub data59.9%
202Qwen3 8B· Epoch AI Benchmarking Hub data59.7%
203Llama 3.3 70B Instruct· Epoch AI Benchmarking Hub data59.5%
204gemini-1.5-pro-002· Epoch AI Benchmarking Hub data59%
205Mistral Small2503· Epoch AI Benchmarking Hub data58.6%
206Llama 4 Scout· Epoch AI Benchmarking Hub data57.9%
207Claude Haiku 3.5· Epoch AI Benchmarking Hub data56.7%
208gemini-1.5-pro-001· Epoch AI Benchmarking Hub data56.5%
209Mistral Large2402· Epoch AI Benchmarking Hub data55.9%
210Mixtral 8x22B Instruct v0.1· Epoch AI Benchmarking Hub data55.1%
211GPT-4o mini· Epoch AI Benchmarking Hub data54.4%
212Llama 3 70B Instruct· Epoch AI Benchmarking Hub data54.2%
213gemini-1.5-flash-002· Epoch AI Benchmarking Hub data53.8%
214Claude 3 Sonnet· Epoch AI Benchmarking Hub data53.6%
215Mistral Small· Epoch AI Benchmarking Hub data53%
216Gemini 2.0 Flash Lite· Epoch AI Benchmarking Hub data52.5%
217Gemma 3 27B IT· Epoch AI Benchmarking Hub data52.5%
218GPT-4.1 nano· Epoch AI Benchmarking Hub data52.5%
219Claude 2· Epoch AI Benchmarking Hub data51.9%
220Ministral 3B· Epoch AI Benchmarking Hub data51.7%
221gemini-1.5-flash-001· Epoch AI Benchmarking Hub data51.4%
222Claude 2.1· Epoch AI Benchmarking Hub data51%
223Gemma 3 4B IT· Epoch AI Benchmarking Hub data50.9%
224Llama 3.1 8B Instruct· Epoch AI Benchmarking Hub data50.9%
225Claude 3 Haiku· Epoch AI Benchmarking Hub data50.1%
226gemini-1.5-flash-8b-001· Epoch AI Benchmarking Hub data50%
227open-mixtral-8x7b· Epoch AI Benchmarking Hub data49.6%
228Gemma 3 12B IT· Epoch AI Benchmarking Hub data48.8%
229Mistral Nemo· Epoch AI Benchmarking Hub data48.6%
230GPT-3.5 Turbo· Epoch AI Benchmarking Hub data48.5%
231gemma2-27b· Epoch AI Benchmarking Hub data48%
232Qwen2.5 7B Instruct· Epoch AI Benchmarking Hub data47.7%
233C4AI Command R 08 2024· Epoch AI Benchmarking Hub data46.4%
234Claude Instant 1.2· Epoch AI Benchmarking Hub data45.8%
235Ministral 8B· Epoch AI Benchmarking Hub data45.7%
236Llama 3 8B Instruct· Epoch AI Benchmarking Hub data43.9%
237open-mistral-7b· Epoch AI Benchmarking Hub data42.5%
238Llama 2 13B Chat· Epoch AI Benchmarking Hub data42.2%
239Llama 2 70B Chat· Epoch AI Benchmarking Hub data41.7%

Showing 201–239 of 239 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)