Skip to content

Benchmark

Forecastbench Overall score

Scores are percentages.

Models measured
85
Scores recorded
95
Sources
1
IQ weight
1.84

Score distribution

62.5%
Mean
63.1%
Median
3.2%
Std dev
58.5%
10th percentile
65.7%
90th percentile
46%
Min
46%66.6%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.84 for chat models, measured over 85 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51Grok 4.20· Epoch AI Benchmarking Hub data63%
52Claude Sonnet 516k· Epoch AI Benchmarking Hub data62.9%
53GPT-5 mini· Epoch AI Benchmarking Hub data62.9%
54Claude 3.5 Sonnet· Epoch AI Benchmarking Hub data62.8%
55GPT-4 Turbo· Epoch AI Benchmarking Hub data62.8%
56GPT 5.4· Epoch AI Benchmarking Hub data62.7%
57Gemini 2.5 Flash· Epoch AI Benchmarking Hub data62.6%
58Claude Opus 4.7· Epoch AI Benchmarking Hub data62.4%
59Gemini 3.1 Flash Lite· Epoch AI Benchmarking Hub data62.4%
60Claude Opus 4.824k· Epoch AI Benchmarking Hub data62.3%
61gemini-2.5-flash-preview-04-17· Epoch AI Benchmarking Hub data62.3%
62DeepSeek V3· Epoch AI Benchmarking Hub data62.2%
63Kimi K2 Instruct· Epoch AI Benchmarking Hub data62.2%
64Llama 3.1 405B Instruct· Epoch AI Benchmarking Hub data62.2%
65Claude 3 Opus· Epoch AI Benchmarking Hub data62%
66GPT 4.1· Epoch AI Benchmarking Hub data62%
67GPT 5.1· Epoch AI Benchmarking Hub data61.9%
68Qwen 3 235B A22B Thinking 2507· Epoch AI Benchmarking Hub data61.9%
69GLM 4.5 Air· Epoch AI Benchmarking Hub data61.6%
70Claude Opus 4.8· Epoch AI Benchmarking Hub data61.5%
71Mistral Large2407· Epoch AI Benchmarking Hub data61.4%
72Qwen2.5 72B Instruct· Epoch AI Benchmarking Hub data61.2%
73GPT 4· Epoch AI Benchmarking Hub data61.1%
74Mistral Large2411· Epoch AI Benchmarking Hub data61.1%
75Gemma 4 31B IT· Epoch AI Benchmarking Hub data61%
76GPT 4o· Epoch AI Benchmarking Hub data60.9%
77Llama 3.3 70B Instruct· Epoch AI Benchmarking Hub data60.9%
78GPT-5.4 mini· Epoch AI Benchmarking Hub data60.8%
79DeepSeek V4 Pro· Epoch AI Benchmarking Hub data60.6%
80DeepSeek V3.1· Epoch AI Benchmarking Hub data60.4%
81Meta Llama 3 70b· Epoch AI Benchmarking Hub data59.8%
82Qwen1.5-110B-Chat· Epoch AI Benchmarking Hub data59.2%
83Llama 3 8B Instruct· Epoch AI Benchmarking Hub data59.1%
84Llama 4 Scout· Epoch AI Benchmarking Hub data58.8%
85GPT-5 nano· Epoch AI Benchmarking Hub data58.6%
86GPT 5.4 Nano· Epoch AI Benchmarking Hub data58.5%
87Mixtral 8x22B Instruct v0.1· Epoch AI Benchmarking Hub data58.5%
88Claude 2.1· Epoch AI Benchmarking Hub data58.4%
89gemini-1.5-pro-001· Epoch AI Benchmarking Hub data58.4%
90gemini-1.5-flash-001· Epoch AI Benchmarking Hub data58.3%
91Mixtral 8x7B Instruct v0.1· Epoch AI Benchmarking Hub data57.9%
92Gemini 2.0 Flash Lite· Epoch AI Benchmarking Hub data56.7%
93Claude 3 Haiku· Epoch AI Benchmarking Hub data54.5%
94Llama-2-70b-chat-hf· Epoch AI Benchmarking Hub data50.2%
95GPT-3.5 Turbo· Epoch AI Benchmarking Hub data46%

Showing 51–95 of 95 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)