Skip to content

Benchmark

Forecastbench Dataset score

Scores are percentages.

Models measured
85
Scores recorded
95
Sources
1
IQ weight
1.74

Score distribution

58.5%
Mean
59.8%
Median
3.7%
Std dev
54.9%
10th percentile
61.7%
90th percentile
35%
Min
35%63.7%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.74 for chat models, measured over 85 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51GPT 5.4· Epoch AI Benchmarking Hub data59.6%
52Qwen 3 235B A22B· Epoch AI Benchmarking Hub data59.6%
53Claude Sonnet 3.7· Epoch AI Benchmarking Hub data59.1%
54Gemini 3.1 Flash Lite· Epoch AI Benchmarking Hub data59.1%
55Claude Sonnet 4· Epoch AI Benchmarking Hub data58.8%
56o3 Mini· Epoch AI Benchmarking Hub data58.7%
57Claude Haiku 4.5· Epoch AI Benchmarking Hub data58.6%
58Claude Opus 4.8· Epoch AI Benchmarking Hub data58.6%
59Claude 3.5 Sonnet· Epoch AI Benchmarking Hub data58.4%
60GPT 4.1· Epoch AI Benchmarking Hub data58.4%
61Kimi K2 Instruct· Epoch AI Benchmarking Hub data58.4%
62Gemini 2.5 Flash· Epoch AI Benchmarking Hub data58.3%
63Claude Opus 4.824k· Epoch AI Benchmarking Hub data58.2%
64Claude 3 Opus· Epoch AI Benchmarking Hub data58.1%
65GPT 5.1· Epoch AI Benchmarking Hub data58%
66GPT-4 Turbo· Epoch AI Benchmarking Hub data57.7%
67Llama 3.1 405B Instruct· Epoch AI Benchmarking Hub data57.6%
68Qwen 3 235B A22B Thinking 2507· Epoch AI Benchmarking Hub data57.6%
69Gemma 4 31B IT· Epoch AI Benchmarking Hub data57.4%
70DeepSeek V3· Epoch AI Benchmarking Hub data57.1%
71DeepSeek V3.1· Epoch AI Benchmarking Hub data57.1%
72GPT-5 nano· Epoch AI Benchmarking Hub data57.1%
73DeepSeek V4 Pro· Epoch AI Benchmarking Hub data56.9%
74GPT-5.4 mini· Epoch AI Benchmarking Hub data56.9%
75GLM 4.5 Air· Epoch AI Benchmarking Hub data56.2%
76Qwen2.5 72B Instruct· Epoch AI Benchmarking Hub data56.2%
77GPT 4· Epoch AI Benchmarking Hub data56.1%
78GPT 5.4 Nano· Epoch AI Benchmarking Hub data55.8%
79Mistral Large2407· Epoch AI Benchmarking Hub data55.8%
80Mistral Large2411· Epoch AI Benchmarking Hub data55.8%
81GPT 4o· Epoch AI Benchmarking Hub data55.3%
82gemini-1.5-pro-001· Epoch AI Benchmarking Hub data55.1%
83Meta Llama 3 70b· Epoch AI Benchmarking Hub data55.1%
84Qwen1.5-110B-Chat· Epoch AI Benchmarking Hub data55.1%
85Llama 3.3 70B Instruct· Epoch AI Benchmarking Hub data55%
86Llama 3 8B Instruct· Epoch AI Benchmarking Hub data54.9%
87Mixtral 8x22B Instruct v0.1· Epoch AI Benchmarking Hub data54%
88gemini-1.5-flash-001· Epoch AI Benchmarking Hub data53.6%
89Mixtral 8x7B Instruct v0.1· Epoch AI Benchmarking Hub data53.3%
90Llama 4 Scout· Epoch AI Benchmarking Hub data53.2%
91Claude 2.1· Epoch AI Benchmarking Hub data52.6%
92Gemini 2.0 Flash Lite· Epoch AI Benchmarking Hub data52.5%
93Claude 3 Haiku· Epoch AI Benchmarking Hub data51%
94Llama-2-70b-chat-hf· Epoch AI Benchmarking Hub data49.4%
95GPT-3.5 Turbo· Epoch AI Benchmarking Hub data35%

Showing 51–95 of 95 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)