Skip to content

Benchmark

Forecastbench Market score

Scores are percentages.

Models measured
85
Scores recorded
95
Sources
1
IQ weight
1.29

Score distribution

67.1%
Mean
67.6%
Median
3.2%
Std dev
63.6%
10th percentile
70.3%
90th percentile
50.9%
Min
50.9%71.3%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.29 for chat models, measured over 85 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51GPT 5.5· Epoch AI Benchmarking Hub data67.5%
52Gemini 3.1 Pro Preview· Epoch AI Benchmarking Hub data67.4%
53Llama 3.1 405B Instruct· Epoch AI Benchmarking Hub data67.4%
54Gemini 2.5 Flash· Epoch AI Benchmarking Hub data67.3%
55GPT 4o· Epoch AI Benchmarking Hub data67.3%
56Mistral Large2411· Epoch AI Benchmarking Hub data67.3%
57Gemini 2.5 Pro Preview 03-25· Epoch AI Benchmarking Hub data67.2%
58Claude Opus 4.824k· Epoch AI Benchmarking Hub data66.9%
59GPT 4· Epoch AI Benchmarking Hub data66.9%
60Qwen2.5 72B Instruct· Epoch AI Benchmarking Hub data66.9%
61Qwen 3 235B A22B Thinking 2507· Epoch AI Benchmarking Hub data66.8%
62Claude Opus 4.6· Epoch AI Benchmarking Hub data66.7%
63GLM 5· Epoch AI Benchmarking Hub data66.6%
64Grok 4.3· Epoch AI Benchmarking Hub data66.6%
65Kimi K2 Instruct· Epoch AI Benchmarking Hub data66.4%
66Claude 3 Opus· Epoch AI Benchmarking Hub data66.3%
67GPT 5.1· Epoch AI Benchmarking Hub data66.3%
68Claude Sonnet 516k· Epoch AI Benchmarking Hub data66.1%
69GPT 5.4· Epoch AI Benchmarking Hub data66.1%
70GPT-5 mini· Epoch AI Benchmarking Hub data66.1%
71Grok 4.20· Epoch AI Benchmarking Hub data66.1%
72Gemini 3.1 Flash Lite· Epoch AI Benchmarking Hub data65.9%
73GPT 4.1· Epoch AI Benchmarking Hub data65.9%
74Llama 4 Scout· Epoch AI Benchmarking Hub data65.2%
75Claude 2.1· Epoch AI Benchmarking Hub data65.1%
76GPT-5.4 mini· Epoch AI Benchmarking Hub data65.1%
77Meta Llama 3 70b· Epoch AI Benchmarking Hub data65.1%
78Claude Opus 4.7· Epoch AI Benchmarking Hub data65%
79Gemma 4 31B IT· Epoch AI Benchmarking Hub data65%
80DeepSeek V4 Pro· Epoch AI Benchmarking Hub data64.7%
81Claude Opus 4.8· Epoch AI Benchmarking Hub data64.6%
82gemini-2.5-flash-preview-04-17· Epoch AI Benchmarking Hub data64.3%
83DeepSeek V3.1· Epoch AI Benchmarking Hub data64%
84gemini-1.5-flash-001· Epoch AI Benchmarking Hub data63.6%
85Llama 3 8B Instruct· Epoch AI Benchmarking Hub data63.6%
86Qwen1.5-110B-Chat· Epoch AI Benchmarking Hub data63.6%
87Mixtral 8x22B Instruct v0.1· Epoch AI Benchmarking Hub data63.5%
88Mixtral 8x7B Instruct v0.1· Epoch AI Benchmarking Hub data63.2%
89gemini-1.5-pro-001· Epoch AI Benchmarking Hub data62%
90GPT 5.4 Nano· Epoch AI Benchmarking Hub data61.3%
91Gemini 2.0 Flash Lite· Epoch AI Benchmarking Hub data61.2%
92GPT-5 nano· Epoch AI Benchmarking Hub data60.2%
93GPT-3.5 Turbo· Epoch AI Benchmarking Hub data60%
94Claude 3 Haiku· Epoch AI Benchmarking Hub data58.3%
95Llama-2-70b-chat-hf· Epoch AI Benchmarking Hub data50.9%

Showing 51–95 of 95 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)