Skip to content

Benchmark

FictionLiveBench 8k

Long context benchmarks measure how well models use information in long inputs. Scores are percentages.

Models measured
54
Scores recorded
57
Sources
1
IQ weight
2.06

Score distribution

64.3%
Mean
61.3%
Median
20.2%
Std dev
41.7%
10th percentile
94.4%
90th percentile
16.7%
Min
16.7%100%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.06 for chat models, measured over 54 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 5medium· Epoch AI Benchmarking Hub data100%
2Grok 4· Epoch AI Benchmarking Hub data100%
3O3 Promedium· Epoch AI Benchmarking Hub data100%
4OpenAI o3medium· Epoch AI Benchmarking Hub data100%
5Claude Sonnet 3.78k· Epoch AI Benchmarking Hub data97.2%
6Qwen 3 235B A22B Thinking 2507· Epoch AI Benchmarking Hub data97.2%
7DeepSeek V3.2 Exphigh· Epoch AI Benchmarking Hub data94.4%
8Grok 4 Fast· Epoch AI Benchmarking Hub data94.4%
9Gemini 2.5 Pro· Epoch AI Benchmarking Hub data91.7%
10Kimi K2.5· Epoch AI Benchmarking Hub data88.9%
11DeepSeek R1· Epoch AI Benchmarking Hub data86.1%
12o1medium· Epoch AI Benchmarking Hub data86.1%
13QwQ 32B· Epoch AI Benchmarking Hub data86.1%
14Gemini 2.5 Pro· Epoch AI Benchmarking Hub data80.6%
15Gemini 2.5 Pro Preview 03-25· Epoch AI Benchmarking Hub data80.6%
16Qwen3 32B· Epoch AI Benchmarking Hub data80%
17Gemini 2.5 Flash Preview 05-20 (no think)· Epoch AI Benchmarking Hub data72.2%
18GPT-4.5 Preview· Epoch AI Benchmarking Hub data72.2%
19Grok 3 Minimedium, beta· Epoch AI Benchmarking Hub data72.2%
20Gemini 2.5 Pro Preview 05-06· Epoch AI Benchmarking Hub data69.4%
21MiniMax M1 80K· Epoch AI Benchmarking Hub data69.4%
22Claude Opus 4· Epoch AI Benchmarking Hub data66.7%
23GPT-5 minimedium· Epoch AI Benchmarking Hub data66.7%
24o4 Minimedium· Epoch AI Benchmarking Hub data66.7%
25Qwen 3 235B A22B· Epoch AI Benchmarking Hub data66.7%
26GLM 4.5· Epoch AI Benchmarking Hub data63.9%
27Qwen Max· Epoch AI Benchmarking Hub data63.9%
28Qwen3 Max· Epoch AI Benchmarking Hub data63.9%
29Qwen3 8B· Epoch AI Benchmarking Hub data61.3%
30Kimi K2 Instruct· Epoch AI Benchmarking Hub data58.3%
31Qwen3 Next 80B A3B Instruct· Epoch AI Benchmarking Hub data58.3%
32Claude Sonnet 4· Epoch AI Benchmarking Hub data55.6%
33DeepSeek V3.1· Epoch AI Benchmarking Hub data55.6%
34GPT 4.1· Epoch AI Benchmarking Hub data55.6%
35Claude Sonnet 3.7· Epoch AI Benchmarking Hub data52.8%
36DeepSeek V3· Epoch AI Benchmarking Hub data52.8%
37DeepSeek V3.2 Exp· Epoch AI Benchmarking Hub data52.8%
38Gemini 2.0 Flash Experimentalthinking· Epoch AI Benchmarking Hub data52.8%
39gemini-2.0-pro-exp-02-05· Epoch AI Benchmarking Hub data52.8%
40GPT-5 nanomedium· Epoch AI Benchmarking Hub data52.8%
41Grok 3beta· Epoch AI Benchmarking Hub data52.8%
42Qwen3 14B· Epoch AI Benchmarking Hub data51.6%
43Gemini 2.0 Flash· Epoch AI Benchmarking Hub data50%
44o3 Minimedium· Epoch AI Benchmarking Hub data50%
45Qwen3 235B A22B Instruct 2507· Epoch AI Benchmarking Hub data50%
46Qwen3 30B A3B· Epoch AI Benchmarking Hub data50%
47Llama 4 Maverick· Epoch AI Benchmarking Hub data48%
48Gemini 2.5 Flash-Litethinking· Epoch AI Benchmarking Hub data47.2%
49gemini-2.5-flash-preview-04-17· Epoch AI Benchmarking Hub data44.4%
50GPT OSS 120Bhigh· Epoch AI Benchmarking Hub data44.4%

Showing 1–50 of 57 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)