Skip to content

Benchmark

FictionLiveBench 60k

Long context benchmarks measure how well models use information in long inputs. Scores are percentages.

Models measured
47
Scores recorded
49
Sources
1
IQ weight
2.12

Score distribution

57%
Mean
55.6%
Median
19.6%
Std dev
33.3%
10th percentile
83.3%
90th percentile
22.2%
Min
22.2%100%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.12 for chat models, measured over 47 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 5medium· Epoch AI Benchmarking Hub data100%
2Grok 4· Epoch AI Benchmarking Hub data97.2%
3O3 Promedium· Epoch AI Benchmarking Hub data94.4%
4Kimi K2.5· Epoch AI Benchmarking Hub data88.9%
5Gemini 2.5 Pro· Epoch AI Benchmarking Hub data83.3%
6Gemini 2.5 Pro· Epoch AI Benchmarking Hub data83.3%
7OpenAI o3medium· Epoch AI Benchmarking Hub data83.3%
8Grok 4 Fast· Epoch AI Benchmarking Hub data80.6%
9Qwen 3 235B A22B Thinking 2507· Epoch AI Benchmarking Hub data77.8%
10Grok 3 Minimedium, beta· Epoch AI Benchmarking Hub data72.2%
11o1medium· Epoch AI Benchmarking Hub data72.2%
12DeepSeek V3.2 Exphigh· Epoch AI Benchmarking Hub data71.4%
13Claude Sonnet 3.78k· Epoch AI Benchmarking Hub data69.4%
14Gemini 2.5 Flash Preview 05-20 (no think)· Epoch AI Benchmarking Hub data69.4%
15GPT-4.5 Preview· Epoch AI Benchmarking Hub data66.7%
16o4 Minimedium· Epoch AI Benchmarking Hub data66.7%
17Grok 3beta· Epoch AI Benchmarking Hub data63.9%
18Gemini 2.5 Pro Preview 05-06· Epoch AI Benchmarking Hub data61.1%
19GPT-5 minimedium· Epoch AI Benchmarking Hub data61.1%
20QwQ 32B· Epoch AI Benchmarking Hub data61.1%
21DeepSeek R1· Epoch AI Benchmarking Hub data58.3%
22Gemini 2.5 Pro Preview 03-25· Epoch AI Benchmarking Hub data58.3%
23gemini-2.5-flash-preview-04-17· Epoch AI Benchmarking Hub data58.3%
24MiniMax M1 80K· Epoch AI Benchmarking Hub data58.3%
25Claude Opus 4· Epoch AI Benchmarking Hub data55.6%
26DeepSeek V3· Epoch AI Benchmarking Hub data55.6%
27Kimi K2 Instruct· Epoch AI Benchmarking Hub data55.6%
28Qwen3 Next 80B A3B Instruct· Epoch AI Benchmarking Hub data55.6%
29GPT 4.1· Epoch AI Benchmarking Hub data52.8%
30DeepSeek V3.1· Epoch AI Benchmarking Hub data50%
31Qwen3 235B A22B Instruct 2507· Epoch AI Benchmarking Hub data50%
32DeepSeek V3.2 Exp· Epoch AI Benchmarking Hub data47.2%
33Gemini 2.0 Flash· Epoch AI Benchmarking Hub data47.2%
34GLM 4.5· Epoch AI Benchmarking Hub data47.2%
35Claude Sonnet 3.7· Epoch AI Benchmarking Hub data44.4%
36o3 Minimedium· Epoch AI Benchmarking Hub data44.4%
37gemini-2.0-pro-exp-02-05· Epoch AI Benchmarking Hub data41.7%
38GPT-4.1 mini· Epoch AI Benchmarking Hub data38.9%
39Claude Sonnet 4· Epoch AI Benchmarking Hub data37.5%
40Gemini 2.0 Flash Experimentalthinking· Epoch AI Benchmarking Hub data36.1%
41GPT-4.1 nano· Epoch AI Benchmarking Hub data36.1%
42GPT-5 nanomedium· Epoch AI Benchmarking Hub data36.1%
43GPT OSS 120B· Epoch AI Benchmarking Hub data33.3%
44Llama 3.3 70B Instruct· Epoch AI Benchmarking Hub data33.3%
45Llama 4 Maverick· Epoch AI Benchmarking Hub data32%
46Gemma 3 27B IT· Epoch AI Benchmarking Hub data30.6%
47Gemini 2.5 Flash-Litethinking· Epoch AI Benchmarking Hub data27.8%
48Llama 4 Scout· Epoch AI Benchmarking Hub data24%
49Nemotron Nano 9b V2· Epoch AI Benchmarking Hub data22.2%

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)