Skip to content

Benchmark

FictionLiveBench 120k

Long context benchmarks measure how well models use information in long inputs. Scores are percentages.

Models measured
39
Scores recorded
40
Sources
1
IQ weight
1.91

Score distribution

57.1%
Mean
59.4%
Median
21.5%
Std dev
33.3%
10th percentile
88.9%
90th percentile
18.8%
Min
18.8%100%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.91 for chat models, measured over 39 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1OpenAI o3medium· Epoch AI Benchmarking Hub data100%
2GPT 5medium· Epoch AI Benchmarking Hub data96.9%
3Grok 4· Epoch AI Benchmarking Hub data96.9%
4Gemini 2.5 Pro· Epoch AI Benchmarking Hub data90.6%
5O3 Promedium· Epoch AI Benchmarking Hub data88.9%
6Gemini 2.5 Pro· Epoch AI Benchmarking Hub data87.5%
7Kimi K2.5· Epoch AI Benchmarking Hub data78.1%
8Grok 4 Fast· Epoch AI Benchmarking Hub data75%
9Gemini 2.5 Pro Preview 03-25· Epoch AI Benchmarking Hub data71.9%
10Gemini 2.5 Pro Preview 05-06· Epoch AI Benchmarking Hub data71.9%
11Gemini 2.5 Flash Preview 05-20 (no think)· Epoch AI Benchmarking Hub data68.8%
12Qwen 3 235B A22B Thinking 2507· Epoch AI Benchmarking Hub data68.8%
13Grok 3 Minimedium, beta· Epoch AI Benchmarking Hub data65.6%
14GPT-4.5 Preview· Epoch AI Benchmarking Hub data63.9%
15Gemini 2.0 Flash· Epoch AI Benchmarking Hub data62.5%
16GPT 4.1· Epoch AI Benchmarking Hub data62.5%
17GPT-5 minimedium· Epoch AI Benchmarking Hub data62.5%
18o4 Minimedium· Epoch AI Benchmarking Hub data62.5%
19Qwen3 Next 80B A3B Instruct· Epoch AI Benchmarking Hub data62.5%
20MiniMax M1 80K· Epoch AI Benchmarking Hub data59.4%
21Grok 3beta· Epoch AI Benchmarking Hub data58.3%
22Claude Sonnet 3.78k· Epoch AI Benchmarking Hub data53.1%
23DeepSeek V3.1· Epoch AI Benchmarking Hub data53.1%
24gemini-2.5-flash-preview-04-17· Epoch AI Benchmarking Hub data53.1%
25o1medium· Epoch AI Benchmarking Hub data53.1%
26GPT-4.1 mini· Epoch AI Benchmarking Hub data46.9%
27Qwen3 235B A22B Instruct 2507· Epoch AI Benchmarking Hub data44.4%
28o3 Minimedium· Epoch AI Benchmarking Hub data43.8%
29Kimi K2 Instruct· Epoch AI Benchmarking Hub data40.6%
30Claude Opus 4· Epoch AI Benchmarking Hub data37.5%
31Gemini 2.0 Flash Experimentalthinking· Epoch AI Benchmarking Hub data37.5%
32gemini-2.0-pro-exp-02-05· Epoch AI Benchmarking Hub data37.5%
33Claude Sonnet 4· Epoch AI Benchmarking Hub data36.4%
34Llama 4 Maverick· Epoch AI Benchmarking Hub data36.4%
35Claude Sonnet 3.7· Epoch AI Benchmarking Hub data34.4%
36DeepSeek R1· Epoch AI Benchmarking Hub data33.3%
37Llama 4 Scout· Epoch AI Benchmarking Hub data27.3%
38Gemini 2.5 Flash-Litethinking· Epoch AI Benchmarking Hub data21.9%
39GPT-5 nanomedium· Epoch AI Benchmarking Hub data21.9%
40GPT-4.1 nano· Epoch AI Benchmarking Hub data18.8%

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)