Skip to content

Benchmark

FictionLiveBench 0

Long context benchmarks measure how well models use information in long inputs. Scores are percentages.

Models measured
54
Scores recorded
55
Sources
1
IQ weight
1.27

Score distribution

91.1%
Mean
100%
Median
12.5%
Std dev
75%
10th percentile
100%
90th percentile
50%
Min
50%100%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.27 for chat models, measured over 54 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4· Epoch AI Benchmarking Hub data100%
2Claude Sonnet 3.7· Epoch AI Benchmarking Hub data100%
3Claude Sonnet 4· Epoch AI Benchmarking Hub data100%
4DeepSeek R1· Epoch AI Benchmarking Hub data100%
5DeepSeek V3.2 Exphigh· Epoch AI Benchmarking Hub data100%
6Gemini 2.0 Flash· Epoch AI Benchmarking Hub data100%
7Gemini 2.0 Flash Experimentalthinking· Epoch AI Benchmarking Hub data100%
8Gemini 2.5 Flash Preview 05-20 (no think)· Epoch AI Benchmarking Hub data100%
9Gemini 2.5 Flash-Litethinking· Epoch AI Benchmarking Hub data100%
10Gemini 2.5 Pro· Epoch AI Benchmarking Hub data100%
11Gemini 2.5 Pro· Epoch AI Benchmarking Hub data100%
12Gemini 2.5 Pro Preview 05-06· Epoch AI Benchmarking Hub data100%
13GLM 4.5· Epoch AI Benchmarking Hub data100%
14GPT 4.1· Epoch AI Benchmarking Hub data100%
15GPT 5medium· Epoch AI Benchmarking Hub data100%
16GPT OSS 120B· Epoch AI Benchmarking Hub data100%
17GPT-4.5 Preview· Epoch AI Benchmarking Hub data100%
18GPT-5 minimedium· Epoch AI Benchmarking Hub data100%
19Grok 4· Epoch AI Benchmarking Hub data100%
20Grok 4 Fast· Epoch AI Benchmarking Hub data100%
21Kimi K2.5· Epoch AI Benchmarking Hub data100%
22Llama 4 Maverick· Epoch AI Benchmarking Hub data100%
23o1medium· Epoch AI Benchmarking Hub data100%
24o3 Minimedium· Epoch AI Benchmarking Hub data100%
25O3 Promedium· Epoch AI Benchmarking Hub data100%
26o4 Minimedium· Epoch AI Benchmarking Hub data100%
27OpenAI o3medium· Epoch AI Benchmarking Hub data100%
28Qwen 3 235B A22B· Epoch AI Benchmarking Hub data100%
29Qwen 3 235B A22B Thinking 2507· Epoch AI Benchmarking Hub data100%
30Qwen3 8B· Epoch AI Benchmarking Hub data100%
31Qwen3 Next 80B A3B Instruct· Epoch AI Benchmarking Hub data100%
32QwQ 32B· Epoch AI Benchmarking Hub data100%
33DeepSeek V3· Epoch AI Benchmarking Hub data87.5%
34DeepSeek V3.1· Epoch AI Benchmarking Hub data87.5%
35Gemini 2.5 Pro Preview 03-25· Epoch AI Benchmarking Hub data87.5%
36gemini-2.0-pro-exp-02-05· Epoch AI Benchmarking Hub data87.5%
37Gemma 3 27B IT· Epoch AI Benchmarking Hub data87.5%
38Grok 3 Minimedium, beta· Epoch AI Benchmarking Hub data87.5%
39Kimi K2 Instruct· Epoch AI Benchmarking Hub data87.5%
40MiniMax M1 80K· Epoch AI Benchmarking Hub data87.5%
41Nemotron Nano 9b V2· Epoch AI Benchmarking Hub data87.5%
42Qwen3 235B A22B Instruct 2507· Epoch AI Benchmarking Hub data87.5%
43Qwen3 30B A3B· Epoch AI Benchmarking Hub data85.7%
44Qwen3 14B· Epoch AI Benchmarking Hub data83.3%
45Qwen3 32B· Epoch AI Benchmarking Hub data80%
46GPT-4.1 mini· Epoch AI Benchmarking Hub data75%
47GPT-5 nanomedium· Epoch AI Benchmarking Hub data75%
48Grok 3beta· Epoch AI Benchmarking Hub data75%
49Llama 3.3 70B Instruct· Epoch AI Benchmarking Hub data75%
50Qwen Max· Epoch AI Benchmarking Hub data75%

Showing 1–50 of 55 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)