Skip to content

Benchmark

FictionLiveBench 400

Long context benchmarks measure how well models use information in long inputs. Scores are percentages.

Models measured
54
Scores recorded
57
Sources
1
IQ weight
1.45

Score distribution

79.9%
Mean
77.8%
Median
16.7%
Std dev
58.3%
10th percentile
100%
90th percentile
44.4%
Min
44.4%100%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.45 for chat models, measured over 54 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Sonnet 3.78k· Epoch AI Benchmarking Hub data100%
2DeepSeek V3.2 Exphigh· Epoch AI Benchmarking Hub data100%
3Gemini 2.5 Pro· Epoch AI Benchmarking Hub data100%
4Gemini 2.5 Pro· Epoch AI Benchmarking Hub data100%
5Grok 4 Fast· Epoch AI Benchmarking Hub data100%
6Kimi K2.5· Epoch AI Benchmarking Hub data100%
7O3 Promedium· Epoch AI Benchmarking Hub data100%
8o4 Minimedium· Epoch AI Benchmarking Hub data100%
9OpenAI o3medium· Epoch AI Benchmarking Hub data100%
10Qwen 3 235B A22B Thinking 2507· Epoch AI Benchmarking Hub data100%
11Gemini 2.5 Flash Preview 05-20 (no think)· Epoch AI Benchmarking Hub data97.2%
12Gemini 2.5 Pro Preview 05-06· Epoch AI Benchmarking Hub data97.2%
13GLM 4.5· Epoch AI Benchmarking Hub data97.2%
14GPT 5medium· Epoch AI Benchmarking Hub data97.2%
15GPT-5 minimedium· Epoch AI Benchmarking Hub data97.2%
16Grok 4· Epoch AI Benchmarking Hub data97.2%
17o1medium· Epoch AI Benchmarking Hub data97.2%
18GPT-4.5 Preview· Epoch AI Benchmarking Hub data94.4%
19DeepSeek R1· Epoch AI Benchmarking Hub data91.7%
20Gemini 2.5 Pro Preview 03-25· Epoch AI Benchmarking Hub data91.7%
21gemini-2.0-pro-exp-02-05· Epoch AI Benchmarking Hub data91.7%
22GPT 4.1· Epoch AI Benchmarking Hub data91.7%
23QwQ 32B· Epoch AI Benchmarking Hub data91.7%
24Qwen3 32B· Epoch AI Benchmarking Hub data90.9%
25Qwen 3 235B A22B· Epoch AI Benchmarking Hub data90%
26MiniMax M1 80K· Epoch AI Benchmarking Hub data86.1%
27Gemini 2.0 Flash Experimentalthinking· Epoch AI Benchmarking Hub data83.3%
28Claude Opus 4· Epoch AI Benchmarking Hub data77.8%
29Claude Sonnet 3.7· Epoch AI Benchmarking Hub data77.8%
30Claude Sonnet 4· Epoch AI Benchmarking Hub data77.8%
31Grok 3 Minimedium, beta· Epoch AI Benchmarking Hub data77.8%
32Qwen3 8B· Epoch AI Benchmarking Hub data77.4%
33GPT OSS 120Bhigh· Epoch AI Benchmarking Hub data75%
34Kimi K2 Instruct· Epoch AI Benchmarking Hub data75%
35Qwen3 Next 80B A3B Instruct· Epoch AI Benchmarking Hub data75%
36Grok 3beta· Epoch AI Benchmarking Hub data72.2%
37GPT OSS 120B· Epoch AI Benchmarking Hub data69.4%
38Qwen Max· Epoch AI Benchmarking Hub data69.4%
39Qwen3 Max· Epoch AI Benchmarking Hub data69.4%
40DeepSeek V3.1· Epoch AI Benchmarking Hub data66.7%
41Gemini 2.5 Flash-Litethinking· Epoch AI Benchmarking Hub data66.7%
42GPT-4.1 mini· Epoch AI Benchmarking Hub data66.7%
43Llama 3.3 70B Instruct· Epoch AI Benchmarking Hub data66.7%
44Qwen3 14B· Epoch AI Benchmarking Hub data64.5%
45Gemini 2.0 Flash· Epoch AI Benchmarking Hub data63.9%
46gemini-2.5-flash-preview-04-17· Epoch AI Benchmarking Hub data63.9%
47o3 Minimedium· Epoch AI Benchmarking Hub data63.9%
48DeepSeek V3· Epoch AI Benchmarking Hub data61.1%
49Qwen3 235B A22B Instruct 2507· Epoch AI Benchmarking Hub data60%
50GPT-5 nanomedium· Epoch AI Benchmarking Hub data58.3%

Showing 1–50 of 57 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)