Skip to content

Benchmark

FictionLiveBench 1k

Long context benchmarks measure how well models use information in long inputs. Scores are percentages.

Models measured
54
Scores recorded
57
Sources
1
IQ weight
1.48

Score distribution

75.9%
Mean
75%
Median
17.6%
Std dev
52.8%
10th percentile
100%
90th percentile
41.7%
Min
41.7%100%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.48 for chat models, measured over 54 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Sonnet 3.78k· Epoch AI Benchmarking Hub data100%
2Gemini 2.5 Pro· Epoch AI Benchmarking Hub data100%
3Gemini 2.5 Pro· Epoch AI Benchmarking Hub data100%
4Grok 4· Epoch AI Benchmarking Hub data100%
5Kimi K2.5· Epoch AI Benchmarking Hub data100%
6o1medium· Epoch AI Benchmarking Hub data100%
7O3 Promedium· Epoch AI Benchmarking Hub data100%
8o4 Minimedium· Epoch AI Benchmarking Hub data100%
9OpenAI o3medium· Epoch AI Benchmarking Hub data100%
10DeepSeek V3.2 Exphigh· Epoch AI Benchmarking Hub data97.2%
11Gemini 2.5 Flash Preview 05-20 (no think)· Epoch AI Benchmarking Hub data94.4%
12GPT 5medium· Epoch AI Benchmarking Hub data94.4%
13Grok 4 Fast· Epoch AI Benchmarking Hub data94.4%
14MiniMax M1 80K· Epoch AI Benchmarking Hub data94.4%
15QwQ 32B· Epoch AI Benchmarking Hub data94.4%
16Qwen3 32B· Epoch AI Benchmarking Hub data93.8%
17Qwen 3 235B A22B Thinking 2507· Epoch AI Benchmarking Hub data91.7%
18Qwen 3 235B A22B· Epoch AI Benchmarking Hub data89.3%
19GPT-5 minimedium· Epoch AI Benchmarking Hub data88.9%
20Gemini 2.5 Pro Preview 05-06· Epoch AI Benchmarking Hub data86.1%
21GLM 4.5· Epoch AI Benchmarking Hub data86.1%
22DeepSeek R1· Epoch AI Benchmarking Hub data83.3%
23Gemini 2.5 Pro Preview 03-25· Epoch AI Benchmarking Hub data83.3%
24GPT-4.5 Preview· Epoch AI Benchmarking Hub data83.3%
25Claude Sonnet 3.7· Epoch AI Benchmarking Hub data80.6%
26gemini-2.0-pro-exp-02-05· Epoch AI Benchmarking Hub data80.6%
27Claude Opus 4· Epoch AI Benchmarking Hub data77.8%
28Grok 3 Minimedium, beta· Epoch AI Benchmarking Hub data77.8%
29GPT 4.1· Epoch AI Benchmarking Hub data75%
30Kimi K2 Instruct· Epoch AI Benchmarking Hub data72.2%
31Qwen3 Next 80B A3B Instruct· Epoch AI Benchmarking Hub data72.2%
32DeepSeek V3· Epoch AI Benchmarking Hub data69.4%
33gemini-2.5-flash-preview-04-17· Epoch AI Benchmarking Hub data69.4%
34Llama 3.3 70B Instruct· Epoch AI Benchmarking Hub data69.4%
35Qwen Max· Epoch AI Benchmarking Hub data69.4%
36Qwen3 Max· Epoch AI Benchmarking Hub data69.4%
37Gemini 2.0 Flash Experimentalthinking· Epoch AI Benchmarking Hub data66.7%
38DeepSeek V3.1· Epoch AI Benchmarking Hub data63.9%
39GPT-5 nanomedium· Epoch AI Benchmarking Hub data63.9%
40Grok 3beta· Epoch AI Benchmarking Hub data63.9%
41Qwen3 8B· Epoch AI Benchmarking Hub data63.3%
42Claude Sonnet 4· Epoch AI Benchmarking Hub data62.5%
43Qwen3 14B· Epoch AI Benchmarking Hub data61.8%
44GPT OSS 120Bhigh· Epoch AI Benchmarking Hub data61.1%
45Gemini 2.0 Flash· Epoch AI Benchmarking Hub data58.3%
46GPT OSS 120B· Epoch AI Benchmarking Hub data58.3%
47o3 Minimedium· Epoch AI Benchmarking Hub data58.3%
48GPT-4.1 mini· Epoch AI Benchmarking Hub data55.6%
49Nemotron Nano 9b V2· Epoch AI Benchmarking Hub data55.6%
50Qwen3 30B A3B· Epoch AI Benchmarking Hub data54.8%

Showing 1–50 of 57 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)