Skip to content

Benchmark

FictionLiveBench 2k

Long context benchmarks measure how well models use information in long inputs. Scores are percentages.

Models measured
54
Scores recorded
57
Sources
1
IQ weight
1.68

Score distribution

70.7%
Mean
72.2%
Median
19.2%
Std dev
47.2%
10th percentile
97.2%
90th percentile
36%
Min
36%100%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.68 for chat models, measured over 54 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 2.5 Pro· Epoch AI Benchmarking Hub data100%
2Grok 4 Fast· Epoch AI Benchmarking Hub data100%
3Kimi K2.5· Epoch AI Benchmarking Hub data100%
4O3 Promedium· Epoch AI Benchmarking Hub data100%
5o4 Minimedium· Epoch AI Benchmarking Hub data100%
6OpenAI o3medium· Epoch AI Benchmarking Hub data100%
7Claude Sonnet 3.78k· Epoch AI Benchmarking Hub data97.2%
8DeepSeek V3.2 Exphigh· Epoch AI Benchmarking Hub data97.2%
9Gemini 2.5 Pro· Epoch AI Benchmarking Hub data97.2%
10GPT 5medium· Epoch AI Benchmarking Hub data97.2%
11o1medium· Epoch AI Benchmarking Hub data94.4%
12MiniMax M1 80K· Epoch AI Benchmarking Hub data91.7%
13Grok 4· Epoch AI Benchmarking Hub data88.9%
14QwQ 32B· Epoch AI Benchmarking Hub data88.9%
15Qwen 3 235B A22B Thinking 2507· Epoch AI Benchmarking Hub data86.1%
16Gemini 2.5 Pro Preview 05-06· Epoch AI Benchmarking Hub data83.3%
17GPT-4.5 Preview· Epoch AI Benchmarking Hub data83.3%
18DeepSeek R1· Epoch AI Benchmarking Hub data82.9%
19GPT-5 minimedium· Epoch AI Benchmarking Hub data80.6%
20Grok 3 Minimedium, beta· Epoch AI Benchmarking Hub data80.6%
21Qwen 3 235B A22B· Epoch AI Benchmarking Hub data80%
22Qwen3 32B· Epoch AI Benchmarking Hub data76.7%
23Gemini 2.0 Flash Experimentalthinking· Epoch AI Benchmarking Hub data75%
24Gemini 2.5 Flash Preview 05-20 (no think)· Epoch AI Benchmarking Hub data75%
25Gemini 2.5 Pro Preview 03-25· Epoch AI Benchmarking Hub data75%
26GLM 4.5· Epoch AI Benchmarking Hub data75%
27Claude Sonnet 3.7· Epoch AI Benchmarking Hub data72.2%
28gemini-2.0-pro-exp-02-05· Epoch AI Benchmarking Hub data72.2%
29Kimi K2 Instruct· Epoch AI Benchmarking Hub data72.2%
30GPT 4.1· Epoch AI Benchmarking Hub data69.4%
31Claude Opus 4· Epoch AI Benchmarking Hub data66.7%
32Claude Sonnet 4· Epoch AI Benchmarking Hub data66.7%
33Qwen3 8B· Epoch AI Benchmarking Hub data66.7%
34Qwen Max· Epoch AI Benchmarking Hub data63.9%
35Qwen3 Max· Epoch AI Benchmarking Hub data63.9%
36Qwen3 Next 80B A3B Instruct· Epoch AI Benchmarking Hub data63.9%
37gemini-2.5-flash-preview-04-17· Epoch AI Benchmarking Hub data61.1%
38Qwen3 14B· Epoch AI Benchmarking Hub data59.4%
39Gemini 2.0 Flash· Epoch AI Benchmarking Hub data55.6%
40GPT OSS 120Bhigh· Epoch AI Benchmarking Hub data55.6%
41Grok 3beta· Epoch AI Benchmarking Hub data55.6%
42Llama 3.3 70B Instruct· Epoch AI Benchmarking Hub data55.6%
43DeepSeek V3· Epoch AI Benchmarking Hub data52.8%
44DeepSeek V3.2 Exp· Epoch AI Benchmarking Hub data52.8%
45Gemini 2.5 Flash-Litethinking· Epoch AI Benchmarking Hub data52.8%
46GPT OSS 120B· Epoch AI Benchmarking Hub data52.8%
47Llama 4 Maverick· Epoch AI Benchmarking Hub data52%
48Qwen3 30B A3B· Epoch AI Benchmarking Hub data51.5%
49DeepSeek V3.1· Epoch AI Benchmarking Hub data50%
50GPT-5 nanomedium· Epoch AI Benchmarking Hub data50%

Showing 1–50 of 57 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)