Skip to content

Benchmark

FictionLiveBench 16k

Long context benchmarks measure how well models use information in long inputs. Scores are percentages.

Models measured
54
Scores recorded
57
Sources
1
IQ weight
1.89

Score distribution

62%
Mean
62.5%
Median
18.3%
Std dev
40.6%
10th percentile
86.1%
90th percentile
25%
Min
25%97.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.89 for chat models, measured over 54 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 5medium· Epoch AI Benchmarking Hub data97.2%
2O3 Promedium· Epoch AI Benchmarking Hub data97.2%
3Grok 4· Epoch AI Benchmarking Hub data94.4%
4Grok 4 Fast· Epoch AI Benchmarking Hub data94.4%
5Gemini 2.5 Pro· Epoch AI Benchmarking Hub data91.7%
6OpenAI o3medium· Epoch AI Benchmarking Hub data88.9%
7Kimi K2.5· Epoch AI Benchmarking Hub data86.1%
8Claude Sonnet 3.78k· Epoch AI Benchmarking Hub data83.3%
9DeepSeek V3.2 Exphigh· Epoch AI Benchmarking Hub data83.3%
10o1medium· Epoch AI Benchmarking Hub data83.3%
11QwQ 32B· Epoch AI Benchmarking Hub data83.3%
12Gemini 2.5 Flash Preview 05-20 (no think)· Epoch AI Benchmarking Hub data77.8%
13o4 Minimedium· Epoch AI Benchmarking Hub data77.8%
14DeepSeek R1· Epoch AI Benchmarking Hub data75%
15Qwen 3 235B A22B Thinking 2507· Epoch AI Benchmarking Hub data75%
16Qwen3 32B· Epoch AI Benchmarking Hub data74.2%
17GPT-5 minimedium· Epoch AI Benchmarking Hub data69.4%
18MiniMax M1 80K· Epoch AI Benchmarking Hub data69.4%
19Qwen 3 235B A22B· Epoch AI Benchmarking Hub data67.7%
20Gemini 2.5 Pro· Epoch AI Benchmarking Hub data66.7%
21Gemini 2.5 Pro Preview 03-25· Epoch AI Benchmarking Hub data66.7%
22Gemini 2.5 Pro Preview 05-06· Epoch AI Benchmarking Hub data66.7%
23Grok 3 Minimedium, beta· Epoch AI Benchmarking Hub data66.7%
24Kimi K2 Instruct· Epoch AI Benchmarking Hub data66.7%
25Qwen Max· Epoch AI Benchmarking Hub data66.7%
26Qwen3 Max· Epoch AI Benchmarking Hub data66.7%
27GPT 4.1· Epoch AI Benchmarking Hub data63.9%
28GPT-4.5 Preview· Epoch AI Benchmarking Hub data63.9%
29Qwen3 14B· Epoch AI Benchmarking Hub data62.5%
30Qwen3 8B· Epoch AI Benchmarking Hub data62.1%
31Claude Opus 4· Epoch AI Benchmarking Hub data61.1%
32Gemini 2.0 Flash· Epoch AI Benchmarking Hub data61.1%
33GLM 4.5· Epoch AI Benchmarking Hub data58.3%
34Grok 3beta· Epoch AI Benchmarking Hub data58.3%
35Qwen3 Next 80B A3B Instruct· Epoch AI Benchmarking Hub data55.6%
36Qwen3 235B A22B Instruct 2507· Epoch AI Benchmarking Hub data52.9%
37DeepSeek V3.1· Epoch AI Benchmarking Hub data52.8%
38DeepSeek V3.2 Exp· Epoch AI Benchmarking Hub data52.8%
39Gemini 2.0 Flash Experimentalthinking· Epoch AI Benchmarking Hub data52.8%
40Claude Sonnet 3.7· Epoch AI Benchmarking Hub data50%
41DeepSeek V3· Epoch AI Benchmarking Hub data50%
42o3 Minimedium· Epoch AI Benchmarking Hub data50%
43Gemini 2.5 Flash-Litethinking· Epoch AI Benchmarking Hub data47.2%
44gemini-2.5-flash-preview-04-17· Epoch AI Benchmarking Hub data47.2%
45Claude Sonnet 4· Epoch AI Benchmarking Hub data46.9%
46Llama 4 Maverick· Epoch AI Benchmarking Hub data46.2%
47GPT OSS 120Bhigh· Epoch AI Benchmarking Hub data44.4%
48GPT-4.1 mini· Epoch AI Benchmarking Hub data44.4%
49GPT-5 nanomedium· Epoch AI Benchmarking Hub data44.4%
50gemini-2.0-pro-exp-02-05· Epoch AI Benchmarking Hub data41.7%

Showing 1–50 of 57 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)