Skip to content

Benchmark

FictionLiveBench 4k

Long context benchmarks measure how well models use information in long inputs. Scores are percentages.

Models measured
54
Scores recorded
56
Sources
1
IQ weight
1.42

Score distribution

69.6%
Mean
72.2%
Median
20%
Std dev
47.2%
10th percentile
94.4%
90th percentile
32%
Min
32%100%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.42 for chat models, measured over 54 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1O3 Promedium· Epoch AI Benchmarking Hub data100%
2OpenAI o3medium· Epoch AI Benchmarking Hub data100%
3Gemini 2.5 Pro· Epoch AI Benchmarking Hub data97.2%
4GPT 5medium· Epoch AI Benchmarking Hub data97.2%
5Grok 4 Fast· Epoch AI Benchmarking Hub data97.2%
6Gemini 2.5 Pro· Epoch AI Benchmarking Hub data94.4%
7Grok 4· Epoch AI Benchmarking Hub data94.4%
8Kimi K2.5· Epoch AI Benchmarking Hub data94.4%
9o1medium· Epoch AI Benchmarking Hub data94.4%
10QwQ 32B· Epoch AI Benchmarking Hub data94.4%
11Claude Sonnet 3.78k· Epoch AI Benchmarking Hub data91.7%
12DeepSeek V3.2 Exphigh· Epoch AI Benchmarking Hub data91.7%
13Gemini 2.5 Flash Preview 05-20 (no think)· Epoch AI Benchmarking Hub data91.7%
14Qwen 3 235B A22B Thinking 2507· Epoch AI Benchmarking Hub data91.7%
15DeepSeek R1· Epoch AI Benchmarking Hub data88.9%
16MiniMax M1 80K· Epoch AI Benchmarking Hub data88.9%
17Qwen3 32B· Epoch AI Benchmarking Hub data86.7%
18GLM 4.5· Epoch AI Benchmarking Hub data83.3%
19GPT-4.5 Preview· Epoch AI Benchmarking Hub data83.3%
20Gemini 2.0 Flash Experimentalthinking· Epoch AI Benchmarking Hub data77.8%
21GPT-5 minimedium· Epoch AI Benchmarking Hub data77.8%
22Grok 3 Minimedium, beta· Epoch AI Benchmarking Hub data77.8%
23o4 Minimedium· Epoch AI Benchmarking Hub data77.8%
24Gemini 2.5 Pro Preview 05-06· Epoch AI Benchmarking Hub data75%
25Qwen3 8B· Epoch AI Benchmarking Hub data74.2%
26Gemini 2.5 Pro Preview 03-25· Epoch AI Benchmarking Hub data72.2%
27Qwen Max· Epoch AI Benchmarking Hub data72.2%
28Qwen3 Max· Epoch AI Benchmarking Hub data72.2%
29Qwen 3 235B A22B· Epoch AI Benchmarking Hub data69%
30Claude Opus 4· Epoch AI Benchmarking Hub data66.7%
31Kimi K2 Instruct· Epoch AI Benchmarking Hub data66.7%
32Qwen3 14B· Epoch AI Benchmarking Hub data64.7%
33GPT 4.1· Epoch AI Benchmarking Hub data63.9%
34Claude Sonnet 3.7· Epoch AI Benchmarking Hub data61.1%
35DeepSeek V3.1· Epoch AI Benchmarking Hub data61.1%
36gemini-2.0-pro-exp-02-05· Epoch AI Benchmarking Hub data61.1%
37Qwen3 Next 80B A3B Instruct· Epoch AI Benchmarking Hub data61.1%
38Claude Sonnet 4· Epoch AI Benchmarking Hub data55.6%
39Gemini 2.5 Flash-Litethinking· Epoch AI Benchmarking Hub data55.6%
40Grok 3beta· Epoch AI Benchmarking Hub data55.6%
41Qwen3 30B A3B· Epoch AI Benchmarking Hub data53.3%
42Qwen3 235B A22B Instruct 2507· Epoch AI Benchmarking Hub data52.9%
43DeepSeek V3· Epoch AI Benchmarking Hub data52.8%
44DeepSeek V3.2 Exp· Epoch AI Benchmarking Hub data50%
45GPT-5 nanomedium· Epoch AI Benchmarking Hub data50%
46Llama 4 Maverick· Epoch AI Benchmarking Hub data48%
47Gemini 2.0 Flash· Epoch AI Benchmarking Hub data47.2%
48gemini-2.5-flash-preview-04-17· Epoch AI Benchmarking Hub data47.2%
49GPT OSS 120B· Epoch AI Benchmarking Hub data47.2%
50o3 Minimedium· Epoch AI Benchmarking Hub data47.2%

Showing 1–50 of 56 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)