Skip to content

Benchmark

SimpleQA Verified

Factuality benchmarks measure how often models get facts right. Scores are percentages. Higher is better.

Models and systems measured
100
Scores recorded
115
Sources
3
IQ weight
1.90

Score distribution

4.7%
Mean
0.451%
Median
14%
Std dev
0.174%
10th percentile
18.9%
90th percentile
0.06%
Min
0.06%73.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.90 for chat models, measured over 99 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model or systemScore
51Qwen3.6 Plus· Epoch AI Benchmarking Hub data44.1%
52Inkling FP4· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim43.9%
53Gemini 3.1 Flash LiteGemini 3.1 Flash-Lite Model Card · high thinking · default sampling · single attempt/pass@1· Reported by Google DeepMind· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.43.3%
54GPT-5.6 Terramax· Epoch AI Benchmarking Hub data43.2%
55DeepSeek V4.1 Flashmax· Epoch AI Benchmarking Hub data42.9%
56DeepSeek V4.1 Flash 1017max· Epoch AI Benchmarking Hub data42.5%
57Doubao Seed Evolvinghigh· Epoch AI Benchmarking Hub data41.9%
58GPT-6 Lunamax· Epoch AI Benchmarking Hub data41.4%
59o1high· Epoch AI Benchmarking Hub data41.1%
60GLM 5.3max· Epoch AI Benchmarking Hub data41%
61GPT-5.6 Lunamax· Epoch AI Benchmarking Hub data41%
62Qwen 3 235B A22B Thinking 2507· Epoch AI Benchmarking Hub data40.4%
63Inkling FP4xhigh· Epoch AI Benchmarking Hub data40.3%
64Doubao Seed 2.0 Prohigh· Epoch AI Benchmarking Hub data39.4%
65GPT 5.2xhigh· Epoch AI Benchmarking Hub data37.1%
66Kimi K2.7 Code· Epoch AI Benchmarking Hub data36.5%
67Claude Sonnet 4.6high· Epoch AI Benchmarking Hub data35.5%
68Kimi K2.6· Epoch AI Benchmarking Hub data34.9%
69GPT 5.2high· Epoch AI Benchmarking Hub data34.3%
70Kimi K2.5· Epoch AI Benchmarking Hub data34.3%
71GLM 5.2max· Epoch AI Benchmarking Hub data34.2%
72DeepSeek V4 FlashDeepSeek V4 official model card · Think Max· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.34.1%
73GLM 5.1· Epoch AI Benchmarking Hub data34%
74Claude Sonnet 5max· Epoch AI Benchmarking Hub data33.7%
75DeepSeek V4 Flash 0731max· Epoch AI Benchmarking Hub data33.6%
76Grok 4.3high· Epoch AI Benchmarking Hub data33.2%
77Claude Sonnet 5xhigh· Epoch AI Benchmarking Hub data32.9%
78Claude Sonnet 4.6max· Epoch AI Benchmarking Hub data32.8%
79GPT 5.2low· Epoch AI Benchmarking Hub data32.8%
80GPT 5.2medium· Epoch AI Benchmarking Hub data32.7%
81GLM 4.7· Epoch AI Benchmarking Hub data32.2%
82Doubao Seed 2.0 Litehigh· Epoch AI Benchmarking Hub data31.5%
83GPT 4.1· Epoch AI Benchmarking Hub data31.1%
84Mai Thinking 1· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim31%
85Claude Sonnet 4.559k· Epoch AI Benchmarking Hub data30.7%
86Grok 4.20reasoning, 0309· Epoch AI Benchmarking Hub data30.2%
87Doubao Seed 2.1 Turbohigh· Epoch AI Benchmarking Hub data29.7%
88GPT-5.4 minihigh· Epoch AI Benchmarking Hub data29.4%
89DeepSeek V4 FlashDeepSeek V4 official model card · Think High· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.28.9%
90GPT 4o· Epoch AI Benchmarking Hub data26%
91Qwen3.5 Plus· Epoch AI Benchmarking Hub data25.4%
92Claude Haiku 5.5max· Epoch AI Benchmarking Hub data23.8%
93Claude Sonnet 4.5· Epoch AI Benchmarking Hub data23.7%
94DeepSeek V4 FlashDeepSeek V4 official model card · Non-Think· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.23.1%
95GPT-5 minihigh· Epoch AI Benchmarking Hub data21.6%
96Inkling Small· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim20.6%
97Qwen3.5 Flash· Epoch AI Benchmarking Hub data20.3%
98Mistral Large 4high· Epoch AI Benchmarking Hub data20%
99o4 Minilow· Epoch AI Benchmarking Hub data19.6%
100Inkling Smallxhigh· Epoch AI Benchmarking Hub data19.1%

Showing 51–100 of 114 results.

Not ranked

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
DeepSeek V4.1 Flash BaseEM · 25-shot· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.42.3%

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · Data by LLM Stats