Skip to content

Benchmark

Vectara Hallucination Rate

Factuality benchmarks measure how often models get facts right. Scores are percentages. Lower is better.

Models measured
94
Scores recorded
97
Sources
1
IQ weight
0.28

Score distribution

10%
Mean
9.5%
Median
4.6%
Std dev
5.2%
10th percentile
14.7%
90th percentile
3.1%
Min
3.1%24.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.28 for chat models, measured over 94 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51Nemotron 3 Nano 30B A3B· Vectara Hallucination Leaderboard data9.6%
52Jamba Large 1.7· Vectara Hallucination Leaderboard data9.7%
53Claude Haiku 4.5· Vectara Hallucination Leaderboard data9.8%
54Claude Opus 5.5· Vectara Hallucination Leaderboard data9.9%
55GLM 5· Vectara Hallucination Leaderboard data10.1%
56Claude Sonnet 4· Vectara Hallucination Leaderboard data10.3%
57Gemini 3.1 Pro Preview· Vectara Hallucination Leaderboard data10.4%
58GPT-5 nano· Vectara Hallucination Leaderboard data10.5%
59Qwen3.5 35B A3B· Vectara Hallucination Leaderboard data10.5%
60Qwen3.5 Flash· Vectara Hallucination Leaderboard data10.5%
61Claude Sonnet 4.6· Vectara Hallucination Leaderboard data10.6%
62Qwen3.5 Plus· Vectara Hallucination Leaderboard data10.7%
63Kimi K2.6· Vectara Hallucination Leaderboard data10.8%
64Claude Opus 4.5· Vectara Hallucination Leaderboard data10.9%
65GPT 5.1low· Vectara Hallucination Leaderboard data10.9%
66Qwen3.5 122B-A10B· Vectara Hallucination Leaderboard data11.2%
67DeepSeek R1· Vectara Hallucination Leaderboard data11.3%
68GLM 4.7· Vectara Hallucination Leaderboard data11.7%
69Claude Opus 4.1· Vectara Hallucination Leaderboard data11.8%
70MiniMax M2.1· Vectara Hallucination Leaderboard data11.8%
71Claude Opus 4· Vectara Hallucination Leaderboard data12%
72Claude Opus 4.7· Vectara Hallucination Leaderboard data12%
73Claude Sonnet 4.5· Vectara Hallucination Leaderboard data12%
74Gemini 4 Argon· Vectara Hallucination Leaderboard data12.1%
75GPT 5.1high· Vectara Hallucination Leaderboard data12.1%
76Qwen3.5 27B· Vectara Hallucination Leaderboard data12.1%
77Claude Opus 4.6· Vectara Hallucination Leaderboard data12.2%
78Mercury 2· Vectara Hallucination Leaderboard data12.3%
79GPT-5.6 Sol· Vectara Hallucination Leaderboard data12.4%
80GPT-5 mini· Vectara Hallucination Leaderboard data12.9%
81MiniMax M2.7· Vectara Hallucination Leaderboard data12.9%
82Gemini 3 Flash· Vectara Hallucination Leaderboard data13.5%
83Gemini 3 Pro Preview· Vectara Hallucination Leaderboard data13.6%
84GPT OSS 120B· Vectara Hallucination Leaderboard data14.2%
85Kimi K2.5· Vectara Hallucination Leaderboard data14.2%
86GPT 5minimal· Vectara Hallucination Leaderboard data14.7%
87jamba-mini-1.7· Vectara Hallucination Leaderboard data14.7%
88GPT 5high· Vectara Hallucination Leaderboard data15.1%
89Kimi K2 Instruct· Vectara Hallucination Leaderboard data17.9%
90o4 Mini· Vectara Hallucination Leaderboard data18.6%
91o4 Minilow· Vectara Hallucination Leaderboard data18.6%
92Ministral 3 14B· Vectara Hallucination Leaderboard data19.4%
93Ministral 3 8B· Vectara Hallucination Leaderboard data21.7%
94Mistral Medium 3.1· Vectara Hallucination Leaderboard data22.7%
95O3 Pro· Vectara Hallucination Leaderboard data23.3%
96Phi 4 Mini· Vectara Hallucination Leaderboard data23.5%
97Ministral 3 3B· Vectara Hallucination Leaderboard data24.2%

Showing 51–97 of 97 results.

Data credits: Vectara Hallucination Leaderboard, Apache-2.0 (licence)