Skip to content

Benchmark

Vectara Hallucination Rate

Factuality benchmarks measure how often models get facts right. Scores are percentages. Lower is better.

Models measured
94
Scores recorded
97
Sources
1
IQ weight
0.28

Score distribution

10%
Mean
9.5%
Median
4.6%
Std dev
5.2%
10th percentile
14.7%
90th percentile
3.1%
Min
3.1%24.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.28 for chat models, measured over 94 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 5.4 Nano· Vectara Hallucination Leaderboard data3.1%
2Gemini 2.5 Flash-Lite· Vectara Hallucination Leaderboard data3.3%
3Phi 4· Vectara Hallucination Leaderboard data3.7%
4Llama 3.3 70B Instruct Turbo· Vectara Hallucination Leaderboard data4.1%
5DeepSeek V4.1 Flash· Vectara Hallucination Leaderboard data4.2%
6Gemma 3 12B IT· Vectara Hallucination Leaderboard data4.4%
7Mistral Large· Vectara Hallucination Leaderboard data4.5%
8Qwen3 8B· Vectara Hallucination Leaderboard data4.8%
9Mistral Small· Vectara Hallucination Leaderboard data5.1%
10Nova Pro 1.0· Vectara Hallucination Leaderboard data5.1%
11Gemma 4 26B A4B· Vectara Hallucination Leaderboard data5.2%
12DeepSeek V3.2 Exp· Vectara Hallucination Leaderboard data5.3%
13jamba-mini-2· Vectara Hallucination Leaderboard data5.3%
14Qwen3 14B· Vectara Hallucination Leaderboard data5.4%
15DeepSeek V3.1· Vectara Hallucination Leaderboard data5.5%
16GPT-5.4 mini· Vectara Hallucination Leaderboard data5.5%
17Nova Micro 1.0· Vectara Hallucination Leaderboard data5.5%
18GPT 4.1· Vectara Hallucination Leaderboard data5.6%
19Qwen3 4B· Vectara Hallucination Leaderboard data5.7%
20Qwen3 32B· Vectara Hallucination Leaderboard data5.9%
21DeepSeek V3· Vectara Hallucination Leaderboard data6.1%
22Nova Lite 1.0· Vectara Hallucination Leaderboard data6.1%
23DeepSeek V3.2· Vectara Hallucination Leaderboard data6.3%
24Gemma 3 4B IT· Vectara Hallucination Leaderboard data6.4%
25GPT-6 Sol· Vectara Hallucination Leaderboard data6.5%
26Command R Plus 08 2024· Vectara Hallucination Leaderboard data6.9%
27Trinity Large Preview· Vectara Hallucination Leaderboard data6.9%
28Gemini 2.5 Pro· Vectara Hallucination Leaderboard data7%
29GPT 5.4· Vectara Hallucination Leaderboard data7%
30Ministral 3B· Vectara Hallucination Leaderboard data7.3%
31Gemma 3 27B IT· Vectara Hallucination Leaderboard data7.4%
32Gemma 4 31B IT· Vectara Hallucination Leaderboard data7.4%
33Ministral 8B· Vectara Hallucination Leaderboard data7.4%
34Llama 4 Scout· Vectara Hallucination Leaderboard data7.7%
35Gemini 2.5 Flash· Vectara Hallucination Leaderboard data7.8%
36Gemini 3.1 Flash Lite· Vectara Hallucination Leaderboard data8.2%
37Llama 4 Maverick· Vectara Hallucination Leaderboard data8.2%
38GPT-5.4 Pro· Vectara Hallucination Leaderboard data8.3%
39GPT 5.2· Vectara Hallucination Leaderboard data8.4%
40DeepSeek V4 Pro· Vectara Hallucination Leaderboard data8.6%
41GPT-6 Astra· Vectara Hallucination Leaderboard data8.7%
42MiniMax M2.5· Vectara Hallucination Leaderboard data9.1%
43Command A 03 2025· Vectara Hallucination Leaderboard data9.3%
44GLM 4.5 Air· Vectara Hallucination Leaderboard data9.3%
45GLM 4.7 Flash· Vectara Hallucination Leaderboard data9.3%
46GPT 5.5· Vectara Hallucination Leaderboard data9.3%
47Qwen 3 235B A22B· Vectara Hallucination Leaderboard data9.3%
48Qwen3 Next 80B A3B· Vectara Hallucination Leaderboard data9.3%
49GLM 4.6· Vectara Hallucination Leaderboard data9.5%
50GPT 4o· Vectara Hallucination Leaderboard data9.6%

Showing 1–50 of 97 results.

Data credits: Vectara Hallucination Leaderboard, Apache-2.0 (licence)