Skip to content

Benchmark

Vectara Factual Consistency

Factuality benchmarks measure how often models get facts right. Scores are percentages.

Models measured
94
Scores recorded
97
Sources
1
IQ weight
0.28

Score distribution

90%
Mean
90.5%
Median
4.6%
Std dev
85.3%
10th percentile
94.8%
90th percentile
75.8%
Min
75.8%96.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.28 for chat models, measured over 94 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 5.4 Nano· Vectara Hallucination Leaderboard data96.9%
2Gemini 2.5 Flash-Lite· Vectara Hallucination Leaderboard data96.7%
3Phi 4· Vectara Hallucination Leaderboard data96.3%
4Llama 3.3 70B Instruct Turbo· Vectara Hallucination Leaderboard data95.9%
5DeepSeek V4.1 Flash· Vectara Hallucination Leaderboard data95.8%
6Gemma 3 12B IT· Vectara Hallucination Leaderboard data95.6%
7Mistral Large· Vectara Hallucination Leaderboard data95.5%
8Qwen3 8B· Vectara Hallucination Leaderboard data95.2%
9Mistral Small· Vectara Hallucination Leaderboard data94.9%
10Nova Pro 1.0· Vectara Hallucination Leaderboard data94.9%
11Gemma 4 26B A4B· Vectara Hallucination Leaderboard data94.8%
12DeepSeek V3.2 Exp· Vectara Hallucination Leaderboard data94.7%
13jamba-mini-2· Vectara Hallucination Leaderboard data94.7%
14Qwen3 14B· Vectara Hallucination Leaderboard data94.6%
15DeepSeek V3.1· Vectara Hallucination Leaderboard data94.5%
16GPT-5.4 mini· Vectara Hallucination Leaderboard data94.5%
17Nova Micro 1.0· Vectara Hallucination Leaderboard data94.5%
18GPT 4.1· Vectara Hallucination Leaderboard data94.4%
19Qwen3 4B· Vectara Hallucination Leaderboard data94.3%
20Qwen3 32B· Vectara Hallucination Leaderboard data94.1%
21DeepSeek V3· Vectara Hallucination Leaderboard data93.9%
22Nova Lite 1.0· Vectara Hallucination Leaderboard data93.9%
23DeepSeek V3.2· Vectara Hallucination Leaderboard data93.7%
24Gemma 3 4B IT· Vectara Hallucination Leaderboard data93.6%
25GPT-6 Sol· Vectara Hallucination Leaderboard data93.5%
26Command R Plus 08 2024· Vectara Hallucination Leaderboard data93.1%
27Trinity Large Preview· Vectara Hallucination Leaderboard data93.1%
28Gemini 2.5 Pro· Vectara Hallucination Leaderboard data93%
29GPT 5.4· Vectara Hallucination Leaderboard data93%
30Ministral 3B· Vectara Hallucination Leaderboard data92.7%
31Gemma 3 27B IT· Vectara Hallucination Leaderboard data92.6%
32Gemma 4 31B IT· Vectara Hallucination Leaderboard data92.6%
33Ministral 8B· Vectara Hallucination Leaderboard data92.6%
34Llama 4 Scout· Vectara Hallucination Leaderboard data92.3%
35Gemini 2.5 Flash· Vectara Hallucination Leaderboard data92.2%
36Gemini 3.1 Flash Lite· Vectara Hallucination Leaderboard data91.8%
37Llama 4 Maverick· Vectara Hallucination Leaderboard data91.8%
38GPT-5.4 Pro· Vectara Hallucination Leaderboard data91.7%
39GPT 5.2· Vectara Hallucination Leaderboard data91.6%
40DeepSeek V4 Pro· Vectara Hallucination Leaderboard data91.4%
41GPT-6 Astra· Vectara Hallucination Leaderboard data91.3%
42MiniMax M2.5· Vectara Hallucination Leaderboard data90.9%
43Command A 03 2025· Vectara Hallucination Leaderboard data90.7%
44GLM 4.5 Air· Vectara Hallucination Leaderboard data90.7%
45GLM 4.7 Flash· Vectara Hallucination Leaderboard data90.7%
46GPT 5.5· Vectara Hallucination Leaderboard data90.7%
47Qwen 3 235B A22B· Vectara Hallucination Leaderboard data90.7%
48Qwen3 Next 80B A3B· Vectara Hallucination Leaderboard data90.7%
49GLM 4.6· Vectara Hallucination Leaderboard data90.5%
50GPT 4o· Vectara Hallucination Leaderboard data90.4%

Showing 1–50 of 97 results.

Data credits: Vectara Hallucination Leaderboard, Apache-2.0 (licence)