Skip to content

Benchmark

Vectara Factual Consistency

Factuality benchmarks measure how often models get facts right. Scores are percentages.

Models measured
94
Scores recorded
97
Sources
1
IQ weight
0.28

Score distribution

90%
Mean
90.5%
Median
4.6%
Std dev
85.3%
10th percentile
94.8%
90th percentile
75.8%
Min
75.8%96.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.28 for chat models, measured over 94 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51Nemotron 3 Nano 30B A3B· Vectara Hallucination Leaderboard data90.4%
52Jamba Large 1.7· Vectara Hallucination Leaderboard data90.3%
53Claude Haiku 4.5· Vectara Hallucination Leaderboard data90.2%
54Claude Opus 5.5· Vectara Hallucination Leaderboard data90.1%
55GLM 5· Vectara Hallucination Leaderboard data89.9%
56Claude Sonnet 4· Vectara Hallucination Leaderboard data89.7%
57Gemini 3.1 Pro Preview· Vectara Hallucination Leaderboard data89.6%
58GPT-5 nano· Vectara Hallucination Leaderboard data89.5%
59Qwen3.5 35B A3B· Vectara Hallucination Leaderboard data89.5%
60Qwen3.5 Flash· Vectara Hallucination Leaderboard data89.5%
61Claude Sonnet 4.6· Vectara Hallucination Leaderboard data89.4%
62Qwen3.5 Plus· Vectara Hallucination Leaderboard data89.3%
63Kimi K2.6· Vectara Hallucination Leaderboard data89.2%
64Claude Opus 4.5· Vectara Hallucination Leaderboard data89.1%
65GPT 5.1low· Vectara Hallucination Leaderboard data89.1%
66Qwen3.5 122B-A10B· Vectara Hallucination Leaderboard data88.8%
67DeepSeek R1· Vectara Hallucination Leaderboard data88.7%
68GLM 4.7· Vectara Hallucination Leaderboard data88.3%
69Claude Opus 4.1· Vectara Hallucination Leaderboard data88.2%
70MiniMax M2.1· Vectara Hallucination Leaderboard data88.2%
71Claude Opus 4· Vectara Hallucination Leaderboard data88%
72Claude Opus 4.7· Vectara Hallucination Leaderboard data88%
73Claude Sonnet 4.5· Vectara Hallucination Leaderboard data88%
74Gemini 4 Argon· Vectara Hallucination Leaderboard data87.9%
75GPT 5.1high· Vectara Hallucination Leaderboard data87.9%
76Qwen3.5 27B· Vectara Hallucination Leaderboard data87.9%
77Claude Opus 4.6· Vectara Hallucination Leaderboard data87.8%
78Mercury 2· Vectara Hallucination Leaderboard data87.7%
79GPT-5.6 Sol· Vectara Hallucination Leaderboard data87.6%
80GPT-5 mini· Vectara Hallucination Leaderboard data87.1%
81MiniMax M2.7· Vectara Hallucination Leaderboard data87.1%
82Gemini 3 Flash· Vectara Hallucination Leaderboard data86.5%
83Gemini 3 Pro Preview· Vectara Hallucination Leaderboard data86.4%
84GPT OSS 120B· Vectara Hallucination Leaderboard data85.8%
85Kimi K2.5· Vectara Hallucination Leaderboard data85.8%
86GPT 5minimal· Vectara Hallucination Leaderboard data85.3%
87jamba-mini-1.7· Vectara Hallucination Leaderboard data85.3%
88GPT 5high· Vectara Hallucination Leaderboard data84.9%
89Kimi K2 Instruct· Vectara Hallucination Leaderboard data82.1%
90o4 Mini· Vectara Hallucination Leaderboard data81.4%
91o4 Minilow· Vectara Hallucination Leaderboard data81.4%
92Ministral 3 14B· Vectara Hallucination Leaderboard data80.6%
93Ministral 3 8B· Vectara Hallucination Leaderboard data78.3%
94Mistral Medium 3.1· Vectara Hallucination Leaderboard data77.3%
95O3 Pro· Vectara Hallucination Leaderboard data76.7%
96Phi 4 Mini· Vectara Hallucination Leaderboard data76.5%
97Ministral 3 3B· Vectara Hallucination Leaderboard data75.8%

Showing 51–97 of 97 results.

Data credits: Vectara Hallucination Leaderboard, Apache-2.0 (licence)