Skip to content

Benchmark

HalluHard Hallucination Rate (medical guidelines)

Factuality benchmarks measure how often models get facts right. Scores are percentages. Lower is better.

Models measured
30
Scores recorded
39
Sources
1
IQ weight
2.75

Score distribution

79.3%
Mean
85.9%
Median
17.2%
Std dev
51.9%
10th percentile
95%
90th percentile
29.2%
Min
29.2%96.1%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.75 for chat models, measured over 30 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.5websearch· HalluHard Leaderboard data docs29.2%
2GLM 5websearch, thinking· HalluHard Leaderboard data docs35%
3Claude Fable 5websearch· HalluHard Leaderboard data docs44.8%
4GPT-6 Astrawebsearch· HalluHard Leaderboard data docs49.1%
5GPT 5.4websearch, thinking· HalluHard Leaderboard data docs51.9%
6GPT-5.3websearch· HalluHard Leaderboard data docs53.8%
7Kimi K2.5websearch· HalluHard Leaderboard data docs67.5%
8GPT-6 Astra· HalluHard Leaderboard data docs70.1%
9GPT-5.3· HalluHard Leaderboard data docs71.1%
10GPT 5.2· HalluHard Leaderboard data docs72.7%
11GPT 5.4thinking· HalluHard Leaderboard data docs72.9%
12GPT 5.2thinking· HalluHard Leaderboard data docs74%
13Claude Fable 5· HalluHard Leaderboard data docs78.5%
14Gemini 3.1 Pro Preview· HalluHard Leaderboard data docs80%
15Claude Opus 4.7· HalluHard Leaderboard data docs82.2%
16Claude Sonnet 5· HalluHard Leaderboard data docs82.8%
17GPT 5thinking· HalluHard Leaderboard data docs83.8%
18Kimi K2.6thinking· HalluHard Leaderboard data docs84.1%
19Claude Opus 4.5· HalluHard Leaderboard data docs85.6%
20Gemini 3 Pro Preview· HalluHard Leaderboard data docs85.9%
21Claude Sonnet 4.5· HalluHard Leaderboard data docs86.1%
22GLM 5thinking· HalluHard Leaderboard data docs87.3%
23DeepSeek Reasoner· HalluHard Leaderboard data docs88.1%
24DeepSeek Chat· HalluHard Leaderboard data docs89%
25Gemini 3 Flash· HalluHard Leaderboard data docs89%
26GLM 5.2· HalluHard Leaderboard data docs89.2%
27Claude Opus 4.6· HalluHard Leaderboard data docs89.3%
28Claude Sonnet 4.6· HalluHard Leaderboard data docs89.7%
29DeepSeek V4 Pro· HalluHard Leaderboard data docs89.9%
30GLM 4.7thinking· HalluHard Leaderboard data docs90.9%
31GPT-5 mini· HalluHard Leaderboard data docs92.7%
32Grok 4thinking· HalluHard Leaderboard data docs92.7%
33GPT 5· HalluHard Leaderboard data docs92.8%
34Kimi K2.5thinking· HalluHard Leaderboard data docs93.2%
35Kimi K2 Thinking· HalluHard Leaderboard data docs95%
36GPT-5 nano· HalluHard Leaderboard data docs95.3%
37Nemotron 3 Ultra· HalluHard Leaderboard data docs95.4%
38Claude Haiku 4.5· HalluHard Leaderboard data docs95.7%
39Grok 4.1 Fastthinking· HalluHard Leaderboard data docs96.1%

Data credits: HalluHard: Fan, Delsad, Flammarion and Andriushchenko (EPFL), arXiv:2602.01031