Skip to content

Benchmark

HalluHard Hallucination Rate (legal cases)

Factuality benchmarks measure how often models get facts right. Scores are percentages. Lower is better.

Models measured
30
Scores recorded
38
Sources
1
IQ weight
2.14

Score distribution

49.1%
Mean
50.1%
Median
15.5%
Std dev
29.9%
10th percentile
67.7%
90th percentile
10.4%
Min
10.4%77.3%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.14 for chat models, measured over 30 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-6 Astrawebsearch· HalluHard Leaderboard data docs10.4%
2GPT 5.4websearch, thinking· HalluHard Leaderboard data docs16.5%
3GPT-5.3websearch· HalluHard Leaderboard data docs26.3%
4GLM 5websearch, thinking· HalluHard Leaderboard data docs26.8%
5GPT 5.4thinking· HalluHard Leaderboard data docs29.9%
6GPT-6 Astra· HalluHard Leaderboard data docs30.6%
7GPT 5.2thinking· HalluHard Leaderboard data docs33.5%
8GPT-5.3· HalluHard Leaderboard data docs34.6%
9Claude Fable 5· HalluHard Leaderboard data docs38.1%
10Claude Fable 5websearch· HalluHard Leaderboard data docs39.6%
11Kimi K2.5websearch· HalluHard Leaderboard data docs42.5%
12Gemini 3.1 Pro Preview· HalluHard Leaderboard data docs43.3%
13Claude Opus 4.5· HalluHard Leaderboard data docs44.8%
14Claude Opus 4.7· HalluHard Leaderboard data docs45.4%
15Gemini 3 Pro Preview· HalluHard Leaderboard data docs46%
16GPT 5.2· HalluHard Leaderboard data docs46.4%
17GPT 5thinking· HalluHard Leaderboard data docs46.9%
18Claude Sonnet 4.6· HalluHard Leaderboard data docs49.5%
19Claude Opus 4.6· HalluHard Leaderboard data docs49.7%
20Claude Sonnet 5· HalluHard Leaderboard data docs50.1%
21Claude Sonnet 4.5· HalluHard Leaderboard data docs51.8%
22Gemini 3 Flash· HalluHard Leaderboard data docs52%
23Kimi K2.6thinking· HalluHard Leaderboard data docs52.5%
24GPT 5· HalluHard Leaderboard data docs52.8%
25DeepSeek Reasoner· HalluHard Leaderboard data docs55.7%
26DeepSeek Chat· HalluHard Leaderboard data docs56.4%
27GLM 5thinking· HalluHard Leaderboard data docs57.3%
28DeepSeek V4 Pro· HalluHard Leaderboard data docs57.9%
29Grok 4thinking· HalluHard Leaderboard data docs60.1%
30GPT-5 mini· HalluHard Leaderboard data docs63.5%
31Kimi K2.5thinking· HalluHard Leaderboard data docs65.2%
32GLM 5.2· HalluHard Leaderboard data docs65.7%
33Claude Haiku 4.5· HalluHard Leaderboard data docs67.1%
34GLM 4.7thinking· HalluHard Leaderboard data docs67.7%
35Nemotron 3 Ultra· HalluHard Leaderboard data docs69.7%
36Kimi K2 Thinking· HalluHard Leaderboard data docs70%
37Grok 4.1 Fastthinking· HalluHard Leaderboard data docs73.4%
38GPT-5 nano· HalluHard Leaderboard data docs77.3%

Data credits: HalluHard: Fan, Delsad, Flammarion and Andriushchenko (EPFL), arXiv:2602.01031