Skip to content

Benchmark

LLM-AggreFact REVEAL

Factuality benchmarks measure how often models get facts right. Scores are percentages. Higher is better.

Models and systems measured
47
Scores recorded
47
Sources
1
IQ weight
No weight published

Score distribution

85.2%
Mean
86.4%
Median
6.4%
Std dev
81.8%
10th percentile
89.1%
90th percentile
50.4%
Min
50.4%91.6%
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
AlignScoregrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data85.3%
Bespoke-Minicheck-7Bgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data88%
Claude-2.1grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data88.2%
Claude-3 Opusgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data83.8%
Claude-3.5 Sonnetgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data89.1%
Clefgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data89.6%
Clef Flashgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data88.1%
d1-ultragrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data91.6%
DAEgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data81.3%
Decider v1.2grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data90.3%
Drex v1.5grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data86.7%
FactCG-DeBERTa-Lgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data88.4%
Gemini-Progrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data85.5%
GPT-3.5-Turbogrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data84.7%
gpt-4-turbo-previewgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data87.8%
gpt-4o-2024-05-13grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data86.5%
gpt-4o-mini-2024-07-18grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data85.8%
Granite Guardian 3.3grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data89.6%
HHEM-2.1-opengrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data86.6%
InternLM2.5-7B-chatgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data77.3%
Jev 1.13.0grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data90.9%
Llama 3.2 1B InstructLlama-3.2-1B-Instruct, grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data50.4%
Llama 3.2 3BLlama-3.2-3B-Instruct, grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data81.9%
Llama 3.3 70B InstructLlama-3.3-70B-Instruct, grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data85.5%
Llama-3-70B-Instructgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data85.9%
Llama-3.1-405B-Instructgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data86.4%
Llama-3.1-70B-Instructgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data86.4%
Llama-3.1-8B-Instructgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data83.5%
MiniCheck-DeBERTa-Lgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data87.3%
MiniCheck-Flan-T5-Lgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data86.2%
MiniCheck-RoBERTa-Lgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data88.8%
Mistral-8x7Bgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data80.8%
Mistral-Largegrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data88.4%
Mistral-Large 2grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data87.7%
Mixtral-8x22Bgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data85.8%
nimble-v4grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data88.6%
nimble-v4-maxgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data89%
PaLM2-Bisongrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data84.2%
Qwen2.5 0.5b InstructQwen2.5-0.5B-Instruct, grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data85.8%
Qwen2.5 72B InstructQwen2.5-72B-Instruct, grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data88.9%
Qwen2.5 7B InstructQwen2.5-7B-Instruct, grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data86.9%
QwQ 32B PreviewQwQ-32B-Preview, grounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data86.2%
SummaC-CVgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data67.7%
SummaC-ZSgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data85.3%
T5-NLI-Mixedgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data87.2%
Tülu-3-70Bgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data82.5%
Tülu-3-8Bgrounded claim checker, REVEAL, renderer-v1· LLM-AggreFact data81.8%

Data credits: LLM-AggreFact, Tang et al. Public numeric grounded checker balanced accuracy; results licence not separately established.