Benchmark
LLM-AggreFact all-11 mean
Factuality benchmarks measure how often models get facts right. Scores are percentages. Higher is better.
- Models and systems measured
- 47
- Scores recorded
- 47
- Sources
- 1
- IQ weight
- No weight published
Score distribution
- 71.7%
- Mean
- 73.7%
- Median
- 6.2%
- Std dev
- 63.6%
- 10th percentile
- 77.8%
- 90th percentile
- 50.3%
- Min
Results
The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.
| Model or system | Score |
|---|---|
| AlignScoregrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 70.5% |
| Bespoke-Minicheck-7Bgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 77.4% |
| Claude-2.1grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 71.1% |
| Claude-3 Opusgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 74.8% |
| Claude-3.5 Sonnetgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 77.2% |
| Clefgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 79% |
| Clef Flashgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 76.6% |
| d1-ultragrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 78.5% |
| DAEgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 65% |
| Decider v1.2grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 78.7% |
| Drex v1.5grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 76.5% |
| FactCG-DeBERTa-Lgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 75.6% |
| Gemini-Progrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 65.4% |
| GPT-3.5-Turbogrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 70.1% |
| gpt-4-turbo-previewgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 76.2% |
| gpt-4o-2024-05-13grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 75.9% |
| gpt-4o-mini-2024-07-18grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 74% |
| Granite Guardian 3.3grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 76.5% |
| HHEM-2.1-opengrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 71.8% |
| InternLM2.5-7B-chatgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 63.6% |
| Jev 1.13.0grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 78.7% |
| Llama 3.2 1B InstructLlama-3.2-1B-Instruct, grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 50.3% |
| Llama 3.2 3BLlama-3.2-3B-Instruct, grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 60% |
| Llama 3.3 70B InstructLlama-3.3-70B-Instruct, grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 74.5% |
| Llama-3-70B-Instructgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 73.7% |
| Llama-3.1-405B-Instructgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 74.4% |
| Llama-3.1-70B-Instructgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 75.1% |
| Llama-3.1-8B-Instructgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 70.3% |
| MiniCheck-DeBERTa-Lgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 73.1% |
| MiniCheck-Flan-T5-Lgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 75% |
| MiniCheck-RoBERTa-Lgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 73.5% |
| Mistral-8x7Bgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 67.4% |
| Mistral-Largegrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 74.2% |
| Mistral-Large 2grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 76.5% |
| Mixtral-8x22Bgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 71.5% |
| nimble-v4grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 77.8% |
| nimble-v4-maxgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 79.5% |
| PaLM2-Bisongrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 66.1% |
| Qwen2.5 0.5b InstructQwen2.5-0.5B-Instruct, grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 64.2% |
| Qwen2.5 72B InstructQwen2.5-72B-Instruct, grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 75.6% |
| Qwen2.5 7B InstructQwen2.5-7B-Instruct, grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 72.8% |
| QwQ 32B PreviewQwQ-32B-Preview, grounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 71.8% |
| SummaC-CVgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 61% |
| SummaC-ZSgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 67.7% |
| T5-NLI-Mixedgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 61.1% |
| Tülu-3-70Bgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 67.6% |
| Tülu-3-8Bgrounded claim checker, all-11 mean, renderer-v1· LLM-AggreFact data | 63.2% |
Data credits: LLM-AggreFact, Tang et al. Public numeric grounded checker balanced accuracy; results licence not separately established.