Skip to content

Benchmark

BoolQ

Scores are percentages.

Models measured
29
Scores recorded
29
Sources
2
IQ weight
1.74

Score distribution

76.4%
Mean
82.5%
Median
12.4%
Std dev
58.1%
10th percentile
87.4%
90th percentile
45.1%
Min
45.1%88.7%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.74 for chat models, measured over 29 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-4o mini· Epoch AI Benchmarking Hub data88.7%
2Text Davinci 003· Epoch AI Benchmarking Hub data88.1%
3Text Davinci 002· Epoch AI Benchmarking Hub data87.7%
4Mistral 7B v0.1· Epoch AI Benchmarking Hub data87.4%
5GPT-3.5 Turbo· Epoch AI Benchmarking Hub data87%
6gemini-1.5-flash-001· Epoch AI Benchmarking Hub data85.8%
7gemma-2-9b· Epoch AI Benchmarking Hub data85.7%
8Gemma 2 27B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim84.8%
9phi-3.5-moe-instruct· Epoch AI Benchmarking Hub data
Values from 2 sources
84.6%
10Gemma 2 9B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim84.2%
11gemma-7b· Epoch AI Benchmarking Hub data83.2%
12Mistral 7B Instruct v0.2· Epoch AI Benchmarking Hub data83.2%
13Llama 3.1 8B Instruct· Epoch AI Benchmarking Hub data82.8%
14mistral-nemo-base-2407· Epoch AI Benchmarking Hub data82.5%
15Llama 2 13b· Epoch AI Benchmarking Hub data82.4%
16Phi 4 Mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim81.2%
17ChatGLM2 6B· Epoch AI Benchmarking Hub data79%
18LLaMA 13B· Epoch AI Benchmarking Hub data78.7%
19Llama 2 7b· Epoch AI Benchmarking Hub data77.9%
20text-davinci-001· Epoch AI Benchmarking Hub data77.5%
21davinci· Epoch AI Benchmarking Hub data72.2%
22gemma-2b· Epoch AI Benchmarking Hub data69.4%
23curie· Epoch AI Benchmarking Hub data65.6%
24Text Curie 001· Epoch AI Benchmarking Hub data62%
25StableLM Tuned Alpha 7B· Epoch AI Benchmarking Hub data59%
26ada· Epoch AI Benchmarking Hub data58.1%
27babbage· Epoch AI Benchmarking Hub data57.4%
28Text Ada 001· Epoch AI Benchmarking Hub data46.4%
29Text Babbage 001· Epoch AI Benchmarking Hub data45.1%

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · Data by LLM Stats