Skip to content

Benchmark

BullshitBench v2 Clear Pushback Rate (all attempts)

Scores are percentages. Higher is better.

Models measured
137
Scores recorded
250
Sources
1
IQ weight
1.07

Score distribution

41.4%
Mean
43%
Median
24.6%
Std dev
9%
10th percentile
74%
90th percentile
2%
Min
2%95%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.07 for chat models, measured over 137 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.8none· BullshitBench data docs95%
2Qwen3.8 Maxminimal· BullshitBench data docs95%
3Claude Opus 4.8xhigh· BullshitBench data docs94%
4Claude Sonnet 4.6high· BullshitBench data docs91%
5Claude Opus 4.5high· BullshitBench data docs90%
6Gemini 4 Hydrogenxhigh· BullshitBench data docs90%
7Qwen4 Maxmax· BullshitBench data docs90%
8Claude Sonnet 4.6none· BullshitBench data docs89%
9Claude Opus 4.6high· BullshitBench data docs87%
10Gemini 4 Neonxhigh· BullshitBench data docs84%
11Claude Opus 4.6none· BullshitBench data docs83%
12Claude Opus 4.7none· BullshitBench data docs83%
13Claude Sonnet 5low· BullshitBench data docs80%
14Mistral Large 4none· BullshitBench data docs80%
15Claude Opus 4.5none· BullshitBench data docs79%
16Claude Sonnet 4.5high· BullshitBench data docs79%
17Qwen 3.8 27Bnone· BullshitBench data docs79%
18Qwen4 Maxnone· BullshitBench data docs79%
19Claude Sonnet 5max· BullshitBench data docs78%
20Qwen3.5 397B A17Bhigh· BullshitBench data docs78%
21Claude Fable 5.1low· BullshitBench data docs77%
22Claude Haiku 4.5high· BullshitBench data docs77%
23Grok 4.7low· BullshitBench data docs75%
24Claude Opus 4.7max· BullshitBench data docs74%
25Claude Sonnet 4.5none· BullshitBench data docs74%
26Kimi K3xhigh· BullshitBench data docs74%
27Claude Opus 5low· BullshitBench data docs73%
28Gemini 4 Carbonhigh· BullshitBench data docs72%
29GLM 5.3minimal· BullshitBench data docs72%
30Grok 4.7xhigh· BullshitBench data docs72%
31Kimi K3minimal· BullshitBench data docs72%
32Qwen3.6 Plusnone· BullshitBench data docs72%
33Qwen3.7 Maxnone· BullshitBench data docs72%
34Claude Haiku 4.5none· BullshitBench data docs71%
35Claude Opus 5xhigh· BullshitBench data docs70%
36Qwen3.8 Flashnone· BullshitBench data docs70%
37Qwen3.8 Omni Flashnone· BullshitBench data docs70%
38GPT-6 Astramax· BullshitBench data docs69%
39Qwen3.5 397B A17Bnone· BullshitBench data docs69%
40Mistral Large 4high· BullshitBench data docs68%
41Gemini 4 Neonlow· BullshitBench data docs67%
42Grok 4.20 Multi-Agentlow· BullshitBench data docs67%
43Claude Fable 5.2max· BullshitBench data docs66%
44Gemini 3.5 Flash Liteminimal· BullshitBench data docs66%
45Gemini 4 Argonlow· BullshitBench data docs66%
46GLM 5.3max· BullshitBench data docs66%
47Grok 4.6minimal· BullshitBench data docs66%
48Gemini 4 Argonhigh· BullshitBench data docs65%
49GPT-6.1 Sollow· BullshitBench data docs65%
50Kimi K2.6none· BullshitBench data docs65%

Showing 1–50 of 250 results.

Data credits: BullshitBench by Peter Gostev, MIT (licence)