Skip to content

Benchmark

BullshitBench v2

Higher scores are better.

Models measured
137
Scores recorded
250
Sources
1
IQ weight
1.14

Score distribution

1
Mean
1.1
Median
0.489
Std dev
0.345
10th percentile
1.6
90th percentile
0.147
Min
0.1471.9
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.14 for chat models, measured over 137 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.8none· BullshitBench data docs1.9
2Claude Opus 4.8xhigh· BullshitBench data docs1.9
3Qwen3.8 Maxminimal· BullshitBench data docs1.9
4Claude Sonnet 4.6high· BullshitBench data docs1.9
5Qwen4 Maxmax· BullshitBench data docs1.9
6Claude Sonnet 4.6none· BullshitBench data docs1.9
7Claude Opus 4.5high· BullshitBench data docs1.8
8Gemini 4 Hydrogenxhigh· BullshitBench data docs1.8
9Claude Opus 4.6high· BullshitBench data docs1.8
10Claude Fable 5.1low· BullshitBench data docs1.8
11Claude Opus 4.6none· BullshitBench data docs1.8
12Claude Opus 4.7none· BullshitBench data docs1.8
13Gemini 4 Neonxhigh· BullshitBench data docs1.8
14Claude Fable 5xhigh· BullshitBench data docs1.8
15Mistral Large 4none· BullshitBench data docs1.7
16Claude Opus 5xhigh· BullshitBench data docs1.7
17Claude Sonnet 5low· BullshitBench data docs1.7
18Claude Sonnet 5max· BullshitBench data docs1.7
19Claude Opus 5low· BullshitBench data docs1.7
20Qwen3.5 397B A17Bhigh· BullshitBench data docs1.7
21Claude Sonnet 4.5high· BullshitBench data docs1.7
22Claude Opus 4.5none· BullshitBench data docs1.7
23Qwen4 Maxnone· BullshitBench data docs1.7
24Kimi K3xhigh· BullshitBench data docs1.7
25Claude Fable 5.1max· BullshitBench data docs1.7
26Kimi K3minimal· BullshitBench data docs1.6
27Claude Fable 5low· BullshitBench data docs1.6
28Claude Haiku 4.5high· BullshitBench data docs1.6
29Claude Opus 5.5low· BullshitBench data docs1.6
30Qwen 3.8 27Bnone· BullshitBench data docs1.6
31Claude Opus 5.5max· BullshitBench data docs1.6
32Claude Haiku 4.5none· BullshitBench data docs1.6
33Claude Opus 4.7max· BullshitBench data docs1.6
34Grok 4.7xhigh· BullshitBench data docs1.6
35GLM 5.3minimal· BullshitBench data docs1.6
36GPT-6 Astramax· BullshitBench data docs1.6
37Grok 4.7low· BullshitBench data docs1.6
38Mistral Large 4high· BullshitBench data docs1.6
39Claude Sonnet 4.5none· BullshitBench data docs1.6
40Qwen3.7 Maxnone· BullshitBench data docs1.6
41GPT-6.1 Solmax· BullshitBench data docs1.6
42Claude Fable 5.2max· BullshitBench data docs1.6
43Claude Fable 5.2low· BullshitBench data docs1.6
44Qwen3.6 Plusnone· BullshitBench data docs1.6
45GLM 5.3max· BullshitBench data docs1.5
46GPT-6 Astralow· BullshitBench data docs1.5
47Qwen3.8 Flashnone· BullshitBench data docs1.5
48Gemini 4 Neonlow· BullshitBench data docs1.5
49GPT-6.1 Sollow· BullshitBench data docs1.5
50Claude Sonnet 5.5low· BullshitBench data docs1.5

Showing 1–50 of 250 results.

Data credits: BullshitBench by Peter Gostev, MIT (licence)