Skip to content

Benchmark

BullshitBench v2 Clear Pushback Rate (all attempts)

Scores are percentages. Higher is better.

Models measured
137
Scores recorded
250
Sources
1
IQ weight
1.07

Score distribution

41.4%
Mean
43%
Median
24.6%
Std dev
9%
10th percentile
74%
90th percentile
2%
Min
2%95%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.07 for chat models, measured over 137 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
101Claude Sonnet 3.7default, thinking· BullshitBench data docs49%
102GPT-6.2 Astralow· BullshitBench data docs49%
103Muse Spark 1.2xhigh· BullshitBench data docs49%
104Nemotron 3 Ultranone· BullshitBench data docs49%
105Gemini 3 Pro Previewlow· BullshitBench data docs48%
106GPT 5.4none· BullshitBench data docs48%
107GPT-5.6 Sollow· BullshitBench data docs48%
108GPT 5.5xhigh· BullshitBench data docs47%
109GPT-5.6 Solmax· BullshitBench data docs47%
110GPT-5.6 Terralow· BullshitBench data docs47%
111Nemotron 3 Super 120B A12Bhigh· BullshitBench data docs47%
112Claude 3.5 Sonnetdefault· BullshitBench data docs46%
113GPT-6.2 Sollow· BullshitBench data docs46%
114Grok 4.3xhigh· BullshitBench data docs46%
115Qwen3 Maxnone· BullshitBench data docs46%
116Claude Fable 5low· BullshitBench data docs45%
117GPT 5.5low· BullshitBench data docs45%
118GPT 5.5none· BullshitBench data docs45%
119GPT-5.1 Chatdefault· BullshitBench data docs45%
120GPT-5.2 Codexlow· BullshitBench data docs45%
121Claude Sonnet 3.7default· BullshitBench data docs44%
122Hy4 Flashhigh· BullshitBench data docs44%
123Claude Opus 4.1none· BullshitBench data docs43%
124Hy4 Flashnone· BullshitBench data docs43%
125Nemotron 3 Super 120B A12Bnone· BullshitBench data docs43%
126Claude Opus 4.1high· BullshitBench data docs42%
127GPT 5.4xhigh· BullshitBench data docs42%
128GPT-6.2 Solmax· BullshitBench data docs42%
129GPT-5.6 Lunamax· BullshitBench data docs41%
130GPT-5.3 Chatdefault· BullshitBench data docs40%
131Nemotron 3 Ultraxhigh· BullshitBench data docs40%
132DeepSeek V4 Flash 0731xhigh· BullshitBench data docs39%
133Gemini 3.6 Flashminimal· BullshitBench data docs39%
134GPT-5 Codexdefault· BullshitBench data docs39%
135GPT-5.2 Codexxhigh· BullshitBench data docs39%
136GPT 5.2none· BullshitBench data docs38%
137MiMo V2.5 Pronone· BullshitBench data docs38%
138Gemini 3.1 Pro Previewlow· BullshitBench data docs37%
139GPT-5.2 Codexhigh· BullshitBench data docs37%
140GPT-5.6 Lunalow· BullshitBench data docs37%
141Gemini 3 Pro Previewhigh· BullshitBench data docs36%
142GPT-5.5 Proxhigh· BullshitBench data docs36%
143DeepSeek V4 Pro 0813max· BullshitBench data docs35%
144Gemini 3.7 Flashminimal· BullshitBench data docs35%
145MiMo V2.5xhigh· BullshitBench data docs35%
146Claude Opus 4default· BullshitBench data docs34%
147GPT-5.5 Promedium· BullshitBench data docs34%
148DeepSeek V4 Pro 0813low· BullshitBench data docs32%
149GPT-5.1 Codexdefault· BullshitBench data docs32%
150GPT-5.4 minihigh· BullshitBench data docs32%

Showing 101–150 of 250 results.

Data credits: BullshitBench by Peter Gostev, MIT (licence)