Skip to content

Benchmark

BullshitBench v2 Clear Pushback Rate (all attempts)

Scores are percentages. Higher is better.

Models measured
137
Scores recorded
250
Sources
1
IQ weight
1.07

Score distribution

41.4%
Mean
43%
Median
24.6%
Std dev
9%
10th percentile
74%
90th percentile
2%
Min
2%95%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.07 for chat models, measured over 137 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
201DeepSeek V4 Pronone· BullshitBench data docs14%
202DeepSeek V4 Proxhigh· BullshitBench data docs14%
203GPT 4.1default· BullshitBench data docs14%
204GPT 5.4 Nanohigh· BullshitBench data docs14%
205GPT 5.4 Nanonone· BullshitBench data docs14%
206Llama 3.1 8B Instructdefault· BullshitBench data docs14%
207DeepSeek V3.2high· BullshitBench data docs13%
208MiMo V2 Flashhigh· BullshitBench data docs13%
209Step 3.5 Flashminimal· BullshitBench data docs13%
210Trinity Large Thinkingxhigh· BullshitBench data docs13%
211GPT 4odefault· BullshitBench data docs12%
212GPT OSS 120Blow· BullshitBench data docs12%
213Doubao-Seed 1.6none· BullshitBench data docs11%
214ERNIE 4.5 VL 424B A47Bxhigh· BullshitBench data docs11%
215Gemini 3.1 Flash Litedefault· BullshitBench data docs11%
216Gemma 4 26B A4Bnone· BullshitBench data docs11%
217Claude 3 Haikudefault· BullshitBench data docs10%
218DeepSeek V3.2none· BullshitBench data docs10%
219Gemini 3 Flashhigh· BullshitBench data docs10%
220Gemini 3 Flashnone· BullshitBench data docs10%
221GPT 5.4 Nanoxhigh· BullshitBench data docs10%
222Grok 4.1 Fastnone· BullshitBench data docs10%
223Kimi K2default· BullshitBench data docs10%
224Nemotron 3 Nano 30B A3Bxhigh· BullshitBench data docs10%
225MiniMax M2.5low· BullshitBench data docs9%
226DeepSeek R1xhigh· BullshitBench data docs8%
227GLM 4.5xhigh· BullshitBench data docs8%
228Hy3 (Deep Infra)xhigh· BullshitBench data docs8%
229MiniMax M2.5high· BullshitBench data docs8%
230MiniMax M2.7high· BullshitBench data docs8%
231o4 Minilow· BullshitBench data docs8%
232DeepSeek R1none· BullshitBench data docs7%
233Doubao-Seed 1.6high· BullshitBench data docs7%
234MiniMax M2.7low· BullshitBench data docs7%
235GLM 4.5none· BullshitBench data docs6%
236Mistral Smallhigh· BullshitBench data docs6%
237Qwen 3 235B A22Bnone· BullshitBench data docs6%
238GPT OSS 120Bhigh· BullshitBench data docs5%
239Jamba Large 1.7default· BullshitBench data docs5%
240Nemotron Nano 9b V2none· BullshitBench data docs5%
241DeepSeek Chatdefault· BullshitBench data docs4%
242ERNIE 4.5 300B A47Bdefault· BullshitBench data docs4%
243Mistral Smallnone· BullshitBench data docs4%
244o4 Minihigh· BullshitBench data docs4%
245ERNIE 4.5 VL 424B A47Bnone· BullshitBench data docs3%
246Gemma 3 27B ITdefault· BullshitBench data docs3%
247Nemotron Nano 9b V2xhigh· BullshitBench data docs3%
248Qwen 3 235B A22Bxhigh· BullshitBench data docs3%
249GPT-4o minidefault· BullshitBench data docs2%
250Mistral Large 3default· BullshitBench data docs2%

Showing 201–250 of 250 results.

Data credits: BullshitBench by Peter Gostev, MIT (licence)