Skip to content

Benchmark

BullshitBench v2

Higher scores are better.

Models measured
137
Scores recorded
250
Sources
1
IQ weight
1.14

Score distribution

1
Mean
1.1
Median
0.489
Std dev
0.345
10th percentile
1.6
90th percentile
0.147
Min
0.1471.9
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.14 for chat models, measured over 137 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
201GPT 5.4 Nanonone· BullshitBench data docs0.5
202GPT 4odefault· BullshitBench data docs0.493
203DeepSeek V4 Pronone· BullshitBench data docs0.487
204Gemini 3.5 Flashminimal· BullshitBench data docs0.487
205Gemini 2.0 Flashdefault· BullshitBench data docs0.48
206DeepSeek V3.2high· BullshitBench data docs0.46
207Hy3 (Deep Infra)xhigh· BullshitBench data docs0.457
208MiniMax M2.5low· BullshitBench data docs0.443
209GPT 4.1default· BullshitBench data docs0.433
210Step 3.5 Flashminimal· BullshitBench data docs0.427
211Trinity Large Thinkingxhigh· BullshitBench data docs0.423
212GPT 5.4 Nanoxhigh· BullshitBench data docs0.417
213Doubao-Seed 1.6high· BullshitBench data docs0.407
214Gemini 3 Flashhigh· BullshitBench data docs0.407
215Gemma 4 26B A4Bnone· BullshitBench data docs0.407
216Gemini 3.1 Flash Litedefault· BullshitBench data docs0.403
217GPT OSS 120Blow· BullshitBench data docs0.402
218DeepSeek V4 Flashxhigh· BullshitBench data docs0.4
219DeepSeek V3.2none· BullshitBench data docs0.397
220Doubao-Seed 1.6none· BullshitBench data docs0.397
221MiMo V2 Flashhigh· BullshitBench data docs0.397
222MiniMax M2.7low· BullshitBench data docs0.397
223MiniMax M2.5high· BullshitBench data docs0.393
224ERNIE 4.5 VL 424B A47Bxhigh· BullshitBench data docs0.357
225Claude 3 Haikudefault· BullshitBench data docs0.345
226DeepSeek R1none· BullshitBench data docs0.343
227Gemini 3 Flashnone· BullshitBench data docs0.337
228GLM 4.5xhigh· BullshitBench data docs0.337
229Nemotron 3 Nano 30B A3Bxhigh· BullshitBench data docs0.33
230Kimi K2default· BullshitBench data docs0.327
231MiniMax M2.7high· BullshitBench data docs0.327
232Mistral Smallhigh· BullshitBench data docs0.327
233DeepSeek R1xhigh· BullshitBench data docs0.31
234Qwen 3 235B A22Bnone· BullshitBench data docs0.307
235GLM 4.5none· BullshitBench data docs0.3
236Grok 4.1 Fastnone· BullshitBench data docs0.3
237GPT OSS 120Bhigh· BullshitBench data docs0.297
238ERNIE 4.5 VL 424B A47Bnone· BullshitBench data docs0.293
239o4 Minihigh· BullshitBench data docs0.293
240o4 Minilow· BullshitBench data docs0.277
241Qwen 3 235B A22Bxhigh· BullshitBench data docs0.273
242ERNIE 4.5 300B A47Bdefault· BullshitBench data docs0.267
243Nemotron Nano 9b V2none· BullshitBench data docs0.257
244Nemotron Nano 9b V2xhigh· BullshitBench data docs0.24
245DeepSeek Chatdefault· BullshitBench data docs0.217
246Jamba Large 1.7default· BullshitBench data docs0.187
247Mistral Large 3default· BullshitBench data docs0.173
248Mistral Smallnone· BullshitBench data docs0.16
249GPT-4o minidefault· BullshitBench data docs0.157
250Gemma 3 27B ITdefault· BullshitBench data docs0.147

Showing 201–250 of 250 results.

Data credits: BullshitBench by Peter Gostev, MIT (licence)