Skip to content

Benchmark

HELM Safety: BBQ - BBQ (ambiguous)

Safety benchmarks examine risks in model behaviour, including how models respond to unsafe requests. Scores are percentages. Lower is better.

Models measured
69
Scores recorded
74
Sources
1
IQ weight
0.18

Score distribution

2.4%
Mean
2.8%
Median
2.6%
Std dev
0.2%
10th percentile
4.9%
90th percentile
-8.4%
Min
-8.4%8%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.18 for chat models, measured over 69 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51Palmyra X5· Stanford HELM specialized suites data docs3.5%
52Command R Plus· Stanford HELM specialized suites data docs3.5%
53Command R+· Stanford HELM specialized suites data docs3.6%
54Gemini 2.5 Pro Preview 03-25· Stanford HELM specialized suites data docs3.6%
55gemini-1.5-flash-001· Stanford HELM specialized suites data docs3.7%
56Llama 4 Maverick· Stanford HELM specialized suites data docs3.7%
57Qwen2.5 72B Instruct Turbo· Stanford HELM specialized suites data docs3.8%
58Gemini 2.5 Flash-Lite· Stanford HELM specialized suites data docs3.8%
59GPT-4.1 nano· Stanford HELM specialized suites data docs4%
60Claude 3 Haiku· Stanford HELM specialized suites data docs4.1%
61Granite 4.0 H Small· Stanford HELM specialized suites data docs4.1%
62Claude 3 Sonnet· Stanford HELM specialized suites data docs4.3%
63Claude 3 Opus· Stanford HELM specialized suites data docs4.6%
64Llama 3.1 405B Instruct Turbo· Stanford HELM specialized suites data docs4.7%
65Llama 4 Scout· Stanford HELM specialized suites data docs4.8%
66GPT-4o mini· Stanford HELM specialized suites data docs4.9%
67Llama3 8B Chat· Stanford HELM specialized suites data docs4.9%
68Olmo 2 32B Instruct· Stanford HELM specialized suites data docs4.9%
69Granite 3.3 8b Instruct· Stanford HELM specialized suites data docs5.1%
70Claude 3.5 Sonnet· Stanford HELM specialized suites data docs5.1%
71Llama 3.1 8B Instruct Turbo· Stanford HELM specialized suites data docs6.5%
72Claude Haiku 4.5· Stanford HELM specialized suites data docs7.2%
73Claude Sonnet 3.7· Stanford HELM specialized suites data docs7.9%
74GPT-4.5 Preview· Stanford HELM specialized suites data docs8%

Showing 51–74 of 74 results.

Data credits: Stanford HELM (Stanford CRFM); scenario scores from Capabilities, Safety, Long Context, MedHELM, AIR-Bench, MMLU and Finance