Skip to content

Benchmark

HELM Safety: BBQ - BBQ (unambiguous)

Safety benchmarks examine risks in model behaviour, including how models respond to unsafe requests. Scores are percentages. Lower is better.

Models measured
69
Scores recorded
74
Sources
1
IQ weight
1.17

Score distribution

-19.6%
Mean
-19.4%
Median
2.6%
Std dev
-21.4%
10th percentile
-16.9%
90th percentile
-34.6%
Min
-34.6%-11.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.17 for chat models, measured over 69 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1DBRX-instruct· Stanford HELM specialized suites data docs-34.6%
2Claude Sonnet 3.7· Stanford HELM specialized suites data docs-24.3%
3Qwen3 Next 80B A3Bthinking· Stanford HELM specialized suites data docs-23.5%
4Granite 3.3 8b Instruct· Stanford HELM specialized suites data docs-22.5%
5GPT-4.5 Preview· Stanford HELM specialized suites data docs-22.2%
6Claude 3.5 Sonnet· Stanford HELM specialized suites data docs-22%
7Claude Sonnet 4setting not named by the source· Stanford HELM specialized suites data docs-22%
8Claude Opus 4thinking, 10k· Stanford HELM specialized suites data docs-21.4%
9GPT-4 Turbo· Stanford HELM specialized suites data docs-21.3%
10GPT 5· Stanford HELM specialized suites data docs-21.2%
11Claude Sonnet 4.5· Stanford HELM specialized suites data docs-21.2%
12Claude Sonnet 4thinking, 10k· Stanford HELM specialized suites data docs-21.2%
13Grok 3 Mini· Stanford HELM specialized suites data docs-21%
14GPT OSS 20b· Stanford HELM specialized suites data docs-21%
15Grok 4· Stanford HELM specialized suites data docs-20.9%
16gemini-1.5-pro-001· Stanford HELM specialized suites data docs-20.6%
17Qwen2.5 72B Instruct Turbo· Stanford HELM specialized suites data docs-20.6%
18GPT 5.1· Stanford HELM specialized suites data docs-20.5%
19Qwen2.5 7B Instruct Turbo· Stanford HELM specialized suites data docs-20.3%
20Gemini 3 Pro Preview· Stanford HELM specialized suites data docs-20.3%
21O1 Mini· Stanford HELM specialized suites data docs-20.3%
22Claude 3 Opus· Stanford HELM specialized suites data docs-20.3%
23Llama 3.1 405B Instruct Turbo· Stanford HELM specialized suites data docs-20.2%
24OpenAI o3· Stanford HELM specialized suites data docs-20.2%
25Claude Haiku 4.5· Stanford HELM specialized suites data docs-20.1%
26o1· Stanford HELM specialized suites data docs-20.1%
27Claude Opus 4setting not named by the source· Stanford HELM specialized suites data docs-20%
28Palmyra X5· Stanford HELM specialized suites data docs-20%
29Llama 4 Maverick· Stanford HELM specialized suites data docs-19.9%
30GPT 4o· Stanford HELM specialized suites data docs-19.8%
31o3 Mini· Stanford HELM specialized suites data docs-19.7%
32GLM 4.5 Air· Stanford HELM specialized suites data docs-19.7%
33gemini-2.0-pro-exp-02-05· Stanford HELM specialized suites data docs-19.7%
34Grok 3· Stanford HELM specialized suites data docs-19.6%
35GPT OSS 120B· Stanford HELM specialized suites data docs-19.4%
36Mixtral 8x22B Instruct v0.1· Stanford HELM specialized suites data docs-19.4%
37Command R+· Stanford HELM specialized suites data docs-19.4%
38GPT-5 mini· Stanford HELM specialized suites data docs-19.4%
39Qwen2 72B Instruct· Stanford HELM specialized suites data docs-19.4%
40DeepSeek R1setting not named by the source· Stanford HELM specialized suites data docs-19.3%
41GPT 4.1· Stanford HELM specialized suites data docs-19.3%
42Qwen3 235B A22B Instruct 2507· Stanford HELM specialized suites data docs-19.3%
43DeepSeek R1setting not named by the source· Stanford HELM specialized suites data docs-19.3%
44Gemini 2.5 Pro Preview 03-25· Stanford HELM specialized suites data docs-19.2%
45Gemini 2.0 Flash-Lite Preview· Stanford HELM specialized suites data docs-19.2%
46Kimi K2 Instruct· Stanford HELM specialized suites data docs-19.2%
47o4 Mini· Stanford HELM specialized suites data docs-19.2%
48Olmo 2 32B Instruct· Stanford HELM specialized suites data docs-19.2%
49gemini-2.5-flash-preview-04-17· Stanford HELM specialized suites data docs-19.2%
50Claude 3 Sonnet· Stanford HELM specialized suites data docs-19%

Showing 1–50 of 74 results.

Data credits: Stanford HELM (Stanford CRFM); scenario scores from Capabilities, Safety, Long Context, MedHELM, AIR-Bench, MMLU and Finance