Skip to content

Benchmark

HELM Safety: BBQ - BBQ (ambiguous)

Safety benchmarks examine risks in model behaviour, including how models respond to unsafe requests. Scores are percentages. Lower is better.

Models measured
69
Scores recorded
74
Sources
1
IQ weight
0.18

Score distribution

2.4%
Mean
2.8%
Median
2.6%
Std dev
0.2%
10th percentile
4.9%
90th percentile
-8.4%
Min
-8.4%8%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.18 for chat models, measured over 69 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1DBRX-instruct· Stanford HELM specialized suites data docs-8.4%
2Qwen3 Next 80B A3Bthinking· Stanford HELM specialized suites data docs-7.7%
3Claude Sonnet 4thinking, 10k· Stanford HELM specialized suites data docs-3.4%
4DeepSeek LLM 67B Chat· Stanford HELM specialized suites data docs-1.8%
5GPT-3.5 Turbosetting not named by the source· Stanford HELM specialized suites data docs-1.3%
6GPT-3.5 Turbosetting not named by the source· Stanford HELM specialized suites data docs-1.1%
7Claude Opus 4thinking, 10k· Stanford HELM specialized suites data docs0%
8GPT OSS 120B· Stanford HELM specialized suites data docs0.167%
9Gemini 3 Pro Preview· Stanford HELM specialized suites data docs0.229%
10GPT OSS 20b· Stanford HELM specialized suites data docs0.33%
11Qwen1.5-72B-Chat· Stanford HELM specialized suites data docs0.358%
12DeepSeek R1setting not named by the source· Stanford HELM specialized suites data docs0.585%
13GPT-3.5 Turbosetting not named by the source· Stanford HELM specialized suites data docs0.639%
14Claude Opus 4setting not named by the source· Stanford HELM specialized suites data docs0.7%
15OpenAI o3· Stanford HELM specialized suites data docs1.1%
16Claude Sonnet 4.5· Stanford HELM specialized suites data docs1.1%
17GPT-5 nano· Stanford HELM specialized suites data docs1.2%
18GPT 5· Stanford HELM specialized suites data docs1.4%
19gemini-2.0-pro-exp-02-05· Stanford HELM specialized suites data docs1.5%
20DeepSeek R1setting not named by the source· Stanford HELM specialized suites data docs1.5%
21gemini-2.5-flash-preview-04-17· Stanford HELM specialized suites data docs1.7%
22Qwen2 72B Instruct· Stanford HELM specialized suites data docs1.8%
23O1 Mini· Stanford HELM specialized suites data docs1.9%
24GPT-5 mini· Stanford HELM specialized suites data docs1.9%
25o1· Stanford HELM specialized suites data docs2.1%
26Claude Sonnet 4setting not named by the source· Stanford HELM specialized suites data docs2.1%
27Grok 4· Stanford HELM specialized suites data docs2.1%
28GLM 4.5 Air· Stanford HELM specialized suites data docs2.2%
29o4 Mini· Stanford HELM specialized suites data docs2.3%
30o3 Mini· Stanford HELM specialized suites data docs2.5%
31Mistral (7B) Instruct v0.3· Stanford HELM specialized suites data docs2.6%
32Mixtral 8x22B Instruct v0.1· Stanford HELM specialized suites data docs2.7%
33Grok 3· Stanford HELM specialized suites data docs2.7%
34Meta Llama 3.1 70B Instruct Turbo· Stanford HELM specialized suites data docs2.7%
35GPT 4.1· Stanford HELM specialized suites data docs2.7%
36DeepSeek V3· Stanford HELM specialized suites data docs2.8%
37Grok 3 Mini· Stanford HELM specialized suites data docs2.8%
38Qwen2.5 7B Instruct Turbo· Stanford HELM specialized suites data docs2.8%
39Gemini 2.0 Flash-Lite Preview· Stanford HELM specialized suites data docs2.8%
40GPT 4o· Stanford HELM specialized suites data docs2.8%
41GPT-4 Turbo· Stanford HELM specialized suites data docs3%
42gemini-1.5-pro-001· Stanford HELM specialized suites data docs3.1%
43Mistral (7B) Instruct v0.1· Stanford HELM specialized suites data docs3.1%
44GPT-4.1 mini· Stanford HELM specialized suites data docs3.1%
45Qwen3 235B A22B Instruct 2507· Stanford HELM specialized suites data docs3.1%
46Gemini 2.0 Flash· Stanford HELM specialized suites data docs3.2%
47Mistral Small· Stanford HELM specialized suites data docs3.2%
48Mixtral 8x7B Instruct v0.1· Stanford HELM specialized suites data docs3.3%
49Kimi K2 Instruct· Stanford HELM specialized suites data docs3.3%
50GPT 5.1· Stanford HELM specialized suites data docs3.4%

Showing 1–50 of 74 results.

Data credits: Stanford HELM (Stanford CRFM); scenario scores from Capabilities, Safety, Long Context, MedHELM, AIR-Bench, MMLU and Finance