Skip to content

Benchmark

HELM Safety: XSTest - LM Evaluated Safety score

Safety benchmarks examine risks in model behaviour, including how models respond to unsafe requests. Scores are percentages. Higher is better.

Models measured
69
Scores recorded
74
Sources
1
IQ weight
1.41

Score distribution

94.8%
Mean
96%
Median
4.7%
Std dev
90.7%
10th percentile
98.2%
90th percentile
68.7%
Min
68.7%98.8%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.41 for chat models, measured over 69 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1gemini-2.5-flash-preview-04-17· Stanford HELM specialized suites data docs98.8%
2DeepSeek R1setting not named by the source· Stanford HELM specialized suites data docs98.8%
3Gemini 2.5 Pro Preview 03-25· Stanford HELM specialized suites data docs98.7%
4GLM 4.5 Air· Stanford HELM specialized suites data docs98.6%
5Qwen3 235B A22B Instruct 2507· Stanford HELM specialized suites data docs98.6%
6Grok 3 Mini· Stanford HELM specialized suites data docs98.4%
7Qwen3 Next 80B A3Bthinking· Stanford HELM specialized suites data docs98.3%
8GPT 5.1· Stanford HELM specialized suites data docs98.2%
9Kimi K2 Instruct· Stanford HELM specialized suites data docs98.2%
10Qwen2.5 72B Instruct Turbo· Stanford HELM specialized suites data docs97.9%
11GPT 4.1· Stanford HELM specialized suites data docs97.9%
12Gemini 2.5 Flash-Lite· Stanford HELM specialized suites data docs97.8%
13GPT-4 Turbo· Stanford HELM specialized suites data docs97.7%
14GPT-5 mini· Stanford HELM specialized suites data docs97.7%
15GPT-5 nano· Stanford HELM specialized suites data docs97.6%
16GPT-4.1 mini· Stanford HELM specialized suites data docs97.4%
17o4 Mini· Stanford HELM specialized suites data docs97.4%
18Gemini 3 Pro Preview· Stanford HELM specialized suites data docs97.3%
19GPT 4o· Stanford HELM specialized suites data docs97.3%
20OpenAI o3· Stanford HELM specialized suites data docs97.3%
21Claude Sonnet 4setting not named by the source· Stanford HELM specialized suites data docs97.2%
22DeepSeek V3· Stanford HELM specialized suites data docs97.1%
23GPT 5· Stanford HELM specialized suites data docs97.1%
24Claude Opus 4setting not named by the source· Stanford HELM specialized suites data docs97%
25Claude Opus 4thinking, 10k· Stanford HELM specialized suites data docs97%
26o1· Stanford HELM specialized suites data docs97%
27O1 Mini· Stanford HELM specialized suites data docs97%
28Qwen2 72B Instruct· Stanford HELM specialized suites data docs96.9%
29Claude Sonnet 4thinking, 10k· Stanford HELM specialized suites data docs96.6%
30Grok 4· Stanford HELM specialized suites data docs96.6%
31Qwen2.5 7B Instruct Turbo· Stanford HELM specialized suites data docs96.6%
32Llama 4 Maverick· Stanford HELM specialized suites data docs96.5%
33Claude Sonnet 3.7· Stanford HELM specialized suites data docs96.4%
34Grok 3· Stanford HELM specialized suites data docs96.4%
35Claude Sonnet 4.5· Stanford HELM specialized suites data docs96.4%
36GPT-4.1 nano· Stanford HELM specialized suites data docs96%
37GPT-4o mini· Stanford HELM specialized suites data docs96%
38Llama 3.1 405B Instruct Turbo· Stanford HELM specialized suites data docs95.9%
39Qwen1.5-72B-Chat· Stanford HELM specialized suites data docs95.7%
40Claude 3.5 Sonnet· Stanford HELM specialized suites data docs95.6%
41Llama3 8B Chat· Stanford HELM specialized suites data docs95.6%
42Llama 4 Scout· Stanford HELM specialized suites data docs95.6%
43Olmo 2 32B Instruct· Stanford HELM specialized suites data docs95.4%
44gemini-2.0-pro-exp-02-05· Stanford HELM specialized suites data docs95.4%
45DeepSeek R1setting not named by the source· Stanford HELM specialized suites data docs95.3%
46Llama 3.1 8B Instruct Turbo· Stanford HELM specialized suites data docs95.3%
47Palmyra X5· Stanford HELM specialized suites data docs95.3%
48Gemini 2.0 Flash· Stanford HELM specialized suites data docs95.3%
49Mistral Small· Stanford HELM specialized suites data docs95.3%
50GPT-4.5 Preview· Stanford HELM specialized suites data docs95.1%

Showing 1–50 of 74 results.

Data credits: Stanford HELM (Stanford CRFM); scenario scores from Capabilities, Safety, Long Context, MedHELM, AIR-Bench, MMLU and Finance