Skip to content

Benchmark

ChemBench Toxicity and Safety

Science benchmarks measure how well models answer science questions. Higher scores are better.

Models measured
42
Scores recorded
51
Sources
1
IQ weight
1.49

Score distribution

0.36
Mean
0.373
Median
0.095
Std dev
0.267
10th percentile
0.474
90th percentile
0.0148
Min
0.01480.516
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.49 for chat models, measured over 42 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Llama 4 Maverickgroq_meta-llama/llama-4-maverick-17b-128e-instruct_e8a077d8· ChemBench verified leaderboard data docs0.516
2DeepSeek R1deepseek-r1-tinyeval· ChemBench verified leaderboard data docs0.483
3Kimi K2.5kimi-k2-5-tinyeval· ChemBench verified leaderboard data docs0.477
4gemini-2.5-flash-preview-04-17gemini-2.5-flash-preview-04-17-base-1750491557· ChemBench verified leaderboard data docs0.476
5O1 Previewo1-preview· ChemBench verified leaderboard data docs0.476
6Claude Opus 4.7claude-opus-4-7-tinyeval· ChemBench verified leaderboard data docs0.474
7DeepSeek V3.2deepseek-v3-2-tinyeval· ChemBench verified leaderboard data docs0.455
8GPT-5 nanoGPT-5-nano-base-1764459351· ChemBench verified leaderboard data docs0.452
9GPT 4ogpt-4o· ChemBench verified leaderboard data docs0.441
10Claude 3.5 Sonnetclaude3.5· ChemBench verified leaderboard data docs0.44
11Kimi K2 Instructkimi-k2-instruct-0905-base-1760364223· ChemBench verified leaderboard data docs0.433
12GPT OSS 120Bgpt-oss-120b-base-1760293341· ChemBench verified leaderboard data docs0.427
13Llama 3.1 405B Instructllama3.1-405b-instruct· ChemBench verified leaderboard data docs0.419
14Qwen3 235B A22B Instruct 2507qwen3-235b-a22b-2507-tinyeval· ChemBench verified leaderboard data docs0.412
15GPT 4gpt-4· ChemBench verified leaderboard data docs0.41
16Claude 3.5 Sonnetclaude3.5-react· ChemBench verified leaderboard data docs0.409
17Claude Opus 4.8claude-opus-4-8-tinyeval· ChemBench verified leaderboard data docs0.403
18Mistral Large 2mistral-large-2-123b· ChemBench verified leaderboard data docs0.396
19GPT OSS 20bgpt-oss-20b-base-1760359009· ChemBench verified leaderboard data docs0.39
20Llama 3.1 70B Instructllama3.1-70b-instruct-T-one· ChemBench verified leaderboard data docs0.39
21MiniMax M2.5minimax-m2-5-tinyeval· ChemBench verified leaderboard data docs0.387
22Claude Opus 4.5claude-opus-4-5-tinyeval· ChemBench verified leaderboard data docs0.384
23Llama 3.1 70B Instructllama3.1-70b-instruct· ChemBench verified leaderboard data docs0.384
24Qwen2.5 32BQwen-2.5-32B-base-1740920072· ChemBench verified leaderboard data docs0.384
25GPT 4ogpt-4o-react· ChemBench verified leaderboard data docs0.375
26Llama 3 70B Instructllama3-70b-instruct-T-one· ChemBench verified leaderboard data docs0.373
27GPT-4o miniGPT-4o-mini-base-1764283510· ChemBench verified leaderboard data docs0.372
28Llama 3 70B Instructllama3-70b-instruct· ChemBench verified leaderboard data docs0.369
29Nemotron Super 3 120bnemotron-super-3-120b-tinyeval· ChemBench verified leaderboard data docs0.366
30Qwen3 Next 80B A3Bqwen3-next-80b-a3b-tinyeval· ChemBench verified leaderboard data docs0.366
31Gemma 2 9B ITgemma-2-9b-it-T-one· ChemBench verified leaderboard data docs0.342
32Gemma 2 9B ITgemma-2-9b-it· ChemBench verified leaderboard data docs0.339
33Claude 3 Haikuclaude-3-haiku-tinyeval· ChemBench verified leaderboard data docs0.338
34GLM 5glm-5-tinyeval· ChemBench verified leaderboard data docs0.338
35Claude 2claude2· ChemBench verified leaderboard data docs0.332
36Phi-3 Medium 4K Instructphi-3-medium-4k-instruct· ChemBench verified leaderboard data docs0.332
37Nemotron Nano 3 30Bnemotron-nano-3-30b-tinyeval· ChemBench verified leaderboard data docs0.329
38Llama 3.1 8B Instructllama3.1-8b-instruct· ChemBench verified leaderboard data docs0.326
39Llama 3 8B Instructllama3-8b-instruct· ChemBench verified leaderboard data docs0.324
40Llama 3 8B Instructllama3-8b-instruct-T-one· ChemBench verified leaderboard data docs0.324
41Llama 3.1 8B Instructllama3.1-8b-instruct-T-one· ChemBench verified leaderboard data docs0.32
42Command R Pluscommand-r+· ChemBench verified leaderboard data docs0.311
43GPT-3.5 Turbogpt-3.5-turbo· ChemBench verified leaderboard data docs0.31
44Gemini Progemini-pro· ChemBench verified leaderboard data docs0.308
45Mixtral 8x7B Instructmixtral-8x7b-instruct· ChemBench verified leaderboard data docs0.27
46Mixtral 8x7B Instructmixtral-8x7b-instruct-T-one· ChemBench verified leaderboard data docs0.267
47Gemma 1.1 7B ITgemma-1-1-7b-it· ChemBench verified leaderboard data docs0.23
48Gemma 1.1 7B ITgemma-1-1-7b-it-T-one· ChemBench verified leaderboard data docs0.216
49Llama 2 70B Chatllama2-70b-chat· ChemBench verified leaderboard data docs0.136
50Llama 2 13B Chatllama2-13b-chat· ChemBench verified leaderboard data docs0.101

Showing 1–50 of 51 results.

Data credits: ChemBench leaderboard, Jablonka group / lamalab-org. Public verified numeric result summaries; results licence not separately stated.