Skip to content

Benchmark

ChemBench Physical Chemistry

Science benchmarks measure how well models answer science questions. Higher scores are better.

Models measured
42
Scores recorded
51
Sources
1
IQ weight
1.30

Score distribution

0.595
Mean
0.642
Median
0.214
Std dev
0.327
10th percentile
0.848
90th percentile
0.0242
Min
0.02420.897
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.30 for chat models, measured over 42 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.7claude-opus-4-7-tinyeval· ChemBench verified leaderboard data docs0.897
2Kimi K2.5kimi-k2-5-tinyeval· ChemBench verified leaderboard data docs0.891
3DeepSeek V3.2deepseek-v3-2-tinyeval· ChemBench verified leaderboard data docs0.879
4Claude Opus 4.8claude-opus-4-8-tinyeval· ChemBench verified leaderboard data docs0.855
5DeepSeek R1deepseek-r1-tinyeval· ChemBench verified leaderboard data docs0.848
6O1 Previewo1-preview· ChemBench verified leaderboard data docs0.848
7Claude Opus 4.5claude-opus-4-5-tinyeval· ChemBench verified leaderboard data docs0.836
8GPT-5 nanoGPT-5-nano-base-1764459351· ChemBench verified leaderboard data docs0.83
9Qwen3 235B A22B Instruct 2507qwen3-235b-a22b-2507-tinyeval· ChemBench verified leaderboard data docs0.818
10MiniMax M2.5minimax-m2-5-tinyeval· ChemBench verified leaderboard data docs0.806
11Qwen3 Next 80B A3Bqwen3-next-80b-a3b-tinyeval· ChemBench verified leaderboard data docs0.788
12GPT OSS 120Bgpt-oss-120b-base-1760293341· ChemBench verified leaderboard data docs0.782
13GPT OSS 20bgpt-oss-20b-base-1760359009· ChemBench verified leaderboard data docs0.776
14Claude 3.5 Sonnetclaude3.5· ChemBench verified leaderboard data docs0.77
15Kimi K2 Instructkimi-k2-instruct-0905-base-1760364223· ChemBench verified leaderboard data docs0.764
16Claude 3.5 Sonnetclaude3.5-react· ChemBench verified leaderboard data docs0.758
17gemini-2.5-flash-preview-04-17gemini-2.5-flash-preview-04-17-base-1750491557· ChemBench verified leaderboard data docs0.745
18Llama 4 Maverickgroq_meta-llama/llama-4-maverick-17b-128e-instruct_e8a077d8· ChemBench verified leaderboard data docs0.739
19GPT 4ogpt-4o· ChemBench verified leaderboard data docs0.715
20Llama 3.1 405B Instructllama3.1-405b-instruct· ChemBench verified leaderboard data docs0.709
21Mistral Large 2mistral-large-2-123b· ChemBench verified leaderboard data docs0.691
22GLM 5glm-5-tinyeval· ChemBench verified leaderboard data docs0.685
23Qwen2.5 32BQwen-2.5-32B-base-1740920072· ChemBench verified leaderboard data docs0.655
24Nemotron Super 3 120bnemotron-super-3-120b-tinyeval· ChemBench verified leaderboard data docs0.648
25GPT 4gpt-4· ChemBench verified leaderboard data docs0.642
26Llama 3.1 70B Instructllama3.1-70b-instruct· ChemBench verified leaderboard data docs0.642
27GPT 4ogpt-4o-react· ChemBench verified leaderboard data docs0.6
28Llama 3 70B Instructllama3-70b-instruct-T-one· ChemBench verified leaderboard data docs0.6
29Llama 3 70B Instructllama3-70b-instruct· ChemBench verified leaderboard data docs0.594
30Claude 3 Haikuclaude-3-haiku-tinyeval· ChemBench verified leaderboard data docs0.582
31GPT-4o miniGPT-4o-mini-base-1764283510· ChemBench verified leaderboard data docs0.582
32Llama 3.1 70B Instructllama3.1-70b-instruct-T-one· ChemBench verified leaderboard data docs0.558
33Llama 3.1 8B Instructllama3.1-8b-instruct· ChemBench verified leaderboard data docs0.509
34Claude 2claude2· ChemBench verified leaderboard data docs0.503
35Gemma 2 9B ITgemma-2-9b-it· ChemBench verified leaderboard data docs0.485
36Gemma 2 9B ITgemma-2-9b-it-T-one· ChemBench verified leaderboard data docs0.467
37Phi-3 Medium 4K Instructphi-3-medium-4k-instruct· ChemBench verified leaderboard data docs0.461
38Gemini Progemini-pro· ChemBench verified leaderboard data docs0.448
39Nemotron Nano 3 30Bnemotron-nano-3-30b-tinyeval· ChemBench verified leaderboard data docs0.442
40Llama 3.1 8B Instructllama3.1-8b-instruct-T-one· ChemBench verified leaderboard data docs0.418
41GPT-3.5 Turbogpt-3.5-turbo· ChemBench verified leaderboard data docs0.4
42Llama 3 8B Instructllama3-8b-instruct· ChemBench verified leaderboard data docs0.37
43Llama 3 8B Instructllama3-8b-instruct-T-one· ChemBench verified leaderboard data docs0.345
44Mixtral 8x7B Instructmixtral-8x7b-instruct-T-one· ChemBench verified leaderboard data docs0.345
45Mixtral 8x7B Instructmixtral-8x7b-instruct· ChemBench verified leaderboard data docs0.333
46Command R Pluscommand-r+· ChemBench verified leaderboard data docs0.327
47Gemma 1.1 7B ITgemma-1-1-7b-it-T-one· ChemBench verified leaderboard data docs0.303
48Gemma 1.1 7B ITgemma-1-1-7b-it· ChemBench verified leaderboard data docs0.291
49Llama 2 70B Chatllama2-70b-chat· ChemBench verified leaderboard data docs0.17
50Llama 2 13B Chatllama2-13b-chat· ChemBench verified leaderboard data docs0.152

Showing 1–50 of 51 results.

Data credits: ChemBench leaderboard, Jablonka group / lamalab-org. Public verified numeric result summaries; results licence not separately stated.