Skip to content

Benchmark

ChemBench Technical Chemistry

Science benchmarks measure how well models answer science questions. Higher scores are better.

Models measured
42
Scores recorded
51
Sources
1
IQ weight
1.53

Score distribution

0.611
Mean
0.65
Median
0.201
Std dev
0.375
10th percentile
0.85
90th percentile
0.025
Min
0.0250.925
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.53 for chat models, measured over 42 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.7claude-opus-4-7-tinyeval· ChemBench verified leaderboard data docs0.925
2Claude 3.5 Sonnetclaude3.5· ChemBench verified leaderboard data docs0.85
3DeepSeek R1deepseek-r1-tinyeval· ChemBench verified leaderboard data docs0.85
4DeepSeek V3.2deepseek-v3-2-tinyeval· ChemBench verified leaderboard data docs0.85
5Kimi K2.5kimi-k2-5-tinyeval· ChemBench verified leaderboard data docs0.85
6O1 Previewo1-preview· ChemBench verified leaderboard data docs0.85
7Claude Opus 4.5claude-opus-4-5-tinyeval· ChemBench verified leaderboard data docs0.825
8Claude Opus 4.8claude-opus-4-8-tinyeval· ChemBench verified leaderboard data docs0.825
9Claude 3.5 Sonnetclaude3.5-react· ChemBench verified leaderboard data docs0.8
10gemini-2.5-flash-preview-04-17gemini-2.5-flash-preview-04-17-base-1750491557· ChemBench verified leaderboard data docs0.775
11Kimi K2 Instructkimi-k2-instruct-0905-base-1760364223· ChemBench verified leaderboard data docs0.775
12GPT 4ogpt-4o· ChemBench verified leaderboard data docs0.75
13GPT OSS 120Bgpt-oss-120b-base-1760293341· ChemBench verified leaderboard data docs0.75
14Llama 4 Maverickgroq_meta-llama/llama-4-maverick-17b-128e-instruct_e8a077d8· ChemBench verified leaderboard data docs0.75
15GLM 5glm-5-tinyeval· ChemBench verified leaderboard data docs0.725
16GPT 4ogpt-4o-react· ChemBench verified leaderboard data docs0.725
17GPT OSS 20bgpt-oss-20b-base-1760359009· ChemBench verified leaderboard data docs0.725
18GPT-5 nanoGPT-5-nano-base-1764459351· ChemBench verified leaderboard data docs0.725
19MiniMax M2.5minimax-m2-5-tinyeval· ChemBench verified leaderboard data docs0.725
20Nemotron Super 3 120bnemotron-super-3-120b-tinyeval· ChemBench verified leaderboard data docs0.725
21Qwen3 235B A22B Instruct 2507qwen3-235b-a22b-2507-tinyeval· ChemBench verified leaderboard data docs0.725
22Qwen3 Next 80B A3Bqwen3-next-80b-a3b-tinyeval· ChemBench verified leaderboard data docs0.725
23GPT 4gpt-4· ChemBench verified leaderboard data docs0.7
24Llama 3.1 405B Instructllama3.1-405b-instruct· ChemBench verified leaderboard data docs0.7
25Mistral Large 2mistral-large-2-123b· ChemBench verified leaderboard data docs0.675
26Llama 3.1 70B Instructllama3.1-70b-instruct· ChemBench verified leaderboard data docs0.65
27GPT-4o miniGPT-4o-mini-base-1764283510· ChemBench verified leaderboard data docs0.625
28Llama 3 70B Instructllama3-70b-instruct· ChemBench verified leaderboard data docs0.625
29Llama 3 8B Instructllama3-8b-instruct-T-one· ChemBench verified leaderboard data docs0.625
30Llama 3 70B Instructllama3-70b-instruct-T-one· ChemBench verified leaderboard data docs0.6
31Llama 3 8B Instructllama3-8b-instruct· ChemBench verified leaderboard data docs0.6
32Nemotron Nano 3 30Bnemotron-nano-3-30b-tinyeval· ChemBench verified leaderboard data docs0.575
33Qwen2.5 32BQwen-2.5-32B-base-1740920072· ChemBench verified leaderboard data docs0.575
34Claude 3 Haikuclaude-3-haiku-tinyeval· ChemBench verified leaderboard data docs0.55
35Llama 3.1 70B Instructllama3.1-70b-instruct-T-one· ChemBench verified leaderboard data docs0.55
36Phi-3 Medium 4K Instructphi-3-medium-4k-instruct· ChemBench verified leaderboard data docs0.55
37Gemma 2 9B ITgemma-2-9b-it· ChemBench verified leaderboard data docs0.525
38Command R Pluscommand-r+· ChemBench verified leaderboard data docs0.5
39Claude 2claude2· ChemBench verified leaderboard data docs0.475
40Gemini Progemini-pro· ChemBench verified leaderboard data docs0.475
41Gemma 2 9B ITgemma-2-9b-it-T-one· ChemBench verified leaderboard data docs0.475
42Llama 3.1 8B Instructllama3.1-8b-instruct· ChemBench verified leaderboard data docs0.45
43GPT-3.5 Turbogpt-3.5-turbo· ChemBench verified leaderboard data docs0.4
44Llama 3.1 8B Instructllama3.1-8b-instruct-T-one· ChemBench verified leaderboard data docs0.4
45Gemma 1.1 7B ITgemma-1-1-7b-it· ChemBench verified leaderboard data docs0.375
46Gemma 1.1 7B ITgemma-1-1-7b-it-T-one· ChemBench verified leaderboard data docs0.375
47Mixtral 8x7B Instructmixtral-8x7b-instruct· ChemBench verified leaderboard data docs0.325
48Mixtral 8x7B Instructmixtral-8x7b-instruct-T-one· ChemBench verified leaderboard data docs0.325
49Llama 2 70B Chatllama2-70b-chat· ChemBench verified leaderboard data docs0.125
50Llama 2 13B Chatllama2-13b-chat· ChemBench verified leaderboard data docs0.1

Showing 1–50 of 51 results.

Data credits: ChemBench leaderboard, Jablonka group / lamalab-org. Public verified numeric result summaries; results licence not separately stated.