Skip to content

Benchmark

ChemBench Organic Chemistry

Science benchmarks measure how well models answer science questions. Higher scores are better.

Models measured
42
Scores recorded
51
Sources
1
IQ weight
1.75

Score distribution

0.654
Mean
0.663
Median
0.193
Std dev
0.472
10th percentile
0.837
90th percentile
0.021
Min
0.0210.914
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.75 for chat models, measured over 42 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.7claude-opus-4-7-tinyeval· ChemBench verified leaderboard data docs0.914
2Claude Opus 4.8claude-opus-4-8-tinyeval· ChemBench verified leaderboard data docs0.904
3Claude Opus 4.5claude-opus-4-5-tinyeval· ChemBench verified leaderboard data docs0.881
4Kimi K2.5kimi-k2-5-tinyeval· ChemBench verified leaderboard data docs0.874
5DeepSeek V3.2deepseek-v3-2-tinyeval· ChemBench verified leaderboard data docs0.865
6Claude 3.5 Sonnetclaude3.5-react· ChemBench verified leaderboard data docs0.837
7Llama 4 Maverickgroq_meta-llama/llama-4-maverick-17b-128e-instruct_e8a077d8· ChemBench verified leaderboard data docs0.832
8O1 Previewo1-preview· ChemBench verified leaderboard data docs0.83
9Claude 3.5 Sonnetclaude3.5· ChemBench verified leaderboard data docs0.826
10DeepSeek R1deepseek-r1-tinyeval· ChemBench verified leaderboard data docs0.825
11gemini-2.5-flash-preview-04-17gemini-2.5-flash-preview-04-17-base-1750491557· ChemBench verified leaderboard data docs0.825
12Qwen3 Next 80B A3Bqwen3-next-80b-a3b-tinyeval· ChemBench verified leaderboard data docs0.821
13GPT OSS 120Bgpt-oss-120b-base-1760293341· ChemBench verified leaderboard data docs0.818
14GPT-5 nanoGPT-5-nano-base-1764459351· ChemBench verified leaderboard data docs0.818
15Kimi K2 Instructkimi-k2-instruct-0905-base-1760364223· ChemBench verified leaderboard data docs0.814
16GPT OSS 20bgpt-oss-20b-base-1760359009· ChemBench verified leaderboard data docs0.797
17GLM 5glm-5-tinyeval· ChemBench verified leaderboard data docs0.786
18Qwen3 235B A22B Instruct 2507qwen3-235b-a22b-2507-tinyeval· ChemBench verified leaderboard data docs0.786
19GPT 4ogpt-4o· ChemBench verified leaderboard data docs0.756
20Llama 3.1 405B Instructllama3.1-405b-instruct· ChemBench verified leaderboard data docs0.756
21MiniMax M2.5minimax-m2-5-tinyeval· ChemBench verified leaderboard data docs0.737
22Mistral Large 2mistral-large-2-123b· ChemBench verified leaderboard data docs0.733
23Nemotron Super 3 120bnemotron-super-3-120b-tinyeval· ChemBench verified leaderboard data docs0.725
24GPT 4ogpt-4o-react· ChemBench verified leaderboard data docs0.719
25GPT 4gpt-4· ChemBench verified leaderboard data docs0.679
26Llama 3.1 70B Instructllama3.1-70b-instruct· ChemBench verified leaderboard data docs0.663
27Qwen2.5 32BQwen-2.5-32B-base-1740920072· ChemBench verified leaderboard data docs0.639
28Llama 3 70B Instructllama3-70b-instruct· ChemBench verified leaderboard data docs0.633
29Llama 3 70B Instructllama3-70b-instruct-T-one· ChemBench verified leaderboard data docs0.633
30GPT-4o miniGPT-4o-mini-base-1764283510· ChemBench verified leaderboard data docs0.629
31Claude 3 Haikuclaude-3-haiku-tinyeval· ChemBench verified leaderboard data docs0.625
32Claude 2claude2· ChemBench verified leaderboard data docs0.593
33GPT-3.5 Turbogpt-3.5-turbo· ChemBench verified leaderboard data docs0.588
34Llama 3.1 8B Instructllama3.1-8b-instruct· ChemBench verified leaderboard data docs0.581
35Llama 3.1 8B Instructllama3.1-8b-instruct-T-one· ChemBench verified leaderboard data docs0.577
36Gemini Progemini-pro· ChemBench verified leaderboard data docs0.567
37Nemotron Nano 3 30Bnemotron-nano-3-30b-tinyeval· ChemBench verified leaderboard data docs0.564
38Llama 3 8B Instructllama3-8b-instruct· ChemBench verified leaderboard data docs0.563
39Phi-3 Medium 4K Instructphi-3-medium-4k-instruct· ChemBench verified leaderboard data docs0.56
40Gemma 2 9B ITgemma-2-9b-it· ChemBench verified leaderboard data docs0.556
41Llama 3.1 70B Instructllama3.1-70b-instruct-T-one· ChemBench verified leaderboard data docs0.553
42Command R Pluscommand-r+· ChemBench verified leaderboard data docs0.551
43Llama 3 8B Instructllama3-8b-instruct-T-one· ChemBench verified leaderboard data docs0.551
44Gemma 2 9B ITgemma-2-9b-it-T-one· ChemBench verified leaderboard data docs0.547
45Mixtral 8x7B Instructmixtral-8x7b-instruct· ChemBench verified leaderboard data docs0.479
46Mixtral 8x7B Instructmixtral-8x7b-instruct-T-one· ChemBench verified leaderboard data docs0.472
47Gemma 1.1 7B ITgemma-1-1-7b-it· ChemBench verified leaderboard data docs0.379
48Gemma 1.1 7B ITgemma-1-1-7b-it-T-one· ChemBench verified leaderboard data docs0.372
49Llama 2 13B Chatllama2-13b-chat· ChemBench verified leaderboard data docs0.153
50Llama 2 70B Chatllama2-70b-chat· ChemBench verified leaderboard data docs0.147

Showing 1–50 of 51 results.

Data credits: ChemBench leaderboard, Jablonka group / lamalab-org. Public verified numeric result summaries; results licence not separately stated.