Skip to content

Benchmark

ChemBench Inorganic Chemistry

Science benchmarks measure how well models answer science questions. Higher scores are better.

Models measured
42
Scores recorded
51
Sources
1
IQ weight
1.56

Score distribution

0.677
Mean
0.696
Median
0.202
Std dev
0.467
10th percentile
0.902
90th percentile
0
Min
00.957
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.56 for chat models, measured over 42 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.7claude-opus-4-7-tinyeval· ChemBench verified leaderboard data docs0.957
2DeepSeek V3.2deepseek-v3-2-tinyeval· ChemBench verified leaderboard data docs0.924
3DeepSeek R1deepseek-r1-tinyeval· ChemBench verified leaderboard data docs0.913
4Kimi K2.5kimi-k2-5-tinyeval· ChemBench verified leaderboard data docs0.913
5Claude Opus 4.8claude-opus-4-8-tinyeval· ChemBench verified leaderboard data docs0.902
6O1 Previewo1-preview· ChemBench verified leaderboard data docs0.902
7Claude Opus 4.5claude-opus-4-5-tinyeval· ChemBench verified leaderboard data docs0.891
8GPT OSS 120Bgpt-oss-120b-base-1760293341· ChemBench verified leaderboard data docs0.891
9GPT OSS 20bgpt-oss-20b-base-1760359009· ChemBench verified leaderboard data docs0.87
10Qwen3 235B A22B Instruct 2507qwen3-235b-a22b-2507-tinyeval· ChemBench verified leaderboard data docs0.87
11GPT-5 nanoGPT-5-nano-base-1764459351· ChemBench verified leaderboard data docs0.859
12Qwen3 Next 80B A3Bqwen3-next-80b-a3b-tinyeval· ChemBench verified leaderboard data docs0.848
13Claude 3.5 Sonnetclaude3.5· ChemBench verified leaderboard data docs0.837
14gemini-2.5-flash-preview-04-17gemini-2.5-flash-preview-04-17-base-1750491557· ChemBench verified leaderboard data docs0.837
15MiniMax M2.5minimax-m2-5-tinyeval· ChemBench verified leaderboard data docs0.837
16Llama 4 Maverickgroq_meta-llama/llama-4-maverick-17b-128e-instruct_e8a077d8· ChemBench verified leaderboard data docs0.826
17Claude 3.5 Sonnetclaude3.5-react· ChemBench verified leaderboard data docs0.804
18GPT 4ogpt-4o· ChemBench verified leaderboard data docs0.804
19Mistral Large 2mistral-large-2-123b· ChemBench verified leaderboard data docs0.793
20Nemotron Super 3 120bnemotron-super-3-120b-tinyeval· ChemBench verified leaderboard data docs0.793
21Kimi K2 Instructkimi-k2-instruct-0905-base-1760364223· ChemBench verified leaderboard data docs0.772
22Llama 3.1 405B Instructllama3.1-405b-instruct· ChemBench verified leaderboard data docs0.772
23Llama 3.1 70B Instructllama3.1-70b-instruct· ChemBench verified leaderboard data docs0.772
24GLM 5glm-5-tinyeval· ChemBench verified leaderboard data docs0.761
25GPT 4ogpt-4o-react· ChemBench verified leaderboard data docs0.728
26GPT 4gpt-4· ChemBench verified leaderboard data docs0.696
27Llama 3.1 70B Instructllama3.1-70b-instruct-T-one· ChemBench verified leaderboard data docs0.696
28GPT-4o miniGPT-4o-mini-base-1764283510· ChemBench verified leaderboard data docs0.685
29Llama 3 70B Instructllama3-70b-instruct-T-one· ChemBench verified leaderboard data docs0.685
30Qwen2.5 32BQwen-2.5-32B-base-1740920072· ChemBench verified leaderboard data docs0.685
31Llama 3 70B Instructllama3-70b-instruct· ChemBench verified leaderboard data docs0.663
32Phi-3 Medium 4K Instructphi-3-medium-4k-instruct· ChemBench verified leaderboard data docs0.63
33Claude 2claude2· ChemBench verified leaderboard data docs0.609
34Gemma 2 9B ITgemma-2-9b-it· ChemBench verified leaderboard data docs0.554
35Mixtral 8x7B Instructmixtral-8x7b-instruct· ChemBench verified leaderboard data docs0.554
36Claude 3 Haikuclaude-3-haiku-tinyeval· ChemBench verified leaderboard data docs0.543
37Gemma 2 9B ITgemma-2-9b-it-T-one· ChemBench verified leaderboard data docs0.543
38GPT-3.5 Turbogpt-3.5-turbo· ChemBench verified leaderboard data docs0.543
39Llama 3 8B Instructllama3-8b-instruct-T-one· ChemBench verified leaderboard data docs0.543
40Nemotron Nano 3 30Bnemotron-nano-3-30b-tinyeval· ChemBench verified leaderboard data docs0.533
41Command R Pluscommand-r+· ChemBench verified leaderboard data docs0.522
42Mixtral 8x7B Instructmixtral-8x7b-instruct-T-one· ChemBench verified leaderboard data docs0.511
43Llama 3.1 8B Instructllama3.1-8b-instruct· ChemBench verified leaderboard data docs0.5
44Llama 3 8B Instructllama3-8b-instruct· ChemBench verified leaderboard data docs0.489
45Llama 3.1 8B Instructllama3.1-8b-instruct-T-one· ChemBench verified leaderboard data docs0.478
46Gemini Progemini-pro· ChemBench verified leaderboard data docs0.467
47Gemma 1.1 7B ITgemma-1-1-7b-it· ChemBench verified leaderboard data docs0.413
48Gemma 1.1 7B ITgemma-1-1-7b-it-T-one· ChemBench verified leaderboard data docs0.413
49Llama 2 13B Chatllama2-13b-chat· ChemBench verified leaderboard data docs0.272
50Llama 2 70B Chatllama2-70b-chat· ChemBench verified leaderboard data docs0.217

Showing 1–50 of 51 results.

Data credits: ChemBench leaderboard, Jablonka group / lamalab-org. Public verified numeric result summaries; results licence not separately stated.