Skip to content

Benchmark

ChemBench Materials Science

Science benchmarks measure how well models answer science questions. Higher scores are better.

Models measured
42
Scores recorded
51
Sources
1
IQ weight
1.27

Score distribution

0.561
Mean
0.602
Median
0.165
Std dev
0.405
10th percentile
0.738
90th percentile
0
Min
00.771
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.27 for chat models, measured over 42 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.7claude-opus-4-7-tinyeval· ChemBench verified leaderboard data docs0.771
2gemini-2.5-flash-preview-04-17gemini-2.5-flash-preview-04-17-base-1750491557· ChemBench verified leaderboard data docs0.75
3GPT 4ogpt-4o· ChemBench verified leaderboard data docs0.75
4Kimi K2 Instructkimi-k2-instruct-0905-base-1760364223· ChemBench verified leaderboard data docs0.75
5GPT OSS 120Bgpt-oss-120b-base-1760293341· ChemBench verified leaderboard data docs0.738
6GPT-5 nanoGPT-5-nano-base-1764459351· ChemBench verified leaderboard data docs0.738
7Kimi K2.5kimi-k2-5-tinyeval· ChemBench verified leaderboard data docs0.735
8GPT OSS 20bgpt-oss-20b-base-1760359009· ChemBench verified leaderboard data docs0.726
9O1 Previewo1-preview· ChemBench verified leaderboard data docs0.726
10DeepSeek R1deepseek-r1-tinyeval· ChemBench verified leaderboard data docs0.723
11Claude 3.5 Sonnetclaude3.5· ChemBench verified leaderboard data docs0.714
12Claude 3.5 Sonnetclaude3.5-react· ChemBench verified leaderboard data docs0.679
13Llama 4 Maverickgroq_meta-llama/llama-4-maverick-17b-128e-instruct_e8a077d8· ChemBench verified leaderboard data docs0.679
14Claude Opus 4.8claude-opus-4-8-tinyeval· ChemBench verified leaderboard data docs0.675
15DeepSeek V3.2deepseek-v3-2-tinyeval· ChemBench verified leaderboard data docs0.675
16Llama 3.1 70B Instructllama3.1-70b-instruct· ChemBench verified leaderboard data docs0.667
17Mistral Large 2mistral-large-2-123b· ChemBench verified leaderboard data docs0.667
18Llama 3.1 405B Instructllama3.1-405b-instruct· ChemBench verified leaderboard data docs0.655
19Llama 3.1 70B Instructllama3.1-70b-instruct-T-one· ChemBench verified leaderboard data docs0.655
20Qwen3 235B A22B Instruct 2507qwen3-235b-a22b-2507-tinyeval· ChemBench verified leaderboard data docs0.639
21Llama 3 70B Instructllama3-70b-instruct· ChemBench verified leaderboard data docs0.631
22Llama 3 70B Instructllama3-70b-instruct-T-one· ChemBench verified leaderboard data docs0.619
23Qwen2.5 32BQwen-2.5-32B-base-1740920072· ChemBench verified leaderboard data docs0.619
24GPT 4gpt-4· ChemBench verified leaderboard data docs0.607
25Claude Opus 4.5claude-opus-4-5-tinyeval· ChemBench verified leaderboard data docs0.602
26Qwen3 Next 80B A3Bqwen3-next-80b-a3b-tinyeval· ChemBench verified leaderboard data docs0.602
27Nemotron Super 3 120bnemotron-super-3-120b-tinyeval· ChemBench verified leaderboard data docs0.59
28GLM 5glm-5-tinyeval· ChemBench verified leaderboard data docs0.578
29Claude 3 Haikuclaude-3-haiku-tinyeval· ChemBench verified leaderboard data docs0.566
30GPT 4ogpt-4o-react· ChemBench verified leaderboard data docs0.56
31Nemotron Nano 3 30Bnemotron-nano-3-30b-tinyeval· ChemBench verified leaderboard data docs0.554
32Phi-3 Medium 4K Instructphi-3-medium-4k-instruct· ChemBench verified leaderboard data docs0.548
33MiniMax M2.5minimax-m2-5-tinyeval· ChemBench verified leaderboard data docs0.53
34Gemma 2 9B ITgemma-2-9b-it· ChemBench verified leaderboard data docs0.524
35Gemini Progemini-pro· ChemBench verified leaderboard data docs0.5
36Gemma 2 9B ITgemma-2-9b-it-T-one· ChemBench verified leaderboard data docs0.5
37GPT-4o miniGPT-4o-mini-base-1764283510· ChemBench verified leaderboard data docs0.488
38GPT-3.5 Turbogpt-3.5-turbo· ChemBench verified leaderboard data docs0.476
39Claude 2claude2· ChemBench verified leaderboard data docs0.464
40Command R Pluscommand-r+· ChemBench verified leaderboard data docs0.464
41Llama 3 8B Instructllama3-8b-instruct-T-one· ChemBench verified leaderboard data docs0.452
42Llama 3 8B Instructllama3-8b-instruct· ChemBench verified leaderboard data docs0.417
43Llama 3.1 8B Instructllama3.1-8b-instruct-T-one· ChemBench verified leaderboard data docs0.417
44Mixtral 8x7B Instructmixtral-8x7b-instruct· ChemBench verified leaderboard data docs0.417
45Llama 3.1 8B Instructllama3.1-8b-instruct· ChemBench verified leaderboard data docs0.405
46Mixtral 8x7B Instructmixtral-8x7b-instruct-T-one· ChemBench verified leaderboard data docs0.405
47Gemma 1.1 7B ITgemma-1-1-7b-it· ChemBench verified leaderboard data docs0.357
48Gemma 1.1 7B ITgemma-1-1-7b-it-T-one· ChemBench verified leaderboard data docs0.357
49Llama 2 70B Chatllama2-70b-chat· ChemBench verified leaderboard data docs0.179
50Llama 2 13B Chatllama2-13b-chat· ChemBench verified leaderboard data docs0.0952

Showing 1–50 of 51 results.

Data credits: ChemBench leaderboard, Jablonka group / lamalab-org. Public verified numeric result summaries; results licence not separately stated.