Skip to content

Benchmark

ChemBench General Chemistry

Science benchmarks measure how well models answer science questions. Higher scores are better.

Models measured
42
Scores recorded
51
Sources
1
IQ weight
1.28

Score distribution

0.651
Mean
0.691
Median
0.225
Std dev
0.423
10th percentile
0.906
90th percentile
0.0201
Min
0.02010.953
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.28 for chat models, measured over 42 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.7claude-opus-4-7-tinyeval· ChemBench verified leaderboard data docs0.953
2Claude Opus 4.8claude-opus-4-8-tinyeval· ChemBench verified leaderboard data docs0.933
3O1 Previewo1-preview· ChemBench verified leaderboard data docs0.933
4Claude Opus 4.5claude-opus-4-5-tinyeval· ChemBench verified leaderboard data docs0.926
5Kimi K2.5kimi-k2-5-tinyeval· ChemBench verified leaderboard data docs0.913
6DeepSeek V3.2deepseek-v3-2-tinyeval· ChemBench verified leaderboard data docs0.906
7GPT OSS 20bgpt-oss-20b-base-1760359009· ChemBench verified leaderboard data docs0.899
8Llama 4 Maverickgroq_meta-llama/llama-4-maverick-17b-128e-instruct_e8a077d8· ChemBench verified leaderboard data docs0.893
9Claude 3.5 Sonnetclaude3.5-react· ChemBench verified leaderboard data docs0.872
10MiniMax M2.5minimax-m2-5-tinyeval· ChemBench verified leaderboard data docs0.872
11GPT OSS 120Bgpt-oss-120b-base-1760293341· ChemBench verified leaderboard data docs0.859
12gemini-2.5-flash-preview-04-17gemini-2.5-flash-preview-04-17-base-1750491557· ChemBench verified leaderboard data docs0.852
13DeepSeek R1deepseek-r1-tinyeval· ChemBench verified leaderboard data docs0.846
14Kimi K2 Instructkimi-k2-instruct-0905-base-1760364223· ChemBench verified leaderboard data docs0.832
15Claude 3.5 Sonnetclaude3.5· ChemBench verified leaderboard data docs0.826
16Qwen3 Next 80B A3Bqwen3-next-80b-a3b-tinyeval· ChemBench verified leaderboard data docs0.819
17GPT-5 nanoGPT-5-nano-base-1764459351· ChemBench verified leaderboard data docs0.812
18GPT 4ogpt-4o· ChemBench verified leaderboard data docs0.805
19Llama 3.1 405B Instructllama3.1-405b-instruct· ChemBench verified leaderboard data docs0.792
20Mistral Large 2mistral-large-2-123b· ChemBench verified leaderboard data docs0.785
21GLM 5glm-5-tinyeval· ChemBench verified leaderboard data docs0.779
22GPT 4ogpt-4o-react· ChemBench verified leaderboard data docs0.758
23Qwen3 235B A22B Instruct 2507qwen3-235b-a22b-2507-tinyeval· ChemBench verified leaderboard data docs0.758
24Nemotron Super 3 120bnemotron-super-3-120b-tinyeval· ChemBench verified leaderboard data docs0.705
25GPT 4gpt-4· ChemBench verified leaderboard data docs0.698
26Llama 3.1 70B Instructllama3.1-70b-instruct· ChemBench verified leaderboard data docs0.691
27Qwen2.5 32BQwen-2.5-32B-base-1740920072· ChemBench verified leaderboard data docs0.671
28Llama 3.1 70B Instructllama3.1-70b-instruct-T-one· ChemBench verified leaderboard data docs0.664
29Llama 3 70B Instructllama3-70b-instruct· ChemBench verified leaderboard data docs0.604
30Llama 3 70B Instructllama3-70b-instruct-T-one· ChemBench verified leaderboard data docs0.604
31GPT-4o miniGPT-4o-mini-base-1764283510· ChemBench verified leaderboard data docs0.597
32Gemma 2 9B ITgemma-2-9b-it-T-one· ChemBench verified leaderboard data docs0.557
33Nemotron Nano 3 30Bnemotron-nano-3-30b-tinyeval· ChemBench verified leaderboard data docs0.557
34Gemma 2 9B ITgemma-2-9b-it· ChemBench verified leaderboard data docs0.544
35Llama 3.1 8B Instructllama3.1-8b-instruct-T-one· ChemBench verified leaderboard data docs0.53
36Claude 2claude2· ChemBench verified leaderboard data docs0.503
37Claude 3 Haikuclaude-3-haiku-tinyeval· ChemBench verified leaderboard data docs0.503
38Llama 3.1 8B Instructllama3.1-8b-instruct· ChemBench verified leaderboard data docs0.503
39Command R Pluscommand-r+· ChemBench verified leaderboard data docs0.497
40GPT-3.5 Turbogpt-3.5-turbo· ChemBench verified leaderboard data docs0.49
41Gemini Progemini-pro· ChemBench verified leaderboard data docs0.483
42Phi-3 Medium 4K Instructphi-3-medium-4k-instruct· ChemBench verified leaderboard data docs0.477
43Mixtral 8x7B Instructmixtral-8x7b-instruct-T-one· ChemBench verified leaderboard data docs0.45
44Llama 3 8B Instructllama3-8b-instruct· ChemBench verified leaderboard data docs0.443
45Llama 3 8B Instructllama3-8b-instruct-T-one· ChemBench verified leaderboard data docs0.436
46Mixtral 8x7B Instructmixtral-8x7b-instruct· ChemBench verified leaderboard data docs0.423
47Gemma 1.1 7B ITgemma-1-1-7b-it-T-one· ChemBench verified leaderboard data docs0.349
48Gemma 1.1 7B ITgemma-1-1-7b-it· ChemBench verified leaderboard data docs0.336
49Llama 2 70B Chatllama2-70b-chat· ChemBench verified leaderboard data docs0.134
50Llama 2 13B Chatllama2-13b-chat· ChemBench verified leaderboard data docs0.114

Showing 1–50 of 51 results.

Data credits: ChemBench leaderboard, Jablonka group / lamalab-org. Public verified numeric result summaries; results licence not separately stated.