Skip to content

Benchmark

TheoremQA

Math benchmarks measure how well models solve maths problems. Scores are percentages.

Models measured
6
Scores recorded
6
Sources
1
IQ weight
2.20

Score distribution

39%
Mean
43.1%
Median
7.1%
Std dev
34%
10th percentile
44.4%
90th percentile
25.3%
Min
25.3%44.4%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.20 for chat models, measured over 6 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Qwen2 72B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim44.4%
2Qwen2.5 32B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim44.1%
3Qwen 2.5 Coder 32B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim43.1%
4Qwen 2.5 14B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim43%
5Qwen2.5 Coder 7B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim34%
6Qwen2 7B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim25.3%

Data credits: Data by LLM Stats