Skip to content

Benchmark

PIQA

Scores are percentages.

Models measured
24
Scores recorded
24
Sources
2
IQ weight
1.25

Score distribution

80.6%
Mean
82.2%
Median
7.3%
Std dev
69.6%
10th percentile
88.6%
90th percentile
55.2%
Min
55.2%88.7%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.25 for chat models, measured over 24 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-4o mini· Epoch AI Benchmarking Hub data88.7%
2phi-3.5-moe-instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
88.6%
3gemini-1.5-flash-002· Epoch AI Benchmarking Hub data87.5%
4Llama 3.1 405B· Epoch AI Benchmarking Hub data85.9%
5gemma-2-9b· Epoch AI Benchmarking Hub data83.7%
6Mixtral 8X7b V0.1· Epoch AI Benchmarking Hub data83.6%
7mistral-nemo-base-2407· Epoch AI Benchmarking Hub data83.5%
8Gemma 2 27B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim83.2%
9Mistral 7B v0.1· Epoch AI Benchmarking Hub data83%
10Qwen2.5 72B· Epoch AI Benchmarking Hub data82.6%
11text-davinci-001· Epoch AI Benchmarking Hub data82.3%
12Mistral (7B) Instruct v0.1· Epoch AI Benchmarking Hub data82.2%
13Mistral 7B Instruct v0.2· Epoch AI Benchmarking Hub data82.2%
14Gemma 2 9B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim81.7%
15gemma-7b· Epoch AI Benchmarking Hub data81.2%
16Llama 3.1 8B Instruct· Epoch AI Benchmarking Hub data81.2%
17Llama 2 13b· Epoch AI Benchmarking Hub data80.8%
18LLaMA 13B· Epoch AI Benchmarking Hub data80.1%
19Llama 2 7b· Epoch AI Benchmarking Hub data78.8%
20Phi 4 Mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim77.6%
21gemma-2b· Epoch AI Benchmarking Hub data77.3%
22ChatGLM2 6B· Epoch AI Benchmarking Hub data69.6%
23StableLM Tuned Alpha 7B· Epoch AI Benchmarking Hub data65.8%
24ERNIE 4.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim55.2%

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · Data by LLM Stats