Skip to content

Benchmark

pfgen-bench Japanese generation: Truthfulness

Higher scores are better.

Models measured
101
Scores recorded
156
Sources
1
IQ weight
1.36

Score distribution

0.7
Mean
0.734
Median
0.215
Std dev
0.392
10th percentile
0.959
90th percentile
0.001
Min
0.0010.988
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.36 for chat models, measured over 101 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51Gemma 2 9B ITcompletion/d29837e· pfgen-bench (Preferred Networks) data docs0.837
52Gemma 2 27B ITcompletion/a3fa605· pfgen-bench (Preferred Networks) data docs0.836
53Qwen 2 (72B)completion/ff6791f· pfgen-bench (Preferred Networks) data docs0.836
54Phi-3 Medium 128K Instructqa/chat/016fa5e· pfgen-bench (Preferred Networks) data docs0.834
55Qwen3 30B A3b Basecompletion/8d6c0f4· pfgen-bench (Preferred Networks) data docs0.833
56Phi-3 Medium 4K Instructcompletion/74cf0cd· pfgen-bench (Preferred Networks) data docs0.826
57Phi-3 Medium 128K Instructcompletion/3c9dc10· pfgen-bench (Preferred Networks) data docs0.825
58Qwen2 72B Instructcompletion/c1ddc4d· pfgen-bench (Preferred Networks) data docs0.825
59Qwen3 14B Basecompletion/3e8d9d4· pfgen-bench (Preferred Networks) data docs0.824
60Llama 3.1 70Bcompletion/19cc458· pfgen-bench (Preferred Networks) data docs0.822
61Qwen2.5 32Bcompletion/b356bba· pfgen-bench (Preferred Networks) data docs0.82
62Meta Llama 3 70bcompletion/a1448c0· pfgen-bench (Preferred Networks) data docs0.809
63Phi-3 Medium 4K Instructqa/chat/d396e2e· pfgen-bench (Preferred Networks) data docs0.807
64Mistral Small 24B Instructcompletion/0b0698f· pfgen-bench (Preferred Networks) data docs0.797
65Mistral Small 24B Instructqa/chat/f9649b1· pfgen-bench (Preferred Networks) data docs0.794
66Qwen2.5 14Bcompletion/174d4f0· pfgen-bench (Preferred Networks) data docs0.79
67Qwen3 8B Basecompletion/57fa665· pfgen-bench (Preferred Networks) data docs0.789
68Gemma 2 2B ITqa/chat/3d0cfea· pfgen-bench (Preferred Networks) data docs0.788
69Llama 3 70B Instructcompletion/8dfb479· pfgen-bench (Preferred Networks) data docs0.78
70Llama 3.1 70B Instructqa/chat/011aba5· pfgen-bench (Preferred Networks) data docs0.779
71Llama 3 70B Instructqa/chat/3c91259· pfgen-bench (Preferred Networks) data docs0.777
72Mistral Small 24B Instructqa/chat/ecbf52c· pfgen-bench (Preferred Networks) data docs0.773
73Llama 3.1 70B Instructcompletion/12b7638· pfgen-bench (Preferred Networks) data docs0.77
74Gemma 2 2B ITcompletion/a5a3763· pfgen-bench (Preferred Networks) data docs0.761
75gemma-2-9bcompletion/161d30c· pfgen-bench (Preferred Networks) data docs0.761
76Gemma 1.1 7B ITqa/chat/7cedd4e· pfgen-bench (Preferred Networks) data docs0.739
77Qwen2 57B A14B Instructcompletion/bafc717· pfgen-bench (Preferred Networks) data docs0.734
78Qwen2 57B A14B Instructqa/chat/d1e4d61· pfgen-bench (Preferred Networks) data docs0.734
79Gemma 1.1 7B ITcompletion/edc5e6e· pfgen-bench (Preferred Networks) data docs0.732
80Qwen-14B-Chatcompletion/bad8c09· pfgen-bench (Preferred Networks) data docs0.718
81Qwen1.5-72B-Chatqa/chat/d1092d0· pfgen-bench (Preferred Networks) data docs0.708
82Qwen2.5 7Bcompletion/4d123a9· pfgen-bench (Preferred Networks) data docs0.707
83Qwen3 4B Basecompletion/af29cd4· pfgen-bench (Preferred Networks) data docs0.706
84Qwen1.5-72Bcompletion/a4094f9· pfgen-bench (Preferred Networks) data docs0.705
85DeepSeek LLM 67B Chatqa/chat/e1b18f0· pfgen-bench (Preferred Networks) data docs0.703
86Qwen1.5-72B-Chatcompletion/088d789· pfgen-bench (Preferred Networks) data docs0.698
87GLM 4 9B 0414completion/77a535b· pfgen-bench (Preferred Networks) data docs0.689
88Qwen2 7B Instructqa/chat/443a1af· pfgen-bench (Preferred Networks) data docs0.688
89Qwen-14B-Chatqa/chat/7fee9c7· pfgen-bench (Preferred Networks) data docs0.675
90Qwen 2 (7B)completion/2fd6882· pfgen-bench (Preferred Networks) data docs0.671
91Qwen2 7B Instructcompletion/880bb2d· pfgen-bench (Preferred Networks) data docs0.67
92Llama 3.1 8Bcompletion/1568d43· pfgen-bench (Preferred Networks) data docs0.65
93Phi-3 Small 8K Instructcompletion/02b97c5· pfgen-bench (Preferred Networks) data docs0.644
94Qwen1.5-32B-Chatcompletion/146f406· pfgen-bench (Preferred Networks) data docs0.642
95Meta Llama 3 8bcompletion/0f9faf5· pfgen-bench (Preferred Networks) data docs0.641
96Gemma 7B ITcompletion/a1c5bf9· pfgen-bench (Preferred Networks) data docs0.64
97Llama 3 8B Instructcompletion/93af24a· pfgen-bench (Preferred Networks) data docs0.637
98Mixtral 8x7B Instruct v0.1completion/024664d· pfgen-bench (Preferred Networks) data docs0.636
99Qwen3 1.7B Basecompletion/26f4649· pfgen-bench (Preferred Networks) data docs0.635
100Gemma 7B ITqa/chat/a29928b· pfgen-bench (Preferred Networks) data docs0.633

Showing 51–100 of 156 results.

Data credits: pfgen-bench, Preferred Networks. Apache-2.0 except LLM-generated outputs; aggregate scores only. (licence)