Skip to content

Benchmark

pfgen-bench Japanese generation: Truthfulness

Higher scores are better.

Models measured
101
Scores recorded
156
Sources
1
IQ weight
1.36

Score distribution

0.7
Mean
0.734
Median
0.215
Std dev
0.392
10th percentile
0.959
90th percentile
0.001
Min
0.0010.988
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.36 for chat models, measured over 101 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1MiniMax Text 01qa/chat/74d4af0· pfgen-bench (Preferred Networks) data docs0.988
2MiniMax Text 01qa/chat/05c1dce· pfgen-bench (Preferred Networks) data docs0.986
3Gemini 2.0 Flash Experimentalqa/chat/c246287, thinking· pfgen-bench (Preferred Networks) data docs0.984
4GPT 4oqa/chat/5e1e9f4· pfgen-bench (Preferred Networks) data docs0.98
5Gemini 2.0 Flash Experimentalqa/chat/5f7c6a9· pfgen-bench (Preferred Networks) data docs0.978
6gemini-1.5-pro-002qa/chat/873c5b2· pfgen-bench (Preferred Networks) data docs0.976
7ChatGPT-4o Latestqa/chat/84cddfa· pfgen-bench (Preferred Networks) data docs0.971
8DeepSeek V3completion/4a38349· pfgen-bench (Preferred Networks) data docs0.969
9ChatGPT-4o Latestqa/chat/c275a7b· pfgen-bench (Preferred Networks) data docs0.968
10Claude Opus 4qa/chat/2dafa10· pfgen-bench (Preferred Networks) data docs0.968
11Qwen Maxqa/chat/fa9558d· pfgen-bench (Preferred Networks) data docs0.968
12Claude Opus 4qa/chat/d33465d· pfgen-bench (Preferred Networks) data docs0.964
13Qwen Maxqa/chat/0944f44· pfgen-bench (Preferred Networks) data docs0.964
14ChatGPT-4o Latestqa/chat/f029db6· pfgen-bench (Preferred Networks) data docs0.963
15gemini-1.5-flash-002qa/chat/b868ea0· pfgen-bench (Preferred Networks) data docs0.96
16Claude 3.5 Sonnetqa/chat/2938dda· pfgen-bench (Preferred Networks) data docs0.959
17GPT-4 Turboqa/chat/e96542c· pfgen-bench (Preferred Networks) data docs0.959
18Qwen 3 235B A22Bcompletion/89946a4· pfgen-bench (Preferred Networks) data docs0.952
19GPT 4qa/chat/8cad1b9· pfgen-bench (Preferred Networks) data docs0.951
20Claude 3 Opusqa/chat/89cdaa9· pfgen-bench (Preferred Networks) data docs0.944
21GPT-3.5 Turboqa/chat/eaeade4· pfgen-bench (Preferred Networks) data docs0.944
22Claude 3 Haikuqa/chat/9e7a056· pfgen-bench (Preferred Networks) data docs0.943
23Qwen2.5 72B Instructqa/chat/5705734· pfgen-bench (Preferred Networks) data docs0.939
24Qwen2.5 72B Instructqa/chat/5e498db· pfgen-bench (Preferred Networks) data docs0.936
25DeepSeek R1qa/chat/5abaa36· pfgen-bench (Preferred Networks) data docs0.929
26Claude 3 Sonnetqa/chat/f70c14d· pfgen-bench (Preferred Networks) data docs0.91
27Nemotron-4 340B Instructqa/chat/0d4b478· pfgen-bench (Preferred Networks) data docs0.91
28Nemotron-4 340B Instructcompletion/e99a1d3· pfgen-bench (Preferred Networks) data docs0.908
29Qwen2.5 32B Instructqa/chat/8a041a4· pfgen-bench (Preferred Networks) data docs0.906
30Mistral Large Instruct 2411qa/chat/c6a3714· pfgen-bench (Preferred Networks) data docs0.902
31Mistral Large Instruct 2411completion/b2f50ab· pfgen-bench (Preferred Networks) data docs0.9
32Mistral Large Instruct 2411qa/chat/564af5f· pfgen-bench (Preferred Networks) data docs0.899
33Qwen2.5 32B Instructqa/chat/01f44c1· pfgen-bench (Preferred Networks) data docs0.898
34QwQ 32B Previewqa/chat/a362164· pfgen-bench (Preferred Networks) data docs0.897
35Qwen 2.5 14B Instructqa/chat/54fea07· pfgen-bench (Preferred Networks) data docs0.893
36Llama 3.1 405Bcompletion/d0c2319· pfgen-bench (Preferred Networks) data docs0.892
37Qwen 2.5 14B Instructqa/chat/4a65a55· pfgen-bench (Preferred Networks) data docs0.889
38Gemma 2 9B ITqa/chat/37ba4f6· pfgen-bench (Preferred Networks) data docs0.88
39Qwen3 8Bqa/chat/667d843· pfgen-bench (Preferred Networks) data docs0.878
40Qwen3 32Bcompletion/9f132a0· pfgen-bench (Preferred Networks) data docs0.876
41Gemma 2 27B ITqa/chat/3329a88· pfgen-bench (Preferred Networks) data docs0.874
42QwQ 32B Previewqa/chat/a240792· pfgen-bench (Preferred Networks) data docs0.867
43Phi 4completion/118d26c· pfgen-bench (Preferred Networks) data docs0.864
44Qwen2.5 72Bcompletion/8c7477d· pfgen-bench (Preferred Networks) data docs0.863
45Qwen2.5 7B Instructqa/chat/20023d7· pfgen-bench (Preferred Networks) data docs0.858
46Qwen2 72B Instructqa/chat/baf6c82· pfgen-bench (Preferred Networks) data docs0.853
47Qwen2.5 7B Instructqa/chat/81e4cf3· pfgen-bench (Preferred Networks) data docs0.847
48Phi 4qa/chat/772e05d· pfgen-bench (Preferred Networks) data docs0.846
49Phi 4qa/chat/af856d5· pfgen-bench (Preferred Networks) data docs0.843
50Qwen3 8Bcompletion/9d7e974· pfgen-bench (Preferred Networks) data docs0.842

Showing 1–50 of 156 results.

Data credits: pfgen-bench, Preferred Networks. Apache-2.0 except LLM-generated outputs; aggregate scores only. (licence)