Skip to content

Benchmark

pfgen-bench Japanese generation: Fluency

Higher scores are better.

Models measured
101
Scores recorded
156
Sources
1
IQ weight
1.56

Score distribution

0.573
Mean
0.557
Median
0.195
Std dev
0.351
10th percentile
0.839
90th percentile
0.001
Min
0.0011
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.56 for chat models, measured over 101 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
101Qwen3 1.7B Basecompletion/26f4649· pfgen-bench (Preferred Networks) data docs0.503
102Phi-3 Small 8K Instructqa/chat/9a69f41· pfgen-bench (Preferred Networks) data docs0.5
103Qwen1.5-14B-Chatqa/chat/577f2a5· pfgen-bench (Preferred Networks) data docs0.5
104Qwen1.5-32B-Chatqa/chat/9df40bc· pfgen-bench (Preferred Networks) data docs0.496
105Qwen1.5-14B-Chatcompletion/043470d· pfgen-bench (Preferred Networks) data docs0.491
106Mixtral 8x7B Instruct v0.1completion/024664d· pfgen-bench (Preferred Networks) data docs0.488
107Llama 3.1 8B Instructqa/chat/e3372e7· pfgen-bench (Preferred Networks) data docs0.487
108gemma2-2bcompletion/4174ff9· pfgen-bench (Preferred Networks) data docs0.486
109Mistral Nemoqa/chat/ab861fe· pfgen-bench (Preferred Networks) data docs0.484
110Llama 3.1 8B Instructcompletion/0db64de· pfgen-bench (Preferred Networks) data docs0.478
111Gemma 2 9B ITqa/chat/37ba4f6· pfgen-bench (Preferred Networks) data docs0.477
112Gemma 1.1 7B ITqa/chat/7cedd4e· pfgen-bench (Preferred Networks) data docs0.475
113Llama-2-7b-chat-hfcompletion/d7e19ff· pfgen-bench (Preferred Networks) data docs0.464
114gemma2-27bcompletion/f9b6424· pfgen-bench (Preferred Networks) data docs0.45
115Mixtral 8X7b V0.1completion/077c688· pfgen-bench (Preferred Networks) data docs0.45
116Gemma 7B ITqa/chat/a29928b· pfgen-bench (Preferred Networks) data docs0.448
117Mixtral 8x7B Instruct v0.1qa/chat/00e933e· pfgen-bench (Preferred Networks) data docs0.448
118Gemma 7B ITcompletion/a1c5bf9· pfgen-bench (Preferred Networks) data docs0.446
119Gemma 2 2B ITqa/chat/3d0cfea· pfgen-bench (Preferred Networks) data docs0.445
120Qwen1.5-7Bcompletion/e51e217· pfgen-bench (Preferred Networks) data docs0.445
121Qwen1.5-7B-Chatcompletion/b75f3a7· pfgen-bench (Preferred Networks) data docs0.431
122Qwen2.5 1.5Bcompletion/35940e1· pfgen-bench (Preferred Networks) data docs0.431
123Qwen1.5-7B-Chatqa/chat/f4731ca· pfgen-bench (Preferred Networks) data docs0.429
124Qwen 2 (1.5B)completion/a177c05· pfgen-bench (Preferred Networks) data docs0.422
125Qwen2 1.5b Instructcompletion/b8f3248· pfgen-bench (Preferred Networks) data docs0.421
126gemma-2bcompletion/078c978· pfgen-bench (Preferred Networks) data docs0.41
127Qwen2 1.5b Instructqa/chat/2419c89· pfgen-bench (Preferred Networks) data docs0.408
128Qwen3 0.6B Basecompletion/f504bef· pfgen-bench (Preferred Networks) data docs0.406
129Gemma 1.1 2B ITcompletion/931162e· pfgen-bench (Preferred Networks) data docs0.404
130Llama-2-13b-chat-hfqa/chat/7db3434· pfgen-bench (Preferred Networks) data docs0.402
131Mistral 7B v0.1completion/7f376df· pfgen-bench (Preferred Networks) data docs0.402
132Qwen1.5-4B-Chatcompletion/843369c· pfgen-bench (Preferred Networks) data docs0.395
133Gemma 2B ITcompletion/652b56f· pfgen-bench (Preferred Networks) data docs0.393
134Mistral 7B Instruct v0.2completion/1aa2ad7· pfgen-bench (Preferred Networks) data docs0.387
135Qwen1.5-1.8B-Chatcompletion/c0f1d5a· pfgen-bench (Preferred Networks) data docs0.381
136Mistral (7B) Instruct v0.1completion/01c558f· pfgen-bench (Preferred Networks) data docs0.367
137Qwen1.5-4Bcompletion/681d1a9· pfgen-bench (Preferred Networks) data docs0.363
138Qwen1.5-1.8Bcompletion/7343413· pfgen-bench (Preferred Networks) data docs0.36
139DeepSeek LLM 67B Chatcompletion/d0ebf05· pfgen-bench (Preferred Networks) data docs0.351
140qwen2-0.5b-instructcompletion/24880e5· pfgen-bench (Preferred Networks) data docs0.351
141Qwen1.5-1.8B-Chatqa/chat/a452608· pfgen-bench (Preferred Networks) data docs0.329
142Mistral (7B) Instruct v0.1qa/chat/1be1e40· pfgen-bench (Preferred Networks) data docs0.32
143Qwen1.5-0.5B-Chatcompletion/4f5c753· pfgen-bench (Preferred Networks) data docs0.317
144Mistral 7B Instruct v0.2qa/chat/a690ecc· pfgen-bench (Preferred Networks) data docs0.316
145Qwen1.5-4B-Chatqa/chat/49c9c69· pfgen-bench (Preferred Networks) data docs0.307
146Qwen1.5-0.5Bcompletion/0e72cbc· pfgen-bench (Preferred Networks) data docs0.296
147phi-2completion/c8c91cf· pfgen-bench (Preferred Networks) data docs0.287
148Llama-2-70b-chat-hfqa/chat/fddf689· pfgen-bench (Preferred Networks) data docs0.271
149qwen2-0.5b-instructqa/chat/34dccbe· pfgen-bench (Preferred Networks) data docs0.265
150Qwen1.5-0.5B-Chatqa/chat/822ff8d· pfgen-bench (Preferred Networks) data docs0.251

Showing 101–150 of 156 results.

Data credits: pfgen-bench, Preferred Networks. Apache-2.0 except LLM-generated outputs; aggregate scores only. (licence)