Skip to content

Benchmark

pfgen-bench Japanese generation: Score

Higher scores are better.

Models measured
101
Scores recorded
156
Sources
1
IQ weight
1.59

Score distribution

0.498
Mean
0.483
Median
0.202
Std dev
0.256
10th percentile
0.804
90th percentile
0.0006
Min
0.00060.958
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.59 for chat models, measured over 101 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
101Qwen3 1.7B Basecompletion/26f4649· pfgen-bench (Preferred Networks) data docs0.406
102mixtral-8x22b-v0.1completion/40ef626· pfgen-bench (Preferred Networks) data docs0.405
103Meta Llama 3 8bcompletion/0f9faf5· pfgen-bench (Preferred Networks) data docs0.405
104Llama 3 8B Instructqa/chat/3d447ac· pfgen-bench (Preferred Networks) data docs0.404
105Phi-3 Small 8K Instructqa/chat/9a69f41· pfgen-bench (Preferred Networks) data docs0.401
106Llama 3.1 8B Instructqa/chat/e3372e7· pfgen-bench (Preferred Networks) data docs0.395
107Qwen1.5-14B-Chatqa/chat/577f2a5· pfgen-bench (Preferred Networks) data docs0.395
108Qwen1.5-14Bcompletion/4e006c4· pfgen-bench (Preferred Networks) data docs0.395
109Mixtral 8x7B Instruct v0.1completion/024664d· pfgen-bench (Preferred Networks) data docs0.391
110Qwen1.5-14B-Chatcompletion/043470d· pfgen-bench (Preferred Networks) data docs0.386
111gemma2-2bcompletion/4174ff9· pfgen-bench (Preferred Networks) data docs0.383
112Mistral Nemoqa/chat/ab861fe· pfgen-bench (Preferred Networks) data docs0.382
113gemma2-27bcompletion/f9b6424· pfgen-bench (Preferred Networks) data docs0.382
114Llama 3.1 8B Instructcompletion/0db64de· pfgen-bench (Preferred Networks) data docs0.374
115Gemma 7B ITcompletion/a1c5bf9· pfgen-bench (Preferred Networks) data docs0.369
116Gemma 7B ITqa/chat/a29928b· pfgen-bench (Preferred Networks) data docs0.367
117Llama-2-7b-chat-hfcompletion/d7e19ff· pfgen-bench (Preferred Networks) data docs0.355
118Mixtral 8X7b V0.1completion/077c688· pfgen-bench (Preferred Networks) data docs0.353
119Gemma 1.1 2B ITcompletion/931162e· pfgen-bench (Preferred Networks) data docs0.351
120Mixtral 8x7B Instruct v0.1qa/chat/00e933e· pfgen-bench (Preferred Networks) data docs0.346
121Qwen1.5-7B-Chatcompletion/b75f3a7· pfgen-bench (Preferred Networks) data docs0.332
122Qwen2.5 1.5Bcompletion/35940e1· pfgen-bench (Preferred Networks) data docs0.332
123Qwen1.5-7Bcompletion/e51e217· pfgen-bench (Preferred Networks) data docs0.332
124Gemma 2B ITcompletion/652b56f· pfgen-bench (Preferred Networks) data docs0.331
125Qwen1.5-7B-Chatqa/chat/f4731ca· pfgen-bench (Preferred Networks) data docs0.329
126Qwen2 1.5b Instructcompletion/b8f3248· pfgen-bench (Preferred Networks) data docs0.319
127Qwen 2 (1.5B)completion/a177c05· pfgen-bench (Preferred Networks) data docs0.317
128Qwen2 1.5b Instructqa/chat/2419c89· pfgen-bench (Preferred Networks) data docs0.315
129Qwen3 0.6B Basecompletion/f504bef· pfgen-bench (Preferred Networks) data docs0.307
130gemma-2bcompletion/078c978· pfgen-bench (Preferred Networks) data docs0.305
131Llama-2-13b-chat-hfqa/chat/7db3434· pfgen-bench (Preferred Networks) data docs0.296
132Mistral 7B v0.1completion/7f376df· pfgen-bench (Preferred Networks) data docs0.291
133Qwen1.5-4B-Chatcompletion/843369c· pfgen-bench (Preferred Networks) data docs0.285
134Qwen1.5-1.8B-Chatcompletion/c0f1d5a· pfgen-bench (Preferred Networks) data docs0.281
135Mistral 7B Instruct v0.2completion/1aa2ad7· pfgen-bench (Preferred Networks) data docs0.277
136DeepSeek LLM 67B Chatcompletion/d0ebf05· pfgen-bench (Preferred Networks) data docs0.267
137Qwen1.5-1.8Bcompletion/7343413· pfgen-bench (Preferred Networks) data docs0.266
138qwen2-0.5b-instructcompletion/24880e5· pfgen-bench (Preferred Networks) data docs0.261
139Mistral (7B) Instruct v0.1completion/01c558f· pfgen-bench (Preferred Networks) data docs0.26
140Qwen1.5-4Bcompletion/681d1a9· pfgen-bench (Preferred Networks) data docs0.256
141Qwen1.5-1.8B-Chatqa/chat/a452608· pfgen-bench (Preferred Networks) data docs0.241
142Qwen1.5-4B-Chatqa/chat/49c9c69· pfgen-bench (Preferred Networks) data docs0.239
143Mistral (7B) Instruct v0.1qa/chat/1be1e40· pfgen-bench (Preferred Networks) data docs0.232
144Mistral 7B Instruct v0.2qa/chat/a690ecc· pfgen-bench (Preferred Networks) data docs0.223
145Qwen1.5-0.5B-Chatcompletion/4f5c753· pfgen-bench (Preferred Networks) data docs0.218
146qwen2-0.5b-instructqa/chat/34dccbe· pfgen-bench (Preferred Networks) data docs0.217
147phi-2completion/c8c91cf· pfgen-bench (Preferred Networks) data docs0.215
148Llama-2-70b-chat-hfqa/chat/fddf689· pfgen-bench (Preferred Networks) data docs0.206
149Qwen1.5-0.5Bcompletion/0e72cbc· pfgen-bench (Preferred Networks) data docs0.204
150Gemma 1.1 2B ITqa/chat/ef0d7c6· pfgen-bench (Preferred Networks) data docs0.183

Showing 101–150 of 156 results.

Data credits: pfgen-bench, Preferred Networks. Apache-2.0 except LLM-generated outputs; aggregate scores only. (licence)