Skip to content

Benchmark

ArmBench-LLM Armenian paraphrase (bleu, 0-shot)

Higher scores are better.

Models measured
27
Scores recorded
28
Sources
1
IQ weight
0.49

Score distribution

16.1
Mean
19
Median
8.1
Std dev
3.8
10th percentile
24.9
90th percentile
0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000081
Min
0.00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008127.8
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.49 for chat models, measured over 27 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 2.5 Flash· ArmBench-LLM Armenian data docs27.8
2GPT 5.4 Nano· ArmBench-LLM Armenian data docs25.9
3GPT-5.4 mini· ArmBench-LLM Armenian data docs25.6
4Gemma 3 12B IT· ArmBench-LLM Armenian data docs24.9
5Claude Sonnet 3.7· ArmBench-LLM Armenian data docs24
6Gemma 4 31B IT· ArmBench-LLM Armenian data docs23.3
7GPT-5.2 Prosetting not named by the source· ArmBench-LLM Armenian data docs21.5
8GPT-5.2 Prosetting not named by the source· ArmBench-LLM Armenian data docs21.5
9GLM 5· ArmBench-LLM Armenian data docs20.9
10GLM 4.7 Flash· ArmBench-LLM Armenian data docs20.3
11Claude Sonnet 4· ArmBench-LLM Armenian data docs19.6
12Grok 4.1 Fast· ArmBench-LLM Armenian data docs19.6
13Grok 4 Fast· ArmBench-LLM Armenian data docs19.1
14Gemma 3 4B IT· ArmBench-LLM Armenian data docs19
15Gemini 3 Pro Preview· ArmBench-LLM Armenian data docs18.7
16Mistral Large 3· ArmBench-LLM Armenian data docs18.6
17Gemma 3 27B IT· ArmBench-LLM Armenian data docs18.1
18Gemini 3 Flash· ArmBench-LLM Armenian data docs17.6
19Gemini 3.1 Pro Preview· ArmBench-LLM Armenian data docs17.2
20Gemini 2.5 Pro· ArmBench-LLM Armenian data docs13.6
21Qwen3 4B Instruct 2507· ArmBench-LLM Armenian data docs9.4
22Ministral 14B· ArmBench-LLM Armenian data docs7
23Mistral Small 3.2 24B Instruct· ArmBench-LLM Armenian data docs7
24Qwen3.5 27B· ArmBench-LLM Armenian data docs4.7
25Qwen3 4B Base· ArmBench-LLM Armenian data docs3.8
26Qwen3.5 35B A3B· ArmBench-LLM Armenian data docs1.5
27Grok 4.20· ArmBench-LLM Armenian data docs1.5
28Qwen3.5 9B· ArmBench-LLM Armenian data docs0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000081

Data credits: ArmBench-LLM by Metric-AI. Public numeric primary task results; no separate results licence asserted.