Skip to content

Benchmark

Benchmark Nusantara jv-B arti-kata (prompt arti-kata-v1, scorer kunci-alias-v1, items 4e682041d284, key 873b6f168502, public slice)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
11
Scores recorded
17
Sources
1
IQ weight
0.70

Score distribution

0.646
Mean
0.815
Median
0.274
Std dev
0.193
10th percentile
0.882
90th percentile
0.156
Min
0.1560.941
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.70 for chat models, measured over 7 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-5.6 Lunarepetition: 2, route openai/gpt-5.6-luna, token cap 600· Benchmark Nusantara (ibahasa) data docs0.896
2GPT-5.6 Lunarepetition: 3, route openai/gpt-5.6-luna, token cap 600· Benchmark Nusantara (ibahasa) data docs0.882
3Gemma 4 31B ITrepetition: 3, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs0.859
4Gemma 4 31B ITrepetition: 2, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs0.852
5DeepSeek V4 Flash 0731repetition: 2, route deepseek/deepseek-v4-flash-0731, token cap 4000· Benchmark Nusantara (ibahasa) data docs0.815
6DeepSeek V4 Flash 0731repetition: 3, route deepseek/deepseek-v4-flash-0731, token cap 4000· Benchmark Nusantara (ibahasa) data docs0.815
7Ling 3.0 Flashrepetition: 1, route inclusionai/ling-3.0-flash, token cap 1000· Benchmark Nusantara (ibahasa) data docs0.511
8Solar Pro 4repetition: 1, route upstage/solar-pro4, token cap 600· Benchmark Nusantara (ibahasa) data docs0.385
9Qwen3 30B A3B Instructrepetition: 1, route qwen/qwen3-30b-a3b-instruct-2507, token cap 1000· Benchmark Nusantara (ibahasa) data docs0.37
10Nemotron 3.5 Lightningrepetition: 1, route nvidia/nemotron-3.5-lightning, token cap 8000· Benchmark Nusantara (ibahasa) data docs0.193

Not ranked

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

ModelScore
Claude Haiku 4.5repetition: 1, route anthropic/claude-haiku-4.5, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs0.607
DeepSeek V4 Flash 0731repetition: 1, route deepseek/deepseek-v4-flash-0731, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs0.815
Gemini 3.5 Flashrepetition: 1, route google/gemini-3.5-flash, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs0.941
Gemma 4 31B ITrepetition: 1, route google/gemma-4-31b-it, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs0.837
GPT-5.6 Lunarepetition: 1, route openai/gpt-5.6-luna, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs0.859
Llama 3.1 8B Instructrepetition: 1, local Q4_K_M, token cap 600· Benchmark Nusantara (ibahasa) data docs0.193
Mistral Small 24B Instructrepetition: 1, route mistralai/mistral-small-24b-instruct-2501, token cap unrecorded, 2501· Benchmark Nusantara (ibahasa) data docs0.156

Data credits: Benchmark Nusantara, ibahasa (M. Khoirul Huda), doi:10.5281/zenodo.21970463. Test results CC BY 4.0, attribution to ibahasa; aggregate results CSV only. (licence)