Skip to content

Benchmark

Benchmark Nusantara jv-arti-kata arti-kata (prompt arti-kata-v1, scorer kunci-alias-v1, items 6dfcc448bbd5, key 0a1bc07f67bc, public slice)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
15
Scores recorded
15
Sources
1
IQ weight
No weight published

Score distribution

0.44
Mean
0.512
Median
0.296
Std dev
0.023
10th percentile
0.814
90th percentile
0.0233
Min

Leaderboard

#ModelScore
1Gemma 4 31B ITrepetition: 1, route google/gemma-4-31b-it, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs0.605
2Solar Pro 4repetition: 1, route upstage/solar-pro4, token cap 600· Benchmark Nusantara (ibahasa) data docs0.093
3Nemotron 3.5 Lightningrepetition: 1, route nvidia/nemotron-3.5-lightning, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs0.0698

Not ranked

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

ModelScore
Claude Haiku 5.5repetition: 1, route anthropic/claude-haiku-5.5, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.535
Claude Sonnet 5.5repetition: 1, route anthropic/claude-sonnet-5.5, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.791
DeepSeek V4 Flash 0731repetition: 1, route deepseek/deepseek-v4-flash-0731, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs0.512
DeepSeek V4.1 Flashrepetition: 1, route deepseek/deepseek-v4.1-flash, token cap 9600· Benchmark Nusantara (ibahasa) data docs0.512
Gemini 3.5 Flashrepetition: 1, route google/gemini-3.5-flash, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs0.93
GLM 5.3repetition: 1, route z-ai/glm-5.3, token cap 600· Benchmark Nusantara (ibahasa) data docs0.326
GLM 5.3 Flashrepetition: 1, route z-ai/glm-5.3-flash, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.256
GPT-5.6 Lunarepetition: 1, route openai/gpt-5.6-luna, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs0.814
GPT-6 Astrarepetition: 1, route openai/gpt-6-astra, token cap 600· Benchmark Nusantara (ibahasa) data docs0.767
Llama 3.1 8B Instructrepetition: 1, local Q4_K_M, token cap 600· Benchmark Nusantara (ibahasa) data docs0.0233
MiMo V2.6 Flashrepetition: 1, route xiaomi/mimo-v2.6-flash, token cap 600· Benchmark Nusantara (ibahasa) data docs0.349
Mistral Small 24B Instructrepetition: 1, route mistralai/mistral-small-24b-instruct-2501, token cap unrecorded, 2501· Benchmark Nusantara (ibahasa) data docs0.0233

Data credits: Benchmark Nusantara, ibahasa (M. Khoirul Huda), doi:10.5281/zenodo.21970463. Test results CC BY 4.0, attribution to ibahasa; aggregate results CSV only. (licence)