Skip to content

Benchmark

Benchmark Nusantara jv-B arti-kata (prompt arti-kata-v1, scorer kunci-alias-v1, items a7ba9fa24e64, key 873b6f168502, withheld slice)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
17
Scores recorded
17
Sources
1
IQ weight
No weight published

Score distribution

0.547
Mean
0.656
Median
0.275
Std dev
0.141
10th percentile
0.828
90th percentile
0.0938
Min
0.09380.906
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

ModelScore
Claude Haiku 4.5repetition: 1, route anthropic/claude-haiku-4.5, token cap 600· Benchmark Nusantara (ibahasa) data docs0.641
Claude Haiku 5.5repetition: 1, route anthropic/claude-haiku-5.5, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.797
Claude Sonnet 5.5repetition: 1, route anthropic/claude-sonnet-5.5, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.844
DeepSeek V4 Flash 0731repetition: 1, route deepseek/deepseek-v4-flash-0731, token cap 3000· Benchmark Nusantara (ibahasa) data docs0.766
Gemini 3.5 Flashrepetition: 1, route google/gemini-3.5-flash, token cap 600· Benchmark Nusantara (ibahasa) data docs0.906
Gemma 4 31B ITrepetition: 1, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs0.797
GLM 5.3repetition: 1, route z-ai/glm-5.3, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.656
GLM 5.3 Flashrepetition: 1, route z-ai/glm-5.3-flash, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.688
GPT-5.6 Lunarepetition: 1, route openai/gpt-5.6-luna, token cap 600· Benchmark Nusantara (ibahasa) data docs0.828
Ling 3.0 Flashrepetition: 1, route inclusionai/ling-3.0-flash, token cap 600· Benchmark Nusantara (ibahasa) data docs0.422
Llama 3.1 8B Instructrepetition: 1, local Q4_K_M, token cap 600· Benchmark Nusantara (ibahasa) data docs0.172
MiMo V2.6 Flashrepetition: 1, route xiaomi/mimo-v2.6-flash, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.672
Mistral Small 24B Instructrepetition: 1, route mistralai/mistral-small-24b-instruct-2501, token cap 600, 2501· Benchmark Nusantara (ibahasa) data docs0.0938
Nemotron 3.5 Lightningrepetition: 1, route nvidia/nemotron-3.5-lightning, token cap 3000· Benchmark Nusantara (ibahasa) data docs0.141
Qwen3 30B A3B Instructrepetition: 1, route qwen/qwen3-30b-a3b-instruct-2507, token cap 600· Benchmark Nusantara (ibahasa) data docs0.359
Seed 2.0 Minirepetition: 1, route bytedance-seed/seed-2.0-mini, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.141
Solar Pro 4repetition: 1, route upstage/solar-pro4, token cap 600· Benchmark Nusantara (ibahasa) data docs0.375

Data credits: Benchmark Nusantara, ibahasa (M. Khoirul Huda), doi:10.5281/zenodo.21970463. Test results CC BY 4.0, attribution to ibahasa; aggregate results CSV only. (licence)