Skip to content

Benchmark

Benchmark Nusantara jv-A terbuka (prompt open-py1, scorer open-py1, items 9f2d79048c32, key e229d011793d, public slice)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
13
Scores recorded
13
Sources
1
IQ weight
1.69

Score distribution

0.667
Mean
0.833
Median
0.364
Std dev
0.167
10th percentile
1
90th percentile
0
Min
01
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.69 for chat models, measured over 13 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Sonnet 5.5repetition: 1, route anthropic/claude-sonnet-5.5, token cap 2400· Benchmark Nusantara (ibahasa) data docs1
2DeepSeek V4.1 Flashrepetition: 1, route deepseek/deepseek-v4.1-flash, token cap 2400· Benchmark Nusantara (ibahasa) data docs1
3Gemini 3.5 Flashrepetition: 1, route google/gemini-3.5-flash, token cap 600· Benchmark Nusantara (ibahasa) data docs1
4Gemma 4 31B ITrepetition: 1, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs1
5GPT-6 Astrarepetition: 1, route openai/gpt-6-astra, token cap 2400· Benchmark Nusantara (ibahasa) data docs1
6Claude Haiku 5.5repetition: 1, route anthropic/claude-haiku-5.5, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.833
7GLM 5.3 Flashrepetition: 1, route z-ai/glm-5.3-flash, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.833
8MiMo V2.6 Flashrepetition: 1, route xiaomi/mimo-v2.6-flash, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.833
9GLM 5.3repetition: 1, route z-ai/glm-5.3, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.5
10Qwen3 30B A3B Instructrepetition: 1, route qwen/qwen3-30b-a3b-instruct-2507, token cap 600· Benchmark Nusantara (ibahasa) data docs0.333
11Llama 3.1 8B Instructrepetition: 1, local Q4_K_M, token cap 600· Benchmark Nusantara (ibahasa) data docs0.167
12Seed 2.0 Minirepetition: 1, route bytedance-seed/seed-2.0-mini, token cap 8000· Benchmark Nusantara (ibahasa) data docs0.167
13Mistral Small 24B Instructrepetition: 1, route mistralai/mistral-small-24b-instruct-2501, token cap 600, 2501· Benchmark Nusantara (ibahasa) data docs0

Data credits: Benchmark Nusantara, ibahasa (M. Khoirul Huda), doi:10.5281/zenodo.21970463. Test results CC BY 4.0, attribution to ibahasa; aggregate results CSV only. (licence)