Benchmark
Benchmark Nusantara jv-B arti-kata (prompt arti-kata-v1, scorer kunci-alias-v1, items 4e682041d284, key 873b6f168502, public slice)
Multilingual benchmarks compare model performance across languages. Higher scores are better.
- Models measured
- 11
- Scores recorded
- 17
- Sources
- 1
- IQ weight
- 0.70
Score distribution
- 0.646
- Mean
- 0.815
- Median
- 0.274
- Std dev
- 0.193
- 10th percentile
- 0.882
- 90th percentile
- 0.156
- Min
This benchmark has a weight of 0.70 for chat models, measured over 7 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | GPT-5.6 Lunarepetition: 2, route openai/gpt-5.6-luna, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.896 |
| 2 | GPT-5.6 Lunarepetition: 3, route openai/gpt-5.6-luna, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.882 |
| 3 | Gemma 4 31B ITrepetition: 3, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.859 |
| 4 | Gemma 4 31B ITrepetition: 2, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.852 |
| 5 | DeepSeek V4 Flash 0731repetition: 2, route deepseek/deepseek-v4-flash-0731, token cap 4000· Benchmark Nusantara (ibahasa) data docs | 0.815 |
| 6 | DeepSeek V4 Flash 0731repetition: 3, route deepseek/deepseek-v4-flash-0731, token cap 4000· Benchmark Nusantara (ibahasa) data docs | 0.815 |
| 7 | Ling 3.0 Flashrepetition: 1, route inclusionai/ling-3.0-flash, token cap 1000· Benchmark Nusantara (ibahasa) data docs | 0.511 |
| 8 | Solar Pro 4repetition: 1, route upstage/solar-pro4, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.385 |
| 9 | Qwen3 30B A3B Instructrepetition: 1, route qwen/qwen3-30b-a3b-instruct-2507, token cap 1000· Benchmark Nusantara (ibahasa) data docs | 0.37 |
| 10 | Nemotron 3.5 Lightningrepetition: 1, route nvidia/nemotron-3.5-lightning, token cap 8000· Benchmark Nusantara (ibahasa) data docs | 0.193 |
Not ranked
The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.
| Model | Score |
|---|---|
| Claude Haiku 4.5repetition: 1, route anthropic/claude-haiku-4.5, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs | 0.607 |
| DeepSeek V4 Flash 0731repetition: 1, route deepseek/deepseek-v4-flash-0731, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs | 0.815 |
| Gemini 3.5 Flashrepetition: 1, route google/gemini-3.5-flash, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs | 0.941 |
| Gemma 4 31B ITrepetition: 1, route google/gemma-4-31b-it, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs | 0.837 |
| GPT-5.6 Lunarepetition: 1, route openai/gpt-5.6-luna, token cap unrecorded· Benchmark Nusantara (ibahasa) data docs | 0.859 |
| Llama 3.1 8B Instructrepetition: 1, local Q4_K_M, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.193 |
| Mistral Small 24B Instructrepetition: 1, route mistralai/mistral-small-24b-instruct-2501, token cap unrecorded, 2501· Benchmark Nusantara (ibahasa) data docs | 0.156 |
Data credits: Benchmark Nusantara, ibahasa (M. Khoirul Huda), doi:10.5281/zenodo.21970463. Test results CC BY 4.0, attribution to ibahasa; aggregate results CSV only. (licence)