Benchmark
Benchmark Nusantara jv-B arti-kata (prompt arti-kata-v1, scorer kunci-alias-v1, items a7ba9fa24e64, key 873b6f168502, withheld slice)
Multilingual benchmarks compare model performance across languages. Higher scores are better.
- Models measured
- 17
- Scores recorded
- 17
- Sources
- 1
- IQ weight
- No weight published
Score distribution
- 0.547
- Mean
- 0.656
- Median
- 0.275
- Std dev
- 0.141
- 10th percentile
- 0.828
- 90th percentile
- 0.0938
- Min
Results
The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.
| Model | Score |
|---|---|
| Claude Haiku 4.5repetition: 1, route anthropic/claude-haiku-4.5, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.641 |
| Claude Haiku 5.5repetition: 1, route anthropic/claude-haiku-5.5, token cap 6000· Benchmark Nusantara (ibahasa) data docs | 0.797 |
| Claude Sonnet 5.5repetition: 1, route anthropic/claude-sonnet-5.5, token cap 6000· Benchmark Nusantara (ibahasa) data docs | 0.844 |
| DeepSeek V4 Flash 0731repetition: 1, route deepseek/deepseek-v4-flash-0731, token cap 3000· Benchmark Nusantara (ibahasa) data docs | 0.766 |
| Gemini 3.5 Flashrepetition: 1, route google/gemini-3.5-flash, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.906 |
| Gemma 4 31B ITrepetition: 1, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.797 |
| GLM 5.3repetition: 1, route z-ai/glm-5.3, token cap 6000· Benchmark Nusantara (ibahasa) data docs | 0.656 |
| GLM 5.3 Flashrepetition: 1, route z-ai/glm-5.3-flash, token cap 6000· Benchmark Nusantara (ibahasa) data docs | 0.688 |
| GPT-5.6 Lunarepetition: 1, route openai/gpt-5.6-luna, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.828 |
| Ling 3.0 Flashrepetition: 1, route inclusionai/ling-3.0-flash, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.422 |
| Llama 3.1 8B Instructrepetition: 1, local Q4_K_M, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.172 |
| MiMo V2.6 Flashrepetition: 1, route xiaomi/mimo-v2.6-flash, token cap 6000· Benchmark Nusantara (ibahasa) data docs | 0.672 |
| Mistral Small 24B Instructrepetition: 1, route mistralai/mistral-small-24b-instruct-2501, token cap 600, 2501· Benchmark Nusantara (ibahasa) data docs | 0.0938 |
| Nemotron 3.5 Lightningrepetition: 1, route nvidia/nemotron-3.5-lightning, token cap 3000· Benchmark Nusantara (ibahasa) data docs | 0.141 |
| Qwen3 30B A3B Instructrepetition: 1, route qwen/qwen3-30b-a3b-instruct-2507, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.359 |
| Seed 2.0 Minirepetition: 1, route bytedance-seed/seed-2.0-mini, token cap 6000· Benchmark Nusantara (ibahasa) data docs | 0.141 |
| Solar Pro 4repetition: 1, route upstage/solar-pro4, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.375 |
Data credits: Benchmark Nusantara, ibahasa (M. Khoirul Huda), doi:10.5281/zenodo.21970463. Test results CC BY 4.0, attribution to ibahasa; aggregate results CSV only. (licence)