Skip to content

Benchmark

NepNewsBench v2 draft core fact recall (Nepali-source questions)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
16
Scores recorded
16
Sources
1
IQ weight
No weight published

Score distribution

0.896
Mean
0.897
Median
0.033
Std dev
0.861
10th percentile
0.947
90th percentile
0.822
Min

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

ModelScore
Claude Fable 5.1Anthropic via openrouter, reasoning low· NepNewsBench v2 (draft candidate release) data0.947
Claude Haiku 4.5Anthropic via openrouter, reasoning off· NepNewsBench v2 (draft candidate release) data0.867
Claude Opus 5Anthropic via openrouter, reasoning off· NepNewsBench v2 (draft candidate release) data0.947
Claude Sonnet 4.6Anthropic via openrouter, reasoning off· NepNewsBench v2 (draft candidate release) data0.924
Claude Sonnet 5Anthropic via openrouter, reasoning off· NepNewsBench v2 (draft candidate release) data0.914
Gemini 3 FlashGoogle AI Studio via openrouter, reasoning off· NepNewsBench v2 (draft candidate release) data0.888
Gemini 3.1 Pro PreviewGoogle AI Studio via openrouter, reasoning low· NepNewsBench v2 (draft candidate release) data0.859
Gemini 3.8 FlashGoogle AI Studio via openrouter, reasoning low· NepNewsBench v2 (draft candidate release) data0.887
Gemma 4 31B ITVenice via openrouter, reasoning off· NepNewsBench v2 (draft candidate release) data0.861
GLM 5.3Z.AI via openrouter, reasoning low· NepNewsBench v2 (draft candidate release) data0.924
GPT-5.6 LunaOpenAI via openrouter, reasoning off· NepNewsBench v2 (draft candidate release) data0.882
GPT-5.6 SolOpenAI via openrouter, reasoning off· NepNewsBench v2 (draft candidate release) data0.894
Grok 4.6xAI via openrouter, reasoning low· NepNewsBench v2 (draft candidate release) data0.907
Kimi K3Moonshot AI via openrouter, reasoning off· NepNewsBench v2 (draft candidate release) data0.918
Qwen 3.8 27BAlibaba via openrouter, reasoning off· NepNewsBench v2 (draft candidate release) data0.822
Qwen3.8 MaxAlibaba via openrouter, reasoning minimal· NepNewsBench v2 (draft candidate release) data0.897

Data credits: Outback Yak Research, NepNewsBench v2 research draft. Public numerical aggregates from a candidate preview that the publisher says is not evidence of publication; no release licence assigned and none asserted here.