Skip to content

Meta · Chat

Llama 3 70B Instruct

Meta 🇺🇸ChatLlama family

Newer version: Llama 3.3 70B Instruct. We compared release dates and version names in our catalogue.

From Replicate's listing:
A 70 billion parameter language model from Meta, fine tuned for chat completions
llminfo IQ
108.2
Market median price
$0.5675 blended / 1M
Max context
131K
Listings
11

Evidence and sources

llminfo IQ 108.2, ranked among chat models. From 199 benchmarks. Plenty of data behind it.How llminfo IQ is computedCoding IQ 76.5, 95% interval 66–87. From 4 benchmark families across 3 sources. Little data behind it, so treat it as a rough estimate. Uses the model's best published results. It has its own scale, separate from llminfo IQ.How Coding IQ is computedFair value (49.9). Costs per task about what its benchmark results predict. From 1 benchmark with measured task costs. Little data behind it, so treat it as a rough estimate.How Value for money worksReleased 18 Apr 2024Maker HQ: US (according to Meta's own website)
All rankings: #9 in Google Trends interest this weekSources: LMArena data, Replicate data docs, llminfo researchPrimary data from LMArena from Replicate: description from llminfo research: release date

Advantages and trade-offs

Compared with 10 chat models, picked mainly by popularity and IQ

Trade-offs

We only make a claim when there's enough data on models of the same type. Updated 22 Nov, 04:34 UTC.

Input & output

as stated by the model's providers
Text
Text

Capabilities

Supported operations and provider-stated features

Function calling

Listings

 Aihubmixaihubmix-Llama-3-70B-InstructSource: AIHubMix data docsNot published by the provider$0.70$0.70
 Flowdotmeta-llama/llama-3-70b-instructSource: FlowDot data8K$0.51$0.74
 Hugging FaceInference by Novita AImeta-llama/Meta-Llama-3-70B-InstructSource: GenAI Prices data8K$0.51$0.74
 OpenRoutermeta-llama/llama-3-70b-instructSource: GenAI Prices dataNot published by the provider$0.30$0.40
 Replicatemeta/meta-llama-3-70b-instructSources: LiteLLM data, Replicate data docs8K$0.65$2.75
 Anyscaleanyscale/meta-llama/Meta-Llama-3-70B-InstructSource: LiteLLM data8K$1.00$1.00
 Fireworks AIfireworks_ai/accounts/fireworks/models/llama-v3-70b-instructSource: LiteLLM data8K$0.90$0.90
 Hyperbolichyperbolic/meta-llama/Meta-Llama-3-70B-InstructSource: LiteLLM data131K$0.12$0.30
 Novita AImeta-llama/llama-3-70b-instructSources: GenAI Prices data, LiteLLM data8K$0.51$0.74

2 listings without a price are hidden.

Benchmarks

Named effort levels run from lowest to highest. Select a result to see its source details.

Multilingual

Bridging the Gap295 related benchmarks · 495 results
Belebele English (en) original
Source: Bridging the Gap (arXiv:2412.12417v1) data
By section · table: A.15 · task: Belebele · language: en · shots: 0
By table · row: base · task: Belebele · language: en · shots: 0
table: 2, task: Belebele, language: en, shots: 0
Belebele 11-language African average
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 1, section: average, task: Belebele
Belebele Afrikaans (af) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: af, shots: 0
Belebele Afrikaans (af) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Belebele · language: af · shots: 0
By dataset · table: A.31 · trainingLanguage: af · quality: High · quantity: 0% · task: Belebele
By dataset · table: A.31 · trainingLanguage: af · quality: Low · quantity: 0% · task: Belebele
table: 2, task: Belebele, language: af, shots: 0
Belebele Afrikaans (af) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: af, shots: 0
Belebele Afrikaans (af) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: af, shots: 0
Show 479 more resultsHide 479 results
Belebele Afrikaans (af) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: af, shots: 0
Belebele Amharic (am) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: am, shots: 0
Belebele Amharic (am) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Belebele · language: am · shots: 0
By dataset · table: A.31 · trainingLanguage: am · quality: High · quantity: 0% · task: Belebele
By dataset · table: A.31 · trainingLanguage: am · quality: Low · quantity: 0% · task: Belebele
table: 2, task: Belebele, language: am, shots: 0
Belebele Amharic (am) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: am, shots: 0
Belebele Amharic (am) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: am, shots: 0
Belebele Amharic (am) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: am, shots: 0
Belebele Bambara (bm) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: bm, shots: 0
Belebele Bambara (bm) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Belebele · language: bm · shots: 0
table: 2, task: Belebele, language: bm, shots: 0
Belebele Bambara (bm) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: bm, shots: 0
Belebele Bambara (bm) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: bm, shots: 0
Belebele Bambara (bm) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: bm, shots: 0
Belebele English (en) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: en, shots: 0
Belebele English (en) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: en, shots: 0
Belebele Igbo (ig) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: ig, shots: 0
Belebele Igbo (ig) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Belebele · language: ig · shots: 0
By dataset · table: A.31 · trainingLanguage: ig · quality: High · quantity: 0% · task: Belebele
By dataset · table: A.31 · trainingLanguage: ig · quality: Low · quantity: 0% · task: Belebele
table: 2, task: Belebele, language: ig, shots: 0
Belebele Igbo (ig) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: ig, shots: 0
Belebele Igbo (ig) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: ig, shots: 0
Belebele Igbo (ig) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: ig, shots: 0
Belebele Sepedi (nso) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: nso, shots: 0
Belebele Sepedi (nso) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Belebele · language: nso · shots: 0
By dataset · table: A.31 · trainingLanguage: nso · quality: High · quantity: 0% · task: Belebele
By dataset · table: A.31 · trainingLanguage: nso · quality: Low · quantity: 0% · task: Belebele
table: 2, task: Belebele, language: nso, shots: 0
Belebele Sepedi (nso) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: nso, shots: 0
Belebele Sepedi (nso) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: nso, shots: 0
Belebele Sepedi (nso) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: nso, shots: 0
Belebele Sesotho (st) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: st, shots: 0
Belebele Sesotho (st) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Belebele · language: st · shots: 0
By dataset · table: A.31 · trainingLanguage: st · quality: High · quantity: 0% · task: Belebele
By dataset · table: A.31 · trainingLanguage: st · quality: Low · quantity: 0% · task: Belebele
table: 2, task: Belebele, language: st, shots: 0
Belebele Sesotho (st) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: st, shots: 0
Belebele Sesotho (st) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: st, shots: 0
Belebele Sesotho (st) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: st, shots: 0
Belebele Setswana (tn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: tn, shots: 0
Belebele Setswana (tn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Belebele · language: tn · shots: 0
By dataset · table: A.31 · trainingLanguage: tn · quality: High · quantity: 0% · task: Belebele
By dataset · table: A.31 · trainingLanguage: tn · quality: Low · quantity: 0% · task: Belebele
table: 2, task: Belebele, language: tn, shots: 0
Belebele Setswana (tn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: tn, shots: 0
Belebele Setswana (tn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: tn, shots: 0
Belebele Setswana (tn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: tn, shots: 0
Belebele Shona (sn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: sn, shots: 0
Belebele Shona (sn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Belebele · language: sn · shots: 0
By dataset · table: A.31 · trainingLanguage: sn · quality: High · quantity: 0% · task: Belebele
By dataset · table: A.31 · trainingLanguage: sn · quality: Low · quantity: 0% · task: Belebele
table: 2, task: Belebele, language: sn, shots: 0
Belebele Shona (sn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: sn, shots: 0
Belebele Shona (sn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: sn, shots: 0
Belebele Shona (sn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: sn, shots: 0
Belebele Tsonga (ts) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: ts, shots: 0
Belebele Tsonga (ts) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Belebele · language: ts · shots: 0
By quality · table: A.31 · dataset: MMLU College Medicine · trainingLanguage: ts · quantity: 0% · task: Belebele
table: 2, task: Belebele, language: ts, shots: 0
Belebele Tsonga (ts) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: ts, shots: 0
Belebele Tsonga (ts) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: ts, shots: 0
Belebele Tsonga (ts) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: ts, shots: 0
Belebele Xhosa (xh) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: xh, shots: 0
Belebele Xhosa (xh) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Belebele · language: xh · shots: 0
By dataset · table: A.31 · trainingLanguage: xh · quality: High · quantity: 0% · task: Belebele
By dataset · table: A.31 · trainingLanguage: xh · quality: Low · quantity: 0% · task: Belebele
table: 2, task: Belebele, language: xh, shots: 0
Belebele Xhosa (xh) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: xh, shots: 0
Belebele Xhosa (xh) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: xh, shots: 0
Belebele Xhosa (xh) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: xh, shots: 0
Belebele Zulu (zu) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: zu, shots: 0
Belebele Zulu (zu) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Belebele · language: zu · shots: 0
By dataset · table: A.31 · trainingLanguage: zu · quality: High · quantity: 0% · task: Belebele
By dataset · table: A.31 · trainingLanguage: zu · quality: Low · quantity: 0% · task: Belebele
table: 2, task: Belebele, language: zu, shots: 0
Belebele Zulu (zu) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: zu, shots: 0
Belebele Zulu (zu) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: zu, shots: 0
Belebele Zulu (zu) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: zu, shots: 0
Clinical Knowledge Afrikaans (af) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: af, shots: 5
Clinical Knowledge Afrikaans (af) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: af, shots: 5
Clinical Knowledge Amharic (am) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: am, shots: 5
Clinical Knowledge Amharic (am) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: am, shots: 5
Clinical Knowledge Bambara (bm) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: bm, shots: 5
Clinical Knowledge Bambara (bm) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: bm, shots: 5
Clinical Knowledge English (en) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: en, shots: 5
Clinical Knowledge English (en) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: en, shots: 5
Clinical Knowledge Igbo (ig) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: ig, shots: 5
Clinical Knowledge Igbo (ig) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: ig, shots: 5
Clinical Knowledge Sepedi (nso) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: nso, shots: 5
Clinical Knowledge Sepedi (nso) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: nso, shots: 5
Clinical Knowledge Sesotho (st) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: st, shots: 5
Clinical Knowledge Sesotho (st) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: st, shots: 5
Clinical Knowledge Setswana (tn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: tn, shots: 5
Clinical Knowledge Setswana (tn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: tn, shots: 5
Clinical Knowledge Shona (sn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: sn, shots: 5
Clinical Knowledge Shona (sn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: sn, shots: 5
Clinical Knowledge Tsonga (ts) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: ts, shots: 5
Clinical Knowledge Tsonga (ts) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: ts, shots: 5
Clinical Knowledge Xhosa (xh) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: xh, shots: 5
Clinical Knowledge Xhosa (xh) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: xh, shots: 5
Clinical Knowledge Zulu (zu) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: zu, shots: 5
Clinical Knowledge Zulu (zu) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: zu, shots: 5
College Medicine Afrikaans (af) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: af, shots: 5
College Medicine Afrikaans (af) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: af, shots: 5
College Medicine Amharic (am) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: am, shots: 5
College Medicine Amharic (am) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: am, shots: 5
College Medicine Bambara (bm) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: bm, shots: 5
College Medicine Bambara (bm) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: bm, shots: 5
College Medicine English (en) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: en, shots: 5
College Medicine English (en) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: en, shots: 5
College Medicine Igbo (ig) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: ig, shots: 5
College Medicine Igbo (ig) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: ig, shots: 5
College Medicine Sepedi (nso) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: nso, shots: 5
College Medicine Sepedi (nso) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: nso, shots: 5
College Medicine Sesotho (st) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: st, shots: 5
College Medicine Sesotho (st) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: st, shots: 5
College Medicine Setswana (tn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: tn, shots: 5
College Medicine Setswana (tn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: tn, shots: 5
College Medicine Shona (sn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: sn, shots: 5
College Medicine Shona (sn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: sn, shots: 5
College Medicine Tsonga (ts) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: ts, shots: 5
College Medicine Tsonga (ts) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: ts, shots: 5
College Medicine Xhosa (xh) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: xh, shots: 5
College Medicine Xhosa (xh) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: xh, shots: 5
College Medicine Zulu (zu) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: zu, shots: 5
College Medicine Zulu (zu) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: zu, shots: 5
MMLU Clinical Knowledge 11-language African average
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 1, section: average, task: MMLU Clinical Knowledge
MMLU Clinical Knowledge Afrikaans (af) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: af, shots: 5
MMLU Clinical Knowledge Afrikaans (af) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Clinical Knowledge · language: af · shots: 5
By quality · table: A.29 · dataset: MMLU College Medicine · trainingLanguage: af · quantity: 0% · task: MMLU Clinical Knowledge
table: 2, task: MMLU Clinical Knowledge, language: af, shots: 5
MMLU Clinical Knowledge Afrikaans (af) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: af, shots: 5
MMLU Clinical Knowledge Amharic (am) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: am, shots: 5
MMLU Clinical Knowledge Amharic (am) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Clinical Knowledge · language: am · shots: 5
By dataset · table: A.29 · trainingLanguage: am · quality: High · quantity: 0% · task: MMLU Clinical Knowledge
By dataset · table: A.29 · trainingLanguage: am · quality: Low · quantity: 0% · task: MMLU Clinical Knowledge
table: 2, task: MMLU Clinical Knowledge, language: am, shots: 5
MMLU Clinical Knowledge Amharic (am) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: am, shots: 5
MMLU Clinical Knowledge Bambara (bm) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: bm, shots: 5
MMLU Clinical Knowledge Bambara (bm) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Clinical Knowledge · language: bm · shots: 5
By quality · table: A.29 · dataset: MMLU College Medicine · trainingLanguage: bm · quantity: 0% · task: MMLU Clinical Knowledge
table: 2, task: MMLU Clinical Knowledge, language: bm, shots: 5
MMLU Clinical Knowledge Bambara (bm) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: bm, shots: 5
MMLU Clinical Knowledge English (en) original
Source: Bridging the Gap (arXiv:2412.12417v1) data
By section · table: A.13 · task: MMLU Clinical Knowledge · language: en · shots: 5
By table · row: base · task: MMLU Clinical Knowledge · language: en · shots: 5
table: 2, task: MMLU Clinical Knowledge, language: en, shots: 5
MMLU Clinical Knowledge Igbo (ig) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: ig, shots: 5
MMLU Clinical Knowledge Igbo (ig) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Clinical Knowledge · language: ig · shots: 5
By dataset · table: A.29 · trainingLanguage: ig · quality: High · quantity: 0% · task: MMLU Clinical Knowledge
By dataset · table: A.29 · trainingLanguage: ig · quality: Low · quantity: 0% · task: MMLU Clinical Knowledge
table: 2, task: MMLU Clinical Knowledge, language: ig, shots: 5
MMLU Clinical Knowledge Igbo (ig) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: ig, shots: 5
MMLU Clinical Knowledge Sepedi (nso) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: nso, shots: 5
MMLU Clinical Knowledge Sepedi (nso) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Clinical Knowledge · language: nso · shots: 5
By dataset · table: A.29 · trainingLanguage: nso · quality: High · quantity: 0% · task: MMLU Clinical Knowledge
By dataset · table: A.29 · trainingLanguage: nso · quality: Low · quantity: 0% · task: MMLU Clinical Knowledge
table: 2, task: MMLU Clinical Knowledge, language: nso, shots: 5
MMLU Clinical Knowledge Sepedi (nso) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: nso, shots: 5
MMLU Clinical Knowledge Sesotho (st) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: st, shots: 5
MMLU Clinical Knowledge Sesotho (st) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Clinical Knowledge · language: st · shots: 5
By quality · table: A.29 · dataset: MMLU College Medicine · trainingLanguage: st · quantity: 0% · task: MMLU Clinical Knowledge
table: 2, task: MMLU Clinical Knowledge, language: st, shots: 5
MMLU Clinical Knowledge Sesotho (st) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: st, shots: 5
MMLU Clinical Knowledge Setswana (tn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: tn, shots: 5
MMLU Clinical Knowledge Setswana (tn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Clinical Knowledge · language: tn · shots: 5
By quality · table: A.29 · dataset: MMLU College Medicine · trainingLanguage: tn · quantity: 0% · task: MMLU Clinical Knowledge
table: 2, task: MMLU Clinical Knowledge, language: tn, shots: 5
MMLU Clinical Knowledge Setswana (tn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: tn, shots: 5
MMLU Clinical Knowledge Shona (sn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: sn, shots: 5
MMLU Clinical Knowledge Shona (sn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Clinical Knowledge · language: sn · shots: 5
By quality · table: A.29 · dataset: MMLU College Medicine · trainingLanguage: sn · quantity: 0% · task: MMLU Clinical Knowledge
table: 2, task: MMLU Clinical Knowledge, language: sn, shots: 5
MMLU Clinical Knowledge Shona (sn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: sn, shots: 5
MMLU Clinical Knowledge Tsonga (ts) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: ts, shots: 5
MMLU Clinical Knowledge Tsonga (ts) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Clinical Knowledge · language: ts · shots: 5
By quality · table: A.29 · dataset: MMLU College Medicine · trainingLanguage: ts · quantity: 0% · task: MMLU Clinical Knowledge
table: 2, task: MMLU Clinical Knowledge, language: ts, shots: 5
MMLU Clinical Knowledge Tsonga (ts) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: ts, shots: 5
MMLU Clinical Knowledge Xhosa (xh) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: xh, shots: 5
MMLU Clinical Knowledge Xhosa (xh) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Clinical Knowledge · language: xh · shots: 5
By quality · table: A.29 · dataset: MMLU College Medicine · trainingLanguage: xh · quantity: 0% · task: MMLU Clinical Knowledge
table: 2, task: MMLU Clinical Knowledge, language: xh, shots: 5
MMLU Clinical Knowledge Xhosa (xh) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: xh, shots: 5
MMLU Clinical Knowledge Zulu (zu) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: zu, shots: 5
MMLU Clinical Knowledge Zulu (zu) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Clinical Knowledge · language: zu · shots: 5
By quality · table: A.29 · dataset: MMLU College Medicine · trainingLanguage: zu · quantity: 0% · task: MMLU Clinical Knowledge
table: 2, task: MMLU Clinical Knowledge, language: zu, shots: 5
MMLU Clinical Knowledge Zulu (zu) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: zu, shots: 5
MMLU College Medicine 11-language African average
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 1, section: average, task: MMLU College Medicine
MMLU College Medicine Afrikaans (af) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: af, shots: 5
MMLU College Medicine Afrikaans (af) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: af, shots: 5
MMLU College Medicine Afrikaans (af) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: af, shots: 5
MMLU College Medicine Amharic (am) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: am, shots: 5
MMLU College Medicine Amharic (am) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: am, shots: 5
MMLU College Medicine Amharic (am) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: am, shots: 5
MMLU College Medicine Bambara (bm) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: bm, shots: 5
MMLU College Medicine Bambara (bm) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: bm, shots: 5
MMLU College Medicine Bambara (bm) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: bm, shots: 5
MMLU College Medicine English (en) original
Source: Bridging the Gap (arXiv:2412.12417v1) data
By section · table: A.12 · task: MMLU College Medicine · language: en · shots: 5
table: 2, task: MMLU College Medicine, language: en, shots: 5
MMLU College Medicine Igbo (ig) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: ig, shots: 5
MMLU College Medicine Igbo (ig) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: ig, shots: 5
MMLU College Medicine Igbo (ig) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: ig, shots: 5
MMLU College Medicine Sepedi (nso) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: nso, shots: 5
MMLU College Medicine Sepedi (nso) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: nso, shots: 5
MMLU College Medicine Sepedi (nso) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: nso, shots: 5
MMLU College Medicine Sesotho (st) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: st, shots: 5
MMLU College Medicine Sesotho (st) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: st, shots: 5
MMLU College Medicine Sesotho (st) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: st, shots: 5
MMLU College Medicine Setswana (tn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: tn, shots: 5
MMLU College Medicine Setswana (tn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: tn, shots: 5
MMLU College Medicine Setswana (tn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: tn, shots: 5
MMLU College Medicine Shona (sn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: sn, shots: 5
MMLU College Medicine Shona (sn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: sn, shots: 5
MMLU College Medicine Shona (sn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: sn, shots: 5
MMLU College Medicine Tsonga (ts) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: ts, shots: 5
MMLU College Medicine Tsonga (ts) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: ts, shots: 5
MMLU College Medicine Tsonga (ts) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: ts, shots: 5
MMLU College Medicine Xhosa (xh) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: xh, shots: 5
MMLU College Medicine Xhosa (xh) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: xh, shots: 5
MMLU College Medicine Xhosa (xh) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: xh, shots: 5
MMLU College Medicine Zulu (zu) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: zu, shots: 5
MMLU College Medicine Zulu (zu) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: zu, shots: 5
MMLU College Medicine Zulu (zu) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: zu, shots: 5
MMLU Virology 11-language African average
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 1, section: average, task: MMLU Virology
MMLU Virology Afrikaans (af) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: af, shots: 5
MMLU Virology Afrikaans (af) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Virology · language: af · shots: 5
table: 2, task: MMLU Virology, language: af, shots: 5
MMLU Virology Afrikaans (af) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: af, shots: 5
MMLU Virology Amharic (am) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: am, shots: 5
MMLU Virology Amharic (am) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Virology · language: am · shots: 5
table: 2, task: MMLU Virology, language: am, shots: 5
MMLU Virology Amharic (am) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: am, shots: 5
MMLU Virology Bambara (bm) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: bm, shots: 5
MMLU Virology Bambara (bm) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Virology · language: bm · shots: 5
table: 2, task: MMLU Virology, language: bm, shots: 5
MMLU Virology Bambara (bm) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: bm, shots: 5
MMLU Virology English (en) original
Source: Bridging the Gap (arXiv:2412.12417v1) data
By section · table: A.14 · task: MMLU Virology · language: en · shots: 5
By table · row: base · task: MMLU Virology · language: en · shots: 5
table: 2, task: MMLU Virology, language: en, shots: 5
MMLU Virology Igbo (ig) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: ig, shots: 5
MMLU Virology Igbo (ig) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Virology · language: ig · shots: 5
table: 2, task: MMLU Virology, language: ig, shots: 5
MMLU Virology Igbo (ig) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: ig, shots: 5
MMLU Virology Sepedi (nso) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: nso, shots: 5
MMLU Virology Sepedi (nso) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Virology · language: nso · shots: 5
By dataset · table: A.30 · trainingLanguage: nso · quality: High · quantity: 0% · task: MMLU Virology
By dataset · table: A.30 · trainingLanguage: nso · quality: Low · quantity: 0% · task: MMLU Virology
table: 2, task: MMLU Virology, language: nso, shots: 5
MMLU Virology Sepedi (nso) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: nso, shots: 5
MMLU Virology Sesotho (st) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: st, shots: 5
MMLU Virology Sesotho (st) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Virology · language: st · shots: 5
By quality · table: A.30 · dataset: Winogrande · trainingLanguage: st · quantity: 0% · task: MMLU Virology
table: 2, task: MMLU Virology, language: st, shots: 5
MMLU Virology Sesotho (st) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: st, shots: 5
MMLU Virology Setswana (tn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: tn, shots: 5
MMLU Virology Setswana (tn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Virology · language: tn · shots: 5
By quality · table: A.30 · dataset: MMLU College Medicine · trainingLanguage: tn · quantity: 0% · task: MMLU Virology
table: 2, task: MMLU Virology, language: tn, shots: 5
MMLU Virology Setswana (tn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: tn, shots: 5
MMLU Virology Shona (sn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: sn, shots: 5
MMLU Virology Shona (sn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Virology · language: sn · shots: 5
By quality · table: A.30 · dataset: MMLU College Medicine · trainingLanguage: sn · quantity: 0% · task: MMLU Virology
table: 2, task: MMLU Virology, language: sn, shots: 5
MMLU Virology Shona (sn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: sn, shots: 5
MMLU Virology Tsonga (ts) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: ts, shots: 5
MMLU Virology Tsonga (ts) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Virology · language: ts · shots: 5
By quality · table: A.30 · dataset: MMLU College Medicine · trainingLanguage: ts · quantity: 0% · task: MMLU Virology
table: 2, task: MMLU Virology, language: ts, shots: 5
MMLU Virology Tsonga (ts) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: ts, shots: 5
MMLU Virology Xhosa (xh) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: xh, shots: 5
MMLU Virology Xhosa (xh) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Virology · language: xh · shots: 5
By dataset · table: A.30 · trainingLanguage: xh · quality: High · quantity: 0% · task: MMLU Virology
By dataset · table: A.30 · trainingLanguage: xh · quality: Low · quantity: 0% · task: MMLU Virology
table: 2, task: MMLU Virology, language: xh, shots: 5
MMLU Virology Xhosa (xh) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: xh, shots: 5
MMLU Virology Zulu (zu) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: zu, shots: 5
MMLU Virology Zulu (zu) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: MMLU Virology · language: zu · shots: 5
By dataset · table: A.30 · trainingLanguage: zu · quality: High · quantity: 0% · task: MMLU Virology
By dataset · table: A.30 · trainingLanguage: zu · quality: Low · quantity: 0% · task: MMLU Virology
table: 2, task: MMLU Virology, language: zu, shots: 5
MMLU Virology Zulu (zu) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: zu, shots: 5
Virology Afrikaans (af) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: af, shots: 5
Virology Afrikaans (af) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: af, shots: 5
Virology Amharic (am) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: am, shots: 5
Virology Amharic (am) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: am, shots: 5
Virology Bambara (bm) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: bm, shots: 5
Virology Bambara (bm) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: bm, shots: 5
Virology English (en) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: en, shots: 5
Virology English (en) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: en, shots: 5
Virology Igbo (ig) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: ig, shots: 5
Virology Igbo (ig) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: ig, shots: 5
Virology Sepedi (nso) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: nso, shots: 5
Virology Sepedi (nso) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: nso, shots: 5
Virology Sesotho (st) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: st, shots: 5
Virology Sesotho (st) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: st, shots: 5
Virology Setswana (tn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: tn, shots: 5
Virology Setswana (tn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: tn, shots: 5
Virology Shona (sn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: sn, shots: 5
Virology Shona (sn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: sn, shots: 5
Virology Tsonga (ts) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: ts, shots: 5
Virology Tsonga (ts) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: ts, shots: 5
Virology Xhosa (xh) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: xh, shots: 5
Virology Xhosa (xh) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: xh, shots: 5
Virology Zulu (zu) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: zu, shots: 5
Virology Zulu (zu) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: zu, shots: 5
Winogrande 11-language African average
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 1, section: average, task: Winogrande
Winogrande Afrikaans (af) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: af, shots: 5
Winogrande Afrikaans (af) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Winogrande · language: af · shots: 5
By dataset · table: A.28 · trainingLanguage: af · quality: High · quantity: 0% · task: Winogrande
By dataset · table: A.28 · trainingLanguage: af · quality: Low · quantity: 0% · task: Winogrande
table: 2, task: Winogrande, language: af, shots: 5
Winogrande Afrikaans (af) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: af, shots: 5
Winogrande Afrikaans (af) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: af, shots: 5
Winogrande Afrikaans (af) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: af, shots: 5
Winogrande Amharic (am) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: am, shots: 5
Winogrande Amharic (am) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Winogrande · language: am · shots: 5
table: 2, task: Winogrande, language: am, shots: 5
Winogrande Amharic (am) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: am, shots: 5
Winogrande Amharic (am) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: am, shots: 5
Winogrande Amharic (am) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: am, shots: 5
Winogrande Bambara (bm) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: bm, shots: 5
Winogrande Bambara (bm) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Winogrande · language: bm · shots: 5
table: 2, task: Winogrande, language: bm, shots: 5
Winogrande Bambara (bm) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: bm, shots: 5
Winogrande Bambara (bm) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: bm, shots: 5
Winogrande Bambara (bm) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: bm, shots: 5
Winogrande English (en) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: en, shots: 5
Winogrande English (en) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: en, shots: 5
Winogrande English (en) original
Source: Bridging the Gap (arXiv:2412.12417v1) data
By section · table: A.11 · task: Winogrande · language: en · shots: 5
By table · row: base · task: Winogrande · language: en · shots: 5
By dataset · table: A.28 · trainingLanguage: en · quality: High · quantity: 0% · task: Winogrande
By dataset · table: A.28 · trainingLanguage: en · quality: Low · quantity: 0% · task: Winogrande
table: 2, task: Winogrande, language: en, shots: 5
Winogrande Igbo (ig) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: ig, shots: 5
Winogrande Igbo (ig) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Winogrande · language: ig · shots: 5
table: 2, task: Winogrande, language: ig, shots: 5
Winogrande Igbo (ig) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: ig, shots: 5
Winogrande Igbo (ig) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: ig, shots: 5
Winogrande Igbo (ig) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: ig, shots: 5
Winogrande Sepedi (nso) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: nso, shots: 5
Winogrande Sepedi (nso) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Winogrande · language: nso · shots: 5
table: 2, task: Winogrande, language: nso, shots: 5
Winogrande Sepedi (nso) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: nso, shots: 5
Winogrande Sepedi (nso) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: nso, shots: 5
Winogrande Sepedi (nso) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: nso, shots: 5
Winogrande Sesotho (st) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: st, shots: 5
Winogrande Sesotho (st) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Winogrande · language: st · shots: 5
table: 2, task: Winogrande, language: st, shots: 5
Winogrande Sesotho (st) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: st, shots: 5
Winogrande Sesotho (st) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: st, shots: 5
Winogrande Sesotho (st) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: st, shots: 5
Winogrande Setswana (tn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: tn, shots: 5
Winogrande Setswana (tn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Winogrande · language: tn · shots: 5
table: 2, task: Winogrande, language: tn, shots: 5
Winogrande Setswana (tn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: tn, shots: 5
Winogrande Setswana (tn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: tn, shots: 5
Winogrande Setswana (tn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: tn, shots: 5
Winogrande Shona (sn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: sn, shots: 5
Winogrande Shona (sn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Winogrande · language: sn · shots: 5
table: 2, task: Winogrande, language: sn, shots: 5
Winogrande Shona (sn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: sn, shots: 5
Winogrande Shona (sn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: sn, shots: 5
Winogrande Shona (sn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: sn, shots: 5
Winogrande Tsonga (ts) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: ts, shots: 5
Winogrande Tsonga (ts) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Winogrande · language: ts · shots: 5
table: 2, task: Winogrande, language: ts, shots: 5
Winogrande Tsonga (ts) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: ts, shots: 5
Winogrande Tsonga (ts) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: ts, shots: 5
Winogrande Tsonga (ts) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: ts, shots: 5
Winogrande Xhosa (xh) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: xh, shots: 5
Winogrande Xhosa (xh) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Winogrande · language: xh · shots: 5
table: 2, task: Winogrande, language: xh, shots: 5
Winogrande Xhosa (xh) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: xh, shots: 5
Winogrande Xhosa (xh) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: xh, shots: 5
Winogrande Xhosa (xh) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: xh, shots: 5
Winogrande Zulu (zu) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: zu, shots: 5
Winogrande Zulu (zu) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
By table · row: base · task: Winogrande · language: zu · shots: 5
table: 2, task: Winogrande, language: zu, shots: 5
Winogrande Zulu (zu) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: zu, shots: 5
Winogrande Zulu (zu) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: zu, shots: 5
Winogrande Zulu (zu) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: zu, shots: 5
SeaLLMs16 related benchmarks · 16 results
Show 10 more resultsHide 10 results

Human preference

Arena ELO27 related benchmarks · 27 results
WildBench26 related benchmarks · 26 results
WB_Elo
Source: WildBench direct data
allenai/WildBench, WB_Elo
gpt4t_reward.Creative Tasks.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09
gpt4t_reward.Information/Advice seeking.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Information/Advice seeking.K=-1, gpt-4-turbo-2024-04-09
gpt4t_reward.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.K: -1, gpt-4-turbo-2024-04-09
gpt4t_reward.Planning & Reasoning.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Planning & Reasoning.K=-1, gpt-4-turbo-2024-04-09
gpt4t_reward.task_macro.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.task_macro.K: -1, gpt-4-turbo-2024-04-09
Show 20 more resultsHide 20 results
haiku_reward.Creative Tasks.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Creative Tasks.K: -1, claude-3-haiku-20240307
haiku_reward.Information/Advice seeking.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Information/Advice seeking.K=-1, claude-3-haiku-20240307
haiku_reward.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.K: -1, claude-3-haiku-20240307
haiku_reward.Planning & Reasoning.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307
haiku_reward.task_macro.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.task_macro.K: -1, claude-3-haiku-20240307
llama_reward.Creative Tasks.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf
llama_reward.Information/Advice seeking.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Information/Advice seeking.K=-1, Llama-2-70b-chat-hf
llama_reward.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf
llama_reward.Planning & Reasoning.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Planning & Reasoning.K=-1, Llama-2-70b-chat-hf
llama_reward.task_macro.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf
mixture_of_rewards.Creative Tasks.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
mixture_of_rewards.Information/Advice seeking.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Information/Advice seeking.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
mixture_of_rewards.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
mixture_of_rewards.Planning & Reasoning.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Planning & Reasoning.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
task_macro_reward.K=-1
Source: WildBench direct data
allenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
WB_score
Source: WildBench direct data
allenai/WildBench, WB_score
WB_score.Creative Tasks
Source: WildBench direct data
allenai/WildBench, WB_score.Creative Tasks
WB_score.Information/Advice seeking
Source: WildBench direct data
allenai/WildBench, WB_score.Information/Advice seeking
WB_score.Planning & Reasoning
Source: WildBench direct data
allenai/WildBench, WB_score.Planning & Reasoning
WB_score.task_macro
Source: WildBench direct data
allenai/WildBench, WB_score.task_macro

Science

ChemBench10 related benchmarks · 20 results
Source: ChemBench verified leaderboard data docs
Benchmarkllama3-70b-instruct-T-onellama3-70b-instruct
ChemBench
Analytical Chemistry
Chemical Preference
General Chemistry
Show 6 more benchmarksHide 6 benchmarks
Benchmarkllama3-70b-instruct-T-onellama3-70b-instruct
Inorganic Chemistry
Materials Science
Organic Chemistry
Physical Chemistry
Technical Chemistry
Toxicity and Safety

Coding

WildBench5 related benchmarks · 5 results
WB_score.Coding & Debugging
Source: WildBench direct data
allenai/WildBench, WB_score.Coding & Debugging
gpt4t_reward.Coding & Debugging.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Coding & Debugging.K=-1, gpt-4-turbo-2024-04-09
haiku_reward.Coding & Debugging.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Coding & Debugging.K=-1, claude-3-haiku-20240307
llama_reward.Coding & Debugging.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf
mixture_of_rewards.Coding & Debugging.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Coding & Debugging.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf

Math

WildBench5 related benchmarks · 5 results
WB_score.Math & Data Analysis
Source: WildBench direct data
allenai/WildBench, WB_score.Math & Data Analysis
gpt4t_reward.Math & Data Analysis.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09
haiku_reward.Math & Data Analysis.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307
llama_reward.Math & Data Analysis.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf
mixture_of_rewards.Math & Data Analysis.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf

Reasoning

Agentic

GAIA4 results
Source: GAIA Leaderboard data · May 2024
Difficulty level
Difficulty not reported

General

Instruction following

Knowledge

Other

Open Greek FinLLM11 related benchmarks · 11 results
QA raw score
Source: TheFinAI public numeric results
greek-contents, QA, QA Raw
FinNum raw score
Source: TheFinAI public numeric results
greek-contents, FinNum, FinNum Raw
FinText raw score
Source: TheFinAI public numeric results
greek-contents, FinText, FinText Raw
Show 5 more resultsHide 5 results
FNS raw score
Source: TheFinAI public numeric results
greek-contents, FNS, FNS Raw
MultiFin raw score
Source: TheFinAI public numeric results
greek-contents, MultiFin, MultiFin Raw
overall normalized index
Source: TheFinAI public numeric results
greek-contents, published-average

Data credits: LMArena, CC BY 4.0 (licence) · Tuka Alhanai, Adam Kasumovic, Mohammad Ghassemi, Aven Zitzelberger, Jessica Lundin and Guillaume Chabot-Couture. Bridging the Gap: Enhancing LLM Performance for Low-Resource African Languages with New Benchmarks, Fine-Tuning, and Cultural Adjustments, arXiv:2412.12417v1. CC BY 4.0 article numeric/protocol extract, restructured with values unchanged. No task or model rights asserted. (licence) · ChemBench leaderboard, Jablonka group / lamalab-org. Public verified numeric result summaries; results licence not separately stated. · Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · EvalPlus Leaderboard and EvalPerf (evalplus.github.io), Apache-2.0; EvalPerf win-rate fractions converted to percentages (licence) · LLM Chess by Maxim Saplin, Apache-2.0 (Copyright [2024] [Maxim Saplin]) (licence) · The Fin AI. Published FinBen result summaries and Open Greek Financial LLM Leaderboard aggregates. Results licence not separately stated; fractional FinBen success metrics converted to percentages. · pfgen-bench, Preferred Networks. Apache-2.0 except LLM-generated outputs; aggregate scores only. (licence) · SeaExam and SeaBench leaderboard, SeaLLMs (DAMO-NLP-SG). Published leaderboard scores; results licence not stated. · SnakeBench (Greg Kamradt / ARC Prize Foundation), MIT (licence) · WildBench, Allen Institute for AI

Also known as: aihubmix-llama-3-70b-instruct, anyscale/meta-llama/meta-llama-3-70b-instruct, fireworks_ai/accounts/fireworks/models/llama-v3-70b-instruct, hyperbolic/meta-llama/meta-llama-3-70b-instruct, meta-llama-3-70b-instruct, meta-llama/llama-3-70b-instruct, meta-llama/meta-llama-3-70b-instruct, meta/meta-llama-3-70b-instruct, novita/meta-llama/llama-3-70b-instruct, replicate/meta/llama-3-70b-instruct, replicate/meta/meta-llama-3-70b-instruct, vertex_ai/meta/llama3-70b-instruct-maas

Like this one. Tap or hover any dotted value, label or icon to see where it comes from and how it is measured. open details when tapped or hovered. How this data works