Skip to content

Meta · Chat

Llama 3.1 70B Instruct

Meta 🇺🇸ChatLlama familyOpen weights. One of its sources says the weights are publicly available.The maker currently plans to deprecate it within the next 180 days (30 Mar 2027). Plans can change.

Newer version: Llama 3.3 70B Instruct. We compared release dates and version names in our catalogue.

From Llmgateway's listing:
Mid-size Llama 3.1 with balanced performance.
llminfo IQ
106.8
Market median price
$0.72 blended / 1M
Max context
131K
Listings
29

Evidence and sources

🤗 meta-llama/Meta-Llama-3.1-70B-Instructllminfo IQ 106.8, ranked among chat models. From 168 benchmarks. Plenty of data behind it.How llminfo IQ is computedCoding IQ 80.3, 95% interval 69–91. From 4 benchmark families across 4 sources. Little data behind it, so treat it as a rough estimate. Uses the model's best published results. It has its own scale, separate from llminfo IQ.How Coding IQ is computedPoor value (41.0). Costs more per task than its benchmark results predict. From 2 benchmarks with measured task costs. Little data behind it, so treat it as a rough estimate.How Value for money worksReleased 16 Jul 2024 Training-data cutoff, as its sources state it Maker HQ: US (according to Meta's own website)
All rankings: #364 on OpenRouter today #367 on OpenRouter this week #369 on OpenRouter this monthSources: LMArena data, LiteLLM data, Models.dev data, OpenRouter data docs, LLMGateway data, Hugging FacePrimary data from LMArena from LiteLLM: deprecation date from Models.dev: weights availability from OpenRouter: knowledge cutoff from LLMGateway: description from Hugging Face: release date

Advantages and trade-offs

Compared with 10 chat models, picked mainly by popularity and IQ

Advantages

Open weights available

The weights are public. 3 of the 8 peers that say either way have open weights.

Trade-offs

Deprecation scheduled

The maker plans to deprecate it on 30 Mar 2027, within the next 180 days.

We only make a claim when there's enough data on models of the same type. Updated 22 Nov, 06:07 UTC.

Input & output

as stated by the model's providers
Text
Text

Capabilities

Supported operations and provider-stated features

VisionFunction callingStructured outputBatch API

Listings

 Aihubmixmeta-llama/llama-3.1-70b-instruct:free · freeSource: AIHubMix data docsNot published by the provider$0.02$0.02
 Hyperbolichyperbolic/meta-llama/Meta-Llama-3.1-70B-InstructSource: LiteLLM data33K$0.12$0.30
 Lambda AIlambda_ai/llama3.1-70b-instruct-fp8 · fp8Source: LiteLLM data131K$0.12$0.30
 Nebius AI Studionebius/meta-llama/Meta-Llama-3.1-70B-InstructSource: LiteLLM data128K$0.13$0.40
 RequestyInference by DeepInfradeepinfra/meta-llama/Meta-Llama-3.1-70B-InstructSource: Requesty data131K$0.23$0.40
 Novita AImeta-llama/llama-3.1-70b-instructSource: GenAI Prices dataNot published by the provider$0.34$0.39
 Flowdotmeta-llama/llama-3.1-70b-instructSource: FlowDot data131K$0.40$0.40
 DeepInfradeepinfra/meta-llama/Meta-Llama-3.1-70B-InstructSource: LiteLLM data131K$0.40$0.40
 AvianMeta-Llama-3.1-70B-InstructSource: GenAI Prices dataNot published by the provider$0.45$0.45
 OVHcloudovhcloud/Meta-Llama-3_1-70B-InstructSource: LiteLLM data131K$0.67$0.67

4 listings without a price are hidden.

Benchmarks

Named effort levels run from lowest to highest. Select a result to see its source details.

Math

GSM8K8-shot · chain of thought · Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
MGSMmultilingual average · Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
mu-MATH7 related benchmarks · 35 results
F1
Source: U-MATH and mu-MATH
F1 · Qwen/Qwen2.5-72B-Instruct
accuracy
Source: U-MATH and mu-MATH
accuracy · Qwen/Qwen2.5-72B-Instruct
MCC
Source: U-MATH and mu-MATH
MCC · Qwen/Qwen2.5-72B-Instruct
NPV
Source: U-MATH and mu-MATH
NPV · Qwen/Qwen2.5-72B-Instruct
PPV
Source: U-MATH and mu-MATH
PPV · Qwen/Qwen2.5-72B-Instruct
TNR
Source: U-MATH and mu-MATH
TNR · Qwen/Qwen2.5-72B-Instruct
Show 5 more resultsHide 5 results
TPR
Source: U-MATH and mu-MATH
TPR · Qwen/Qwen2.5-72B-Instruct
U-MATH7 related benchmarks · 21 results
algebra accuracy
Source: U-MATH and mu-MATH
Task set · algebra · gpt-4o-2024-08-06
differential_calc accuracy
Source: U-MATH and mu-MATH
Task set · differential_calc · gpt-4o-2024-08-06
integral_calc accuracy
Source: U-MATH and mu-MATH
Task set · integral_calc · gpt-4o-2024-08-06
multivariable_calculus accuracy
Source: U-MATH and mu-MATH
Task set · multivariable_calculus · gpt-4o-2024-08-06
overall accuracy
Source: U-MATH and mu-MATH
Task set · u_math · gpt-4o-2024-08-06
precalculus_review accuracy
Source: U-MATH and mu-MATH
Task set · precalculus_review · gpt-4o-2024-08-06
Show 3 more resultsHide 3 results
sequences_series accuracy
Source: U-MATH and mu-MATH
Task set · sequences_series · gpt-4o-2024-08-06

Coding

CodeElo28 related benchmarks · 28 results
HumanEval2 results
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
official instruction-tuned table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
MBPP+base split · EvalPlus · Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
official instruction-tuned table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
official instruction-tuned table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated

Human preference

Multilingual

Science

ChemBench10 related benchmarks · 20 results
Source: ChemBench verified leaderboard data docs
Benchmarkllama3.1-70b-instruct-T-onellama3.1-70b-instruct
ChemBench
Analytical Chemistry
Chemical Preference
General Chemistry
Show 6 more benchmarksHide 6 benchmarks
Benchmarkllama3.1-70b-instruct-T-onellama3.1-70b-instruct
Inorganic Chemistry
Materials Science
Organic Chemistry
Physical Chemistry
Technical Chemistry
Toxicity and Safety
GPQA2 results
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated

Long context

BABILong avg(qa1-5)9 related benchmarks · 9 results
0k
Source: BABILong v0 data docs
Meta-Llama-3.1-70B-Instruct, avg, 0k
1k
Source: BABILong v0 data docs
Meta-Llama-3.1-70B-Instruct, avg, 1k
2k
Source: BABILong v0 data docs
Meta-Llama-3.1-70B-Instruct, avg, 2k
4k
Source: BABILong v0 data docs
Meta-Llama-3.1-70B-Instruct, avg, 4k
8k
Source: BABILong v0 data docs
Meta-Llama-3.1-70B-Instruct, avg, 8k
16k
Source: BABILong v0 data docs
Meta-Llama-3.1-70B-Instruct, avg, 16k
Show 3 more resultsHide 3 results
32k
Source: BABILong v0 data docs
Meta-Llama-3.1-70B-Instruct, avg, 32k
64k
Source: BABILong v0 data docs
Meta-Llama-3.1-70B-Instruct, avg, 64k
128k
Source: BABILong v0 data docs
Meta-Llama-3.1-70B-Instruct, avg, 128k

Instruction following

IFEval2 results
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
official instruction-tuned table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated

Agentic

BFCLSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Nexus2 results
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
official instruction-tuned table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated

Reasoning

API-Bank2 results
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
official instruction-tuned table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
ARC-CSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
ARC-ChallengeReported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DROPSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

Knowledge

MMLU3 related benchmarks · 6 results
MMLU
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
0-shot · chain of thought
5-shot · macro_avg/acc
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
(CoT)
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Pro
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
5-shot · chain of thought
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated

General

Other

Gorilla API Benchofficial instruction-tuned table · Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.

Data credits: ACEBench, MIT (licence) · BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · Data by LLM Stats · LMArena, CC BY 4.0 (licence) · RMT-team, Kuratov et al. Official v0 numeric CSV; results licence not separately stated. · Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · ChemBench leaderboard, Jablonka group / lamalab-org. Public verified numeric result summaries; results licence not separately stated. · CodeElo (Qwen). Aggregate scores from the official results file; no results licence asserted. The evaluation scaffold agreement is not presented as a licence for these scores · EQ-Bench (Samuel J. Paech) — cite paech2023eqbench · Keelemudelite mõõdupuu: Independent LLM leaderboard for Estonian. Eesti Keele Instituut. https://moodupuu.eki.ee, CC BY 4.0. Latest per-benchmark results selected with partial-run flags retained. (licence) · EvalPlus Leaderboard and EvalPerf (evalplus.github.io), Apache-2.0; EvalPerf win-rate fractions converted to percentages (licence) · JudgeBench, Sijun Tan et al., ScalerLab · Toloka, U-MATH and mu-MATH public leaderboard. MIT Space; generated solution outputs retain underlying model terms and are not ingested. (licence) · pfgen-bench, Preferred Networks. Apache-2.0 except LLM-generated outputs; aggregate scores only. (licence) · NVIDIA RULER, Hsieh et al. Official README numeric table; no separate results licence asserted. · SeaExam and SeaBench leaderboard, SeaLLMs (DAMO-NLP-SG). Published leaderboard scores; results licence not stated.

Also known as: aihubmix-llama-3-1-70b-instruct, aws-bedrock/llama-3.1-70b-instruct, deepinfra/meta-llama/meta-llama-3.1-70b-instruct, fireworks_ai/accounts/fireworks/models/llama-v3p1-70b-instruct, hyperbolic/meta-llama/meta-llama-3.1-70b-instruct, lambda_ai/llama3.1-70b-instruct-fp8, meta-llama-3.1-70b-instruct, meta-llama/llama-3.1-70b-instruct, meta-llama/llama-3.1-70b-instruct:free, meta.llama3-1-70b-instruct-v1:0, meta/llama-3.1-70b-instruct, nebius/meta-llama/meta-llama-3.1-70b-instruct, nim/meta/llama-3.1-70b-instruct, oci/meta.llama-3.1-70b-instruct, ovhcloud/meta-llama-3_1-70b-instruct, perplexity/llama-3.1-70b-instruct, us.meta.llama3-1-70b-instruct-v1:0, vertex_ai/meta/llama-3.1-70b-instruct-maas, wandb/meta-llama/llama-3.1-70b-instruct

Like this one. Tap or hover any dotted value, label or icon to see where it comes from and how it is measured. open details when tapped or hovered. How this data works