Skip to content

Benchmark

MTEB(eng, v1) - Summarization

Embedding benchmarks measure how well models find related items, for example when matching a search to relevant results. Scores are percentages.

Models measured
36
Scores recorded
36
Sources
1
IQ weight
0.19

Score distribution

30.7%
Mean
30.8%
Median
1.1%
Std dev
29.6%
10th percentile
32%
90th percentile
27.5%
Min
27.5%32.6%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.19 for embedding models, measured over 36 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Text Embedding 004Summarization· MTEB Leaderboard data32.6%
2bge-large-enSummarization· MTEB Leaderboard data32.1%
3Granite Embedding 278m MultilingualSummarization· MTEB Leaderboard data32%
4UAE-Large-V1Summarization· MTEB Leaderboard data32%
5Text Embedding 005Summarization· MTEB Leaderboard data31.9%
6Qwen3 Embedding 4BSummarization· MTEB Leaderboard data31.8%
7Gte LargeSummarization· MTEB Leaderboard data31.7%
8BGE Large En V1.5Summarization· MTEB Leaderboard data31.6%
9Jina Embeddings v2 Base ENSummarization· MTEB Leaderboard data31.6%
10Paraphrase Multilingual MPNet v2Summarization· MTEB Leaderboard data31.6%
11E5 Mistral 7B InstructSummarization· MTEB Leaderboard data31.5%
12Qwen3 Embedding 8BSummarization· MTEB Leaderboard data31.3%
13BGE multilingual gemma2Summarization· MTEB Leaderboard data31.2%
14Gte BaseSummarization· MTEB Leaderboard data31.2%
15Text Embedding 3 SmallSummarization· MTEB Leaderboard data31.1%
16BGE Base En V1.5Summarization· MTEB Leaderboard data31.1%
17All MiniLM L6 V2Summarization· MTEB Leaderboard data30.8%
18Text Embedding Ada 002Summarization· MTEB Leaderboard data30.8%
19BGE En IclSummarization· MTEB Leaderboard data30.8%
20Jina Embedding B EN v1Summarization· MTEB Leaderboard data30.7%
21NV-Embed-v2Summarization· MTEB Leaderboard data30.7%
22snowflake-arctic-embed-l-v2.0Summarization· MTEB Leaderboard data30.6%
23Multilingual E5 Large InstructSummarization· MTEB Leaderboard data30.5%
24nomic-embed-text-v1.5Summarization· MTEB Leaderboard data30.4%
25E5 Base V2Summarization· MTEB Leaderboard data30.3%
26E5 Large V2Summarization· MTEB Leaderboard data30.2%
27bge-small-en-v1.5Summarization· MTEB Leaderboard data30.1%
28nomic-embed-text-v1Summarization· MTEB Leaderboard data30.1%
29Text Embedding 3 LargeSummarization· MTEB Leaderboard data29.9%
30Qwen3 Embedding 0.6BSummarization· MTEB Leaderboard data29.8%
31Jina Embeddings v3Summarization· MTEB Leaderboard data29.7%
32Multilingual E5 LargeSummarization· MTEB Leaderboard data29.6%
33snowflake-arctic-embed-m-v2.0Summarization· MTEB Leaderboard data29.5%
34Granite Embedding Small English R2Summarization· MTEB Leaderboard data28.7%
35All MiniLM L12 V2Summarization· MTEB Leaderboard data27.9%
36All Mpnet Base V2Summarization· MTEB Leaderboard data27.5%