Skip to content

Benchmark

MTEB(eng, v2) - Summarization

Embedding benchmarks measure how well models find related items, for example when matching a search to relevant results. Scores are percentages.

Models measured
40
Scores recorded
40
Sources
1
IQ weight
0.79

Score distribution

31.2%
Mean
32%
Median
5.2%
Std dev
26.7%
10th percentile
35.2%
90th percentile
6%
Min
6%38.3%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.79 for embedding models, measured over 40 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini Embedding 001Summarization· MTEB Leaderboard data38.3%
2Text Multilingual Embedding 002Summarization· MTEB Leaderboard data36.8%
3E5 Mistral 7B InstructSummarization· MTEB Leaderboard data36.6%
4Text Embedding 004Summarization· MTEB Leaderboard data36.1%
5NV-Embed-v2Summarization· MTEB Leaderboard data35.2%
6embeddinggemma-2Summarization· MTEB Leaderboard data35.1%
7Text Embedding 005Summarization· MTEB Leaderboard data35.1%
8Qwen3 Embedding 8BSummarization· MTEB Leaderboard data34.8%
9BGE M3Summarization· MTEB Leaderboard data34.8%
10Qwen3 Embedding 4BSummarization· MTEB Leaderboard data34.4%
11Text Embedding 3 LargeSummarization· MTEB Leaderboard data34.3%
12E5 Base V2Summarization· MTEB Leaderboard data34.3%
13snowflake-arctic-embed-l-v2.0Summarization· MTEB Leaderboard data33.8%
14Qwen3 Embedding 0.6BSummarization· MTEB Leaderboard data33.4%
15BGE Large En V1.5Summarization· MTEB Leaderboard data33.1%
16Gte LargeSummarization· MTEB Leaderboard data32.9%
17Jina Embeddings v3Summarization· MTEB Leaderboard data32.9%
18Text Embedding 3 SmallSummarization· MTEB Leaderboard data32.5%
19E5 Large V2Summarization· MTEB Leaderboard data32.3%
20Jina Embeddings v5 Omni NanoSummarization· MTEB Leaderboard data32%
21Jina Embeddings v5 Text NanoSummarization· MTEB Leaderboard data32%
22Jina Embeddings v5 Omni SmallSummarization· MTEB Leaderboard data31.8%
23Jina Embeddings v5 Text SmallSummarization· MTEB Leaderboard data31.8%
24Multilingual E5 LargeSummarization· MTEB Leaderboard data31.4%
25Gte BaseSummarization· MTEB Leaderboard data30.9%
26nomic-embed-text-v1Summarization· MTEB Leaderboard data30.4%
27snowflake-arctic-embed-m-v2.0Summarization· MTEB Leaderboard data30.3%
28BGE Base En V1.5Summarization· MTEB Leaderboard data30.2%
29UAE-Large-V1Summarization· MTEB Leaderboard data30.1%
30Multilingual E5 Large InstructSummarization· MTEB Leaderboard data29.9%
31Granite Embedding 278m MultilingualSummarization· MTEB Leaderboard data29.7%
32Paraphrase Multilingual MPNet v2Summarization· MTEB Leaderboard data29.5%
33bge-small-en-v1.5Summarization· MTEB Leaderboard data28.9%
34nomic-embed-text-v1.5Summarization· MTEB Leaderboard data28.6%
35Jina Embedding B EN v1Summarization· MTEB Leaderboard data27.1%
36Granite Embedding Small English R2Summarization· MTEB Leaderboard data26.7%
37All MiniLM L6 V2Summarization· MTEB Leaderboard data26%
38All MiniLM L12 V2Summarization· MTEB Leaderboard data23.8%
39All Mpnet Base V2Summarization· MTEB Leaderboard data23.7%
40Jina Embeddings v2 Base ENSummarization· MTEB Leaderboard data6%