Skip to content

Benchmark

ChemTEB(v1.1) - PairClassification

Embedding benchmarks measure how well models find related items, for example when matching a search to relevant results. Scores are percentages.

Models measured
27
Scores recorded
27
Sources
1
IQ weight
0.76

Score distribution

57.9%
Mean
56.7%
Median
3.7%
Std dev
54.3%
10th percentile
63.1%
90th percentile
52.4%
Min
52.4%64.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.76 for embedding models, measured over 27 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Text Embedding 3 LargePairClassification· MTEB Extra Text Boards data64.9%
2Qwen3 Embedding 4BPairClassification· MTEB Extra Text Boards data64.8%
3Qwen3 Embedding 8BPairClassification· MTEB Extra Text Boards data63.6%
4Jina Embeddings v5 Text SmallPairClassification· MTEB Extra Text Boards data63.1%
5Jina Embeddings v5 Omni NanoPairClassification· MTEB Extra Text Boards data62.3%
6Titan Text Embeddings V2PairClassification· MTEB Extra Text Boards data61.6%
7Text Embedding 3 SmallPairClassification· MTEB Extra Text Boards data61.1%
8Qwen3 Embedding 0.6BPairClassification· MTEB Extra Text Boards data60.7%
9Multilingual E5 Large InstructPairClassification· MTEB Extra Text Boards data59.6%
10Titan Embeddings G1 - TextPairClassification· MTEB Extra Text Boards data59.4%
11Text Embedding Ada 002PairClassification· MTEB Extra Text Boards data58.6%
12BGE Large En V1.5PairClassification· MTEB Extra Text Boards data58%
13Multilingual E5 LargePairClassification· MTEB Extra Text Boards data57%
14BGE Base En V1.5PairClassification· MTEB Extra Text Boards data56.7%
15bge-large-enPairClassification· MTEB Extra Text Boards data55.9%
16nomic-embed-text-v1.5PairClassification· MTEB Extra Text Boards data55.8%
17E5 Large V2PairClassification· MTEB Extra Text Boards data55.6%
18bge-small-en-v1.5PairClassification· MTEB Extra Text Boards data55.4%
19BGE M3PairClassification· MTEB Extra Text Boards data55.4%
20Embed Multilingual V3.0PairClassification· MTEB Extra Text Boards data55.3%
21E5 Base V2PairClassification· MTEB Extra Text Boards data55%
22nomic-embed-text-v1PairClassification· MTEB Extra Text Boards data55%
23All MiniLM L12 V2PairClassification· MTEB Extra Text Boards data54.4%
24Embed English V3.0PairClassification· MTEB Extra Text Boards data54.3%
25All MiniLM L6 V2PairClassification· MTEB Extra Text Boards data53.7%
26All Mpnet Base V2PairClassification· MTEB Extra Text Boards data52.9%
27Multi Qa Mpnet Base Dot V1PairClassification· MTEB Extra Text Boards data52.4%