Skip to content

Benchmark

FinBen ECTSUM BartScore

Higher scores are better.

Models measured
7
Scores recorded
7
Sources
1
IQ weight
0.00

Score distribution

-5.1
Mean
-5.2
Median
0.13
Std dev
-5.2
10th percentile
-4.9
90th percentile
-5.2
Min
-5.2-4.9
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.00 for chat models, measured over 7 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Llama 2 70bpublished-demo-leaderboard, ECTSUM, BartScore· TheFinAI public numeric results-4.9
2gemini-1.0-propublished-demo-leaderboard, ECTSUM, BartScore· TheFinAI public numeric results-4.9
3DeepSeek V3published-demo-leaderboard, ECTSUM, BartScore· TheFinAI public numeric results-5.2
4GPT-3.5 Turbopublished-demo-leaderboard, ECTSUM, BartScore· TheFinAI public numeric results-5.2
5GPT-4 Turbopublished-demo-leaderboard, ECTSUM, BartScore· TheFinAI public numeric results-5.2
6Llama-2-7b-chat-hfpublished-demo-leaderboard, ECTSUM, BartScore· TheFinAI public numeric results-5.2
7Mixtral 8X7b V0.1published-demo-leaderboard, ECTSUM, BartScore· TheFinAI public numeric results-5.2

Data credits: The Fin AI. Published FinBen result summaries and Open Greek Financial LLM Leaderboard aggregates. Results licence not separately stated; fractional FinBen success metrics converted to percentages.