Benchmark
FinBen EDTSUM BartScore
Higher scores are better.
Benchmark sourceScores from TheFinAI public numeric results.
- Models measured
- 7
- Scores recorded
- 7
- Sources
- 1
- IQ weight
- 0.65
Score distribution
- -3.9
- Mean
- -3.8
- Median
- 0.268
- Std dev
- -4
- 10th percentile
- -3.6
- 90th percentile
- -4.5
- Min
This benchmark has a weight of 0.65 for chat models, measured over 7 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | GPT-4 Turbopublished-demo-leaderboard, EDTSUM, BartScore· TheFinAI public numeric results | -3.6 |
| 2 | GPT-3.5 Turbopublished-demo-leaderboard, EDTSUM, BartScore· TheFinAI public numeric results | -3.6 |
| 3 | DeepSeek V3published-demo-leaderboard, EDTSUM, BartScore· TheFinAI public numeric results | -3.8 |
| 4 | Llama 2 70bpublished-demo-leaderboard, EDTSUM, BartScore· TheFinAI public numeric results | -3.8 |
| 5 | gemini-1.0-propublished-demo-leaderboard, EDTSUM, BartScore· TheFinAI public numeric results | -3.9 |
| 6 | Llama-2-7b-chat-hfpublished-demo-leaderboard, EDTSUM, BartScore· TheFinAI public numeric results | -4 |
| 7 | Mixtral 8X7b V0.1published-demo-leaderboard, EDTSUM, BartScore· TheFinAI public numeric results | -4.5 |