Skip to content

Benchmark

FinBen BigData22 MCC

Higher scores are better.

Models measured
15
Scores recorded
15
Sources
1
IQ weight
0.00

Score distribution

0.028
Mean
0.025
Median
0.041
Std dev
-0.023
10th percentile
0.066
90th percentile
-0.025
Min
-0.0250.142
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.00 for chat models, measured over 15 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Mistral (7B) Instruct v0.3published-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results0.142
2DeepSeek R1 Distill Qwen 1.5Bpublished-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results0.0655
3DeepSeek V3published-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results0.0632
4Llama-2-7b-chat-hfpublished-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results0.05
5gemini-1.0-propublished-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results0.04
6GPT-4 Turbopublished-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results0.03
7Qwen2.5 7B Instructpublished-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results0.0261
8Llama 3.2 3Bpublished-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results0.0248
9Mixtral 8X7b V0.1published-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results0.02
10Qwen2 7B Instructpublished-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results0.01
11Qwen2.5 7Bpublished-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results0.0055
12Llama 2 70bpublished-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results0
13codellama-7bpublished-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results-0.01
14DeepSeek R1 Distill Qwen 7Bpublished-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results-0.0233
15GPT-3.5 Turbopublished-demo-leaderboard, BigData22, MCC· TheFinAI public numeric results-0.025

Data credits: The Fin AI. Published FinBen result summaries and Open Greek Financial LLM Leaderboard aggregates. Results licence not separately stated; fractional FinBen success metrics converted to percentages.