Skip to content

Benchmark

FinBen taiwan F1

Scores are percentages. Higher is better.

Models measured
16
Scores recorded
16
Sources
1
IQ weight
0.00

Score distribution

66.2%
Mean
95%
Median
37.3%
Std dev
17%
10th percentile
95.2%
90th percentile
0%
Min
0%95.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.00 for chat models, measured over 16 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1codellama-7bpublished-demo-leaderboard, taiwan, F1· TheFinAI public numeric results95.2%
2DeepSeek R1 Distill Qwen 1.5Bpublished-demo-leaderboard, taiwan, F1· TheFinAI public numeric results95.2%
3DeepSeek R1 Distill Qwen 7Bpublished-demo-leaderboard, taiwan, F1· TheFinAI public numeric results95.2%
4Qwen2.5 7B Instructpublished-demo-leaderboard, taiwan, F1· TheFinAI public numeric results95.2%
5Qwen2.5 7Bpublished-demo-leaderboard, taiwan, F1· TheFinAI public numeric results95.2%
6gemini-1.0-propublished-demo-leaderboard, taiwan, F1· TheFinAI public numeric results95%
7Llama-2-7b-chat-hfpublished-demo-leaderboard, taiwan, F1· TheFinAI public numeric results95%
8Mixtral 8X7b V0.1published-demo-leaderboard, taiwan, F1· TheFinAI public numeric results95%
9DeepSeek V3published-demo-leaderboard, taiwan, F1· TheFinAI public numeric results88.4%
10Mistral (7B) Instruct v0.3published-demo-leaderboard, taiwan, F1· TheFinAI public numeric results86.9%
11GPT-4 Turbopublished-demo-leaderboard, taiwan, F1· TheFinAI public numeric results55%
12code-llama-7b-instructpublished-demo-leaderboard, taiwan, F1· TheFinAI public numeric results31.1%
13GPT-3.5 Turbopublished-demo-leaderboard, taiwan, F1· TheFinAI public numeric results20%
14Llama 2 70bpublished-demo-leaderboard, taiwan, F1· TheFinAI public numeric results17%
15Llama 3.2 3Bpublished-demo-leaderboard, taiwan, F1· TheFinAI public numeric results0.2%
16Qwen2 7B Instructpublished-demo-leaderboard, taiwan, F1· TheFinAI public numeric results0%

Data credits: The Fin AI. Published FinBen result summaries and Open Greek Financial LLM Leaderboard aggregates. Results licence not separately stated; fractional FinBen success metrics converted to percentages.