V4FinBench: Benchmarking Tabular Foundation Models, LLMs, and Standard Methods on Corporate Bankruptcy Prediction
V4FinBench excels in corporate bankruptcy prediction using TabPFN and Llama-3-8B models.
Key Findings
Methodology
V4FinBench utilizes company data from the Visegrád Group, combining TabPFN and Llama-3-8B models for corporate bankruptcy prediction. The dataset includes over one million company-year records, covering 131 features and six prediction horizons. With imbalance-aware fine-tuning, TabPFN surpasses gradient boosting in F1-score and ROC-AUC.
Key Results
- TabPFN exceeds gradient boosting in F1-score and ROC-AUC for long-term predictions, especially beyond two years.
- Llama-3-8B lags behind gradient boosting in ROC-AUC across all horizons, with a larger F1-score gap at longer horizons.
- External evaluation on the American Bankruptcy Dataset shows V4FinBench-finetuned TabPFN outperforms vanilla TabPFN.
Significance
This study provides a large-scale, public benchmark dataset for corporate bankruptcy prediction, addressing the limitations of existing small datasets. By comparing TabPFN and Llama-3-8B models, it offers new insights into model performance under imbalanced data, contributing to the financial prediction field.
Technical Contribution
V4FinBench offers a large-scale public dataset for financial prediction and demonstrates TabPFN's advantage in long-term predictions through imbalance-aware fine-tuning. The study also verifies the model's transferability across different datasets.
Novelty
V4FinBench is the first to provide a large-scale corporate bankruptcy prediction dataset, combining multi-horizon prediction and imbalance data handling, showcasing TabPFN's potential in long-term predictions.
Limitations
- The dataset is limited to four Central European countries and may not apply to other regions.
- Bankruptcy labels are based on financial distress criteria, not legal bankruptcy.
- Model performance under extreme imbalance still requires further validation.
Future Work
Future research could expand to datasets from other regions and explore more fine-tuning strategies to enhance model applicability across different economic contexts.
AI Executive Summary
Corporate bankruptcy prediction is a crucial task in finance, yet existing public datasets are small, limiting model evaluation and application. To address this, researchers introduced V4FinBench, a dataset containing over one million company-year records from four Central European countries. This dataset supports multi-horizon prediction and provides a benchmark for model evaluation under imbalanced data.
In the study, TabPFN, through imbalance-aware fine-tuning, excelled in long-term predictions, surpassing traditional gradient boosting methods. Meanwhile, Llama-3-8B lagged behind in ROC-AUC across all horizons, highlighting its limitations on structured data. External evaluation on the American Bankruptcy Dataset confirmed the transferability of the V4FinBench-finetuned TabPFN.
While V4FinBench offers a significant benchmark for corporate bankruptcy prediction, its limitations include geographic scope and label definition. Future research could expand to datasets from other regions and explore more fine-tuning strategies to enhance model applicability and accuracy.
Deep Analysis
Background
Corporate bankruptcy prediction is vital in finance, yet existing public datasets are small, limiting model evaluation and application. Common datasets like the UCI Polish Companies Bankruptcy dataset and the Taiwanese Bankruptcy Prediction dataset are only a few thousand to tens of thousands of records, inadequate for training and evaluating large-scale models.
Core Problem
Existing corporate bankruptcy prediction datasets are small, limiting the training and evaluation of large-scale models. Additionally, multi-horizon prediction and data imbalance add to the prediction challenge.
Innovation
V4FinBench provides a large-scale corporate bankruptcy prediction dataset, supporting multi-horizon prediction and demonstrating TabPFN's advantage in long-term predictions through imbalance-aware fine-tuning. This dataset fills the gap left by existing small datasets.
Methodology
- �� The dataset is sourced from company data of the Visegrád Group, containing over one million company-year records.
- �� TabPFN and Llama-3-8B models are used for prediction, with imbalance-aware fine-tuning and QLoRA fine-tuning, respectively.
- �� Experiments use five-fold cross-validation, with evaluation metrics including F1-score and ROC-AUC.
Experiments
Experiments use the V4FinBench dataset, covering six prediction horizons. Baseline models include gradient boosting, logistic regression, etc. TabPFN is fine-tuned with imbalance-aware strategies, and Llama-3-8B is fine-tuned with QLoRA. Evaluation metrics are F1-score and ROC-AUC.
Results
TabPFN exceeds gradient boosting in F1-score and ROC-AUC for long-term predictions, especially beyond two years. Llama-3-8B lags behind gradient boosting in ROC-AUC across all horizons, with a larger F1-score gap at longer horizons.
Applications
V4FinBench can be used to evaluate and develop new corporate bankruptcy prediction models, particularly in scenarios with imbalanced data and multi-horizon predictions. Its scale and diversity make it suitable for financial institutions and researchers.
Limitations & Outlook
V4FinBench's geographic scope is limited to four Central European countries and may not apply to other regions. Additionally, bankruptcy labels are based on financial distress criteria, not legal bankruptcy. Model performance under extreme imbalance still requires further validation.
Plain Language Accessible to non-experts
Imagine you're in a large shopping mall with many stores, each with its own financial status. V4FinBench is like a big database recording the financial information of these stores. Researchers use this database to predict which stores might go bankrupt. Just like you walk around the mall observing customer numbers, inventory, and sales, researchers analyze this data to assess the stores' business status. TabPFN and Llama-3-8B are like two smart assistants helping researchers make predictions faster and more accurately.
ELI14 Explained like you're 14
Imagine you're playing a simulation game where you manage companies in a virtual city. Each company has its financial data, like income, expenses, and profits. V4FinBench is like a super tool in the game, helping you collect and analyze this data. Researchers use this tool to predict which companies might go bankrupt, just like you need to decide which companies need more attention and support in the game. TabPFN and Llama-3-8B are like two super assistants in the game, helping you make decisions faster to ensure your city's economy thrives.
Glossary
TabPFN
A model for tabular data prediction, capable of in-context learning.
Used as one of the models for corporate bankruptcy prediction, improved through imbalance-aware fine-tuning.
Llama-3-8B
A large language model fine-tuned using QLoRA for corporate bankruptcy prediction.
Compared with TabPFN in experiments to evaluate its performance on structured data.
ROC-AUC
A metric for evaluating classification model performance, with higher values indicating better performance.
Used to assess the performance of TabPFN and Llama-3-8B models in bankruptcy prediction.
F1-score
A classification performance metric that considers both precision and recall.
Used in experiments to evaluate model performance under imbalanced data.
QLoRA
A method for fine-tuning large language models, reducing computational costs.
Used to fine-tune the Llama-3-8B model for the corporate bankruptcy prediction task.
Open Questions Unanswered questions from this research
- 1 How to apply V4FinBench's dataset and models in other regions?
- 2 How to further improve model performance under extreme imbalance?
- 3 How to incorporate legal bankruptcy information to enhance label accuracy?
Applications
Immediate Applications
Financial Institution Risk Assessment
Financial institutions can use the V4FinBench dataset and models to assess corporate bankruptcy risk and optimize credit decisions.
Long-term Vision
Global Financial Prediction Standard
V4FinBench could serve as a global standard for financial prediction, promoting cross-border financial risk assessment.
Abstract
Corporate bankruptcy prediction is a high-stakes financial task characterized by severe class imbalance and multi-horizon forecasting demands. Public datasets supporting it remain scarce and small: widely used free benchmarks contain between 6,000 and 80,000 company-year observations, while larger resources are behind subscription paywalls. To address this gap, we introduce V4FinBench, a benchmark of over one million company-year records from the Visegràd Group (V4) economies (2006-2021), with 131 financial and non-financial features, six prediction horizons, and a composite distress criterion jointly capturing solvency, profitability, and liquidity deterioration. V4FinBench is designed to support the evaluation of tabular and foundation-model methods under realistic class imbalance, with positive rates between 0.19% and 0.36%. We provide reference evaluations of standard tabular baselines, finetuned TabPFN, and QLoRA-finetuned Llama-3-8B. With imbalance-aware finetuning, TabPFN matches or exceeds gradient boosting at longer time horizons on both $F_1$-score and ROC-AUC. In contrast, Llama-3-8B trails gradient boosting on ROC-AUC at every horizon and is generally weaker on $F_1$-score, with the gap widening sharply beyond the immediate horizon. In an external evaluation on the American Bankruptcy Dataset, the V4FinBench-finetuned TabPFN checkpoint improves over vanilla TabPFN, suggesting that adaptation captures transferable financial-distress structure rather than only V4-specific patterns. V4FinBench is publicly released to support further evaluation and development of prediction methods on realistic financial data.