CoChat
Sign Up
PreprintCited by 4Financial Distress and Bankruptcy PredictionCustomer churn and segmentationMachine Learning in Healthcare

The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.

FinPT: Financial Risk Prediction with Profile Tuning on Pretrained Foundation Models

Open access
Yin, Yuwei, Yang, Yazheng, Yang, Jian, Liu, Qi·2023·arXiv
DOI 10.48550/arxiv.2308.00065arXiv 2308.00065
Open this paper in CoChat — free accountOpen PDF

Abstract

Financial risk prediction plays a crucial role in the financial sector. Machine learning methods have been widely applied for automatically detecting potential risks and thus saving the cost of labor. However, the development in this field is lagging behind in recent years by the following two facts: 1) the algorithms used are somewhat outdated, especially in the context of the fast advance of generative AI and large language models (LLMs); 2) the lack of a unified and open-sourced financial benchmark has impeded the related research for years. To tackle these issues, we propose FinPT and FinBench: the former is a novel approach for financial risk prediction that conduct Profile Tuning on large pretrained foundation models, and the latter is a set of high-quality datasets on financial risks such as default, fraud, and churn. In FinPT, we fill the financial tabular data into the pre-defined instruction template, obtain natural-language customer profiles by prompting LLMs, and fine-tune large foundation models with the profile text to make predictions. We demonstrate the effectiveness of the proposed FinPT by experimenting with a range of representative strong baselines on FinBench. The analytical studies further deepen the understanding of LLMs for financial risk prediction.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2308.00065

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Most existing financial benchmarks focus on static question-answering, such as FinQA, TAT-QA, and FinBench.

    “However, most existing benchmarks focus on static question-answering, such as FinQA, TAT-QA, and FinBench (Chen et al., 2021; Zhu et al., 2021; Yin et al., 2023).”

    From StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets? (2025) · cites this paper as Yin et al., 2023

    The citation to Yin et al. is offered for FinBench as an example of a static question-answering benchmark. The paper’s §4 (page unavailable) instead describes FinBench as a benchmark of financial-risk prediction datasets, and §5.5 identifies its tasks as binary classification, not question-answering. This does not assess the other two cited benchmarks. (LLM, conf=0.89)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Most existing financial benchmarks for LLM agents focus on static question-answering tasks and fail to capture the dynamics of realistic trading scenarios.

    “Most existing financial benchmarks focus on static question-answering, failing to capture the dynamics of real-market trading.”

    From StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets? (2025) · cites this paper as Yin et al., 2023

    The Introduction and §4 Benchmark discuss the lack of a unified financial-risk benchmark and describe FinBench’s default, fraud, and churn datasets. Searches for static question-answering benchmarks and realistic trading scenarios found no passage assessing whether existing benchmarks capture trading dynamics. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 41

  1. Dogu Araci. 2019. FinBERT: Financial Sentiment Analysis with Pre-trained Language Models. CoRR abs/1908.10063 (2019). arXiv:1908.10063 · arXiv
  2. Sercan Ö. Arik and Tomas Pfister. 2021. TabNet: Attentive Interpretable Tabular Learning. In AAAI 2021. AAAI Press, Virtual Event, 6679-6687.
  3. Jiaqi Bai, Zhao Yan, Jian Yang, Xinnian Liang, Hongcheng Guo, and Zhoujun Li. 2023. KnowPrefix-Tuning: A Two-Stage Prefix-Tuning Framework for Knowledge- Grounded Dialogue Generation. CoRR abs/2306.15430 (2023). arXiv:2306.15430 · arXiv
  4. Christopher M. Bishop. 2007. Pattern recognition and machine learning, 5th Edition. Springer, New York, NY, USA.
  5. Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, and Samuel Weinbach. 2022. GPT-NeoX-20B: An Open-Source Autore- gressive Language Model. CoRR abs/2204.06745 (2022). arXiv:2204.06745 · arXiv
  6. Vadim Borisov, Tobias Leemann, Kathrin Seßler, Johannes Haug, Martin Pawel- czyk, and Gjergji Kasneci. 2022. Deep neural networks and tabular data: A survey. IEEE Transactions on Neural Networks and Learning Systems 1, 1 (2022), 1-21.
  7. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877-1901.
  8. Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In SIGKDD 2016. ACM, San Francisco, CA, USA, 785-794.
  9. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Web- son, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yan- ping Huang, Andrew M. Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. 2022. Scaling Instruction-Finetuned Language Models. CoRR abs/2210.11416 (2022). arXiv:2210.11416 · arXiv
  10. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guil- laume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised Cross-lingual Representation Learning at Scale. In ACL 2020. Association for Computational Linguistics, Online Event, 8440-8451. https://doi.org/10.18653/v1/2020.acl-main.747 · DOI · Paper page
  11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT 2019. Association for Computational Linguistics, Minneapolis, MN, USA, 4171-4186.
  12. Matthew F Dixon, Igor Halperin, and Paul Bilokon. 2020. Machine learning in finance. Vol. 1170. Springer, New York, NY, USA.
  13. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, Lei Li, and Zhifang Sui. 2023. A Survey for In-context Learning. CoRR abs/2301.00234 (2023). arXiv:2301.00234 · arXiv
  14. John W Goodell, Satish Kumar, Weng Marc Lim, and Debidutta Pattnaik. 2021. Artificial intelligence and machine learning in finance: Identifying foundations, themes, and research clusters from bibliometric analysis. Journal of Behavioral and Experimental Finance 32 (2021), 100577.
  15. Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: A Factorization-Machine based Neural Network for CTR Prediction. In IJCAI 2017. ijcai.org, Melbourne, Australia, 1725-1731.
  16. Tin Kam Ho. 1995. Random decision forests. In Third International Conference on Document Analysis and Recognition, ICDAR 1995, Vol. 1. IEEE Computer Society, Montreal, Canada, 278-282.
  17. Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. LightGBM: A Highly Efficient Gradient Boosting Decision Tree. In NeurIPS 2017. NeurIPS, Long Beach, CA, USA, 3146-3154.
  18. Andy Liaw, Matthew Wiener, et al. 2002. Classification and Regression by Ran- domForest. R news 2, 3 (2002), 18-22.
  19. Zhuang Liu, Degen Huang, Kaiyu Huang, Zhuang Li, and Jun Zhao. 2020. Fin- BERT: A Pre-trained Financial Language Representation Model for Financial Text Mining. In IJCAI 2020. ijcai.org, Yokohama, Japan, 4513-4519.
  20. Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In ICLR 2019. OpenReview.net, New Orleans, LA, USA.
  21. Shuming Ma, Jian Yang, Haoyang Huang, Zewen Chi, Li Dong, Dongdong Zhang, Hany Hassan Awadalla, Alexandre Muzio, Akiko Eriguchi, Saksham Singhal, Xia Song, Arul Menezes, and Furu Wei. 2020. XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders. CoRR abs/2012.15547 (2020). arXiv:2012.15547 · arXiv
  22. Ryszard Stanislaw Michalski, Jaime Guillermo Carbonell, and Tom M Mitchell. 2013. Machine learning: An artificial intelligence approach. Springer Science & Business Media, New York, NY, USA.
  23. Noella Nazareth and Yeruva Venkata Ramana Reddy. 2023. Financial applications of machine learning: A literature review. Expert Syst. Appl. 219 (2023), 119640.
  24. OpenAI. 2023. GPT-4 Technical Report. CoRR abs/2303.08774 (2023). arXiv:2303.08774 · arXiv
  25. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Train- ing language models to follow instructions with human feedback. In NeurIPS 2022, Vol. 35. NeurIPS, New Orleans, LA, USA, 27730-27744.
  26. Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. Instruction Tuning with GPT-4. CoRR abs/2304.03277 (2023). arXiv:2304.03277 · arXiv
  27. Liudmila Ostroumova Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. 2018. CatBoost: unbiased boosting with categorical features. In NeurIPS 2018, Vol. 31. NeurIPS, Montréal, Canada, 6639-6649.
  28. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
  29. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
  30. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research 21, 1 (2020), 5485-5551.
  31. Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilic, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, Niklas Muen- nighoff, Albert Villanova del Moral, Olatunji Ruwase, Rachel Bawden, Stas Bek- man, Angelina McMillan-Major, Iz Beltagy, Huu Nguyen, Lucile Saulnier, Samson Tan, Pedro Ortiz Suarez, Victor Sanh, Hugo Laurençon, Yacine Jernite, Julien Launay, Margaret Mitchell, Colin Raffel, Aaron Gokaslan, Adi Simhi, Aitor Soroa, Alham Fikri Aji, Amit Alfassy, Anna Rogers, Ariel Kreisberg Nitzav, Canwen Xu, Chenghao Mou, Chris Emezue, Christopher Klamm, Colin Leong, Daniel van Strien, David Ifeoluwa Adelani, and et al. 2022. BLOOM: A 176B- Parameter Open-Access Multilingual Language Model. CoRR abs/2211.05100 (2022). arXiv:2211.05100 · arXiv
  32. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In NeurIPS 2017, Vol. 30. NeurIPS, Long Beach, CA, USA, 5998-6008.
  33. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022. Finetuned Language Models are Zero-Shot Learners. In ICLR 2022. OpenReview.net, Virtual Event.
  34. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824-24837.
  35. Yutaro Yamada, Ofir Lindenbaum, Sahand Negahban, and Yuval Kluger. 2020. Feature Selection using Stochastic Gates. In ICML 2020 (Proceedings of Machine Learning Research, Vol. 119). PMLR, PMLR, Virtual Event, 10648-10659.
  36. Jian Yang, Shuming Ma, Li Dong, Shaohan Huang, Haoyang Huang, Yuwei Yin, Dongdong Zhang, Liqun Yang, Furu Wei, and Zhoujun Li. 2023. GanLM: Encoder- Decoder Pre-training with an Auxiliary Discriminator. In ACL 2023. Association for Computational Linguistics, Toronto, Canada, 9394-9412.
  37. Jian Yang, Shuming Ma, Haoyang Huang, Dongdong Zhang, Li Dong, Shaohan Huang, Alexandre Muzio, Saksham Singhal, Hany Hassan, Xia Song, and Furu Wei. 2021. Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task. In WMT@EMNLP 2021. Association for Computational Linguistics, Online Event, 446-455.
  38. Jian Yang, Shuming Ma, Dongdong Zhang, Shuangzhi Wu, Zhoujun Li, and Ming Zhou. 2020. Alternating Language Modeling for Cross-Lingual Pre-Training. In AAAI 2020, IAAI 2020, EAAI 2020. AAAI Press, New York, NY, USA, 9386-9393.
  39. Yi Yang, Mark Christopher Siy Uy, and Allen Huang. 2020. FinBERT: A Pretrained Language Model for Financial Communications. CoRR abs/2006.08097 (2020). arXiv:2006.08097 · arXiv
  40. Jinsung Yoon, Yao Zhang, James Jordon, and Mihaela van der Schaar. 2020. Vime: Extending the success of self-and semi-supervised learning to tabular domain. Advances in Neural Information Processing Systems 33 (2020), 11033-11043.
  41. Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji, Qiben Yan, Lifang He, Hao Peng, Jianxin Li, Jia Wu, Ziwei Liu, Pengtao Xie, Caiming Xiong, Jian Pei, Philip S. Yu, and Lichao Sun. 2023. A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT. CoRR abs/2302.09419 (2023). arXiv:2302.09419 · arXiv

Cited by

Sorted by citations

  • StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets?

    Chen, Y., Yao, Z., Liu, Y., Xin, A., Ye, J., Yu, J., Hou, L., & Li, J. (2025) · arXiv · Cited by 62

    “However, existing financial benchmarks for LLM agents largely focus on static question-answering tasks (Chen et al., 2021; Zhu et al., 2021; Yin et al., 2023), which are designed to test the financial knowledge of LLMs but fail to capture the dynamics of realistic trading scenarios.”

Metadata

Authors
Yin, Y., Yang, Y., Yang, J., & Liu, Q.
Affiliations
  • University of Hong Kong (HK)
  • Alibaba Group (Cayman Islands) (KY)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
DOI
10.48550/arxiv.2308.00065
arXiv
2308.00065
OpenAlex
W4385507501
Cited by
4
References
1
Citations by year
2026: 1 · 2024: 3
Type
posted content
Language
en
Open access
Licence
Locations
  • RePEc: Research Papers in Economics (submittedVersion)
  • arXiv (Cornell University) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Business, Management and Accounting › Accounting › Financial Distress and Bankruptcy Prediction
  • Social Sciences › Business, Management and Accounting › Marketing › Customer churn and segmentation
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
financial risk prediction, large language models, foundation models, financial fraud detection, customer churn, tabular data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review