CoChat
Sign Up
PreprintCited by 1Topic ModelingArtificial Intelligence ApplicationsStock Market Forecasting Methods

FinMaster: A Holistic Benchmark for Mastering Full-Pipeline Financial Workflows with LLMs

Open access
Jiang, Junzhe, Yang, Chang, Cui, Aixin, Jin, Sihan, Wang, Ruiyu, Li, Bo, Huang, Xiao, Sun, Dongning +1 more·2025·arXiv
DOI 10.48550/arxiv.2505.13533arXiv 2505.13533
Open this paper in CoChat — free accountOpen PDF

Abstract

Financial tasks are pivotal to global economic stability; however, their execution faces challenges including labor intensive processes, low error tolerance, data fragmentation, and tool limitations. Although large language models (LLMs) have succeeded in various natural language processing tasks and have shown potential in automating workflows through reasoning and contextual understanding, current benchmarks for evaluating LLMs in finance lack sufficient domain-specific data, have simplistic task design, and incomplete evaluation frameworks. To address these gaps, this article presents FinMaster, a comprehensive financial benchmark designed to systematically assess the capabilities of LLM in financial literacy, accounting, auditing, and consulting. Specifically, FinMaster comprises three main modules: i) FinSim, which builds simulators that generate synthetic, privacy-compliant financial data for companies to replicate market dynamics; ii) FinSuite, which provides tasks in core financial domains, spanning 183 tasks of various types and difficulty levels; and iii) FinEval, which develops a unified interface for evaluation. Extensive experiments over state-of-the-art LLMs reveal critical capability gaps in financial reasoning, with accuracy dropping from over 90% on basic tasks to merely 40% on complex scenarios requiring multi-step reasoning. This degradation exhibits the propagation of computational errors, where single-metric calculations initially demonstrating 58% accuracy decreased to 37% in multimetric scenarios. To the best of our knowledge, FinMaster is the first benchmark that covers full-pipeline financial workflows with challenging tasks. We hope that FinMaster can bridge the gap between research and industry practitioners, driving the adoption of LLMs in real-world financial practices to enhance efficiency and accuracy.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2505.13533

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Few benchmarks assess how AI models perform at real accounting work.

    “Few benchmarks assess how AI models perform at real accounting work.”

    From APEX-Accounting (2026) · cites this paper as Jiang et al., 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 26

  1. Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. · arXiv
  2. [2]Edward I Altman, Małgorzata Iwanicz-Drozdowska, Erkki K Laitinen, and Arto Suvas. Finan- cial distress prediction in an international context: A review and empirical analysis of altman's z-score model. Journal of international financial management & accounting, 28(2):131–171, 2017.
  3. [5]Miriam Bruhn, Dean Karlan, and Antoinette Schoar. The impact of consulting services on small and medium enterprises: Evidence from a randomized trial in mexico. Journal of Political Economy, 126(2):635–687, 2018.
  4. [6]Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, et al. Finqa: A dataset of numerical reasoning over financial data. arXiv preprint arXiv:2109.00122, 2021. · arXiv
  5. [7]Barry J Epstein, Ralph Nach, and Steven M Bragg. Wiley GAAP 2010: Interpretation and application of generally accepted accounting principles. John Wiley & Sons, 2009.
  6. [8]Jayne Godfrey, Allan Hodgson, Ann Tarca, Jane Hamilton, and Scott Holmen. Accounting. John Wiley & Sons, Inc, 2010.
  7. [9]Yifan Hu, Yuante Li, Peiyuan Liu, Yuxia Zhu, Naiqi Li, Tao Dai, Shu-tao Xia, Dawei Cheng, and Changjun Jiang. Fintsb: A comprehensive and practical benchmark for financial time series forecasting. arXiv preprint arXiv:2502.18834, 2025. · arXiv
  8. [10]Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen. Financebench: A new benchmark for financial question answering. arXiv preprint arXiv:2311.11944, 2023. · arXiv
  9. [11]Rik Koncel-Kedziorski, Michael Krumdick, Viet Lai, Varshini Reddy, Charles Lovering, and Chris Tanner. Bizbench: A quantitative reasoning benchmark for business and finance. arXiv preprint arXiv:2311.06602, 2023. · arXiv
  10. [12]Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. · arXiv
  11. [13]Xiao-Yang Liu, Guoxuan Wang, Hongyang Yang, and Daochen Zha. Fingpt: Democratizing internet-scale data for financial large language models. arXiv preprint arXiv:2307.10485, 2023. · arXiv
  12. [14]OpenAI. GPT-4 technical report. arXiv preprint arXiv:2303.08774, 2023. · arXiv
  13. 12th International Conference on Green Technology (ICGT 2022), pages 93–101. Atlantis Press, 2023.
  14. [16]Luis Paulo Guimarães dos Santos, Anderson José Freitas de Cerqueira, and César Valentim de Oliveira Carvalho. An experimental analysis of the effect of recordkeeping over direct reciprocity. Revista Contabilidade & Finanças, 32(86):359–375, 2020. https://github.com/BerriAI/litellm
  15. [17]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. · arXiv
  16. [18]Naomi S Soderstrom and Kevin Jialin Sun. Ifrs adoption and accounting quality: a review. European accounting review, 16(4):675–702, 2007.
  17. [19]Mark T Soliman. The use of dupont analysis by market participants. The accounting review, 83(3):823–853, 2008.
  18. [20]Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. · arXiv
  19. [21]Qianqian Xie, Weiguang Han, Zhengyu Chen, Ruoyu Xiang, Xiao Zhang, Yueru He, Mengxi Xiao, Dong Li, Yongfu Dai, Duanyu Feng, et al. Finben: A holistic financial benchmark for large language models. Advances in Neural Information Processing Systems, 37:95716–95743, 2024.
  20. [22]Qianqian Xie, Weiguang Han, Xiao Zhang, Yanzhao Lai, Min Peng, Alejandro Lopez-Lira, and Jimin Huang. Pixiu: A large language model, instruction data and evaluation benchmark for finance. arXiv preprint arXiv:2306.05443, 2023. · arXiv
  21. [23]Noga Ben Yoash, Meni Brief, Oded Ovadia, Gil Shenderovitz, Moshik Mishaeli, Rachel Lemberg, and Eitam Sheetrit. SECQUE: A benchmark for evaluating real-world financial analysis capabilities. arXiv preprint arXiv:2504.04596, 2025. · arXiv
  22. [24]Yangyang Yu, Zhiyuan Yao, Haohang Li, Zhiyang Deng, Yuechen Jiang, Yupeng Cao, Zhi Chen, Jordan Suchow, Zhenyu Cui, Rong Liu, et al. Fincon: A synthesized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making. Advances in Neural Information Processing Systems, 37:137010–137045, 2024.
  23. [25]Hanyu Zhang, Boyu Qiu, Yuhao Feng, Shuqi Li, Qian Ma, Xiyuan Zhang, Qiang Ju, Dong Yan, and Jian Xie. Baichuan4-finance technical report. arXiv preprint arXiv:2412.15270, 2024. · arXiv
  24. [26]Liwen Zhang, Weige Cai, Zhaowei Liu, Zhi Yang, Wei Dai, Yujie Liao, Qianru Qin, Yifei Li, Xingyu Liu, Zhiqiang Liu, et al. Fineval: A chinese financial domain knowledge evaluation benchmark for large language models. arXiv preprint arXiv:2308.09975, 2023. · arXiv
  25. [27]Wentao Zhang, Lingxuan Zhao, Haochong Xia, Shuo Sun, Jiaze Sun, Molei Qin, Xinyi Li, Yuqing Zhao, Yilei Zhao, Xinyu Cai, et al. Finagent: A multimodal foundation agent for financial trading: Tool-augmented, diversified, and generalist. arXiv e-prints, pages arXiv–2402, 2024.
  26. [28]Jie Zhu, Qian Chen, Huaixia Dou, Junhui Li, Lifan Guo, Feng Chen, and Chi Zhang. DianJin- R1: Evaluating and enhancing financial reasoning in large language models. arXiv preprint arXiv:2504.15716, 2025. · arXiv

Cited by

Sorted by citations

  • APEX-Accounting

    Benchek, J., Bennett, A., Kern, J., Stevens, R., Sultan, R., Ching, C., Popiel, H., Mittal, V., Mercier, F., Foody, B., et al. (2026) · arXiv · doi:10.48550/arxiv.2607.27189

    “FinMaster (Jiang et al., 2025) spans the financial pipeline with text-only, simulator-generated tasks, and AccountingBench (Penrose, 2025) evaluates agents on closing the books for one real SaaS company against a human CPA.”

Related

Authors
Jiang, J., Yang, C., Cui, A., Jin, S., Wang, R., Li, B., Huang, X., Sun, D., & Wang, X.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
DOI
10.48550/arxiv.2505.13533
arXiv
2505.13533
OpenAlex
W4415017594
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • RePEc: Research Papers in Economics (submittedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
Keywords
large language models, financial benchmarks, LLM evaluation, financial literacy, auditing, financial consulting, synthetic financial data, financial simulation, multi-step reasoning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review