FinQA: A Dataset of Numerical Reasoning over Financial Data

“For all models, we use the Adam optimizer (Kingma and Ba, 2015).”

“Retriever The full page of the financial report can go beyond 2,000 tokens, which cannot be coped with the current popular QA models (Devlin et al., 2019).”

“, 2019) and RoBERTa (Liu et al., 2019), our proposed approach outperforms all other baselines and achieves an execution accuracy of 65.”

“3703 scales linearly with sequence length, like the Longformer (Beltagy et al., 2020).”

“Financial QA is more challenging than classic QA (Rajpurkar et al., 2018; Yang et al., 2018) because it requires the system to spot relevant information across heterogeneous sources, such as tables and unstructured texts, and then create a numerical reasoning path to connect all the information.”

“Financial QA is more challenging than classic QA (Rajpurkar et al., 2018; Yang et al., 2018) because it requires the system to spot relevant information across heterogeneous sources, such as tables and unstructured texts, and then create a numerical reasoning path to connect all the information.”

“This way the model is able to learn the program structures as patterns from very large-scale data (~40k for MathQA), but may fail on learning the reasoning paths.”

“The performance of using FinBert is no better than BERT-large, mostly because its pre-training corpus is limited (~30M words from news articles).”

“Another kind of QA datasets related to ours is the math word problem datasets, like MaWPS (Koncel-Kedziorski et al., 2016), MathQA (Amini et al.”

“• FinQA [13]: numerical reasoning over financial reports (241 documents, 564 queries).”

“An operational deployment integrating automated semantic annotation with minute-level audience measurement data, demonstrating the applicability of multimodal video understanding to content-based demographic analytics.”

“We report the breakdown of per-query cost on DocFinQA in Table 8.”

“Notably, recent evaluations indicate that advanced LLMs now surpass human performance on established financial reasoning datasets, including TAT-QA (Zhu et al. 2021), TabMWP (Zhu et al. 2021), FinQA (Chen et al. 2021), and DocMath (Zhao et al. 2023).”

“Harmful actions are operationalized according to domain-specific safety policies: medically un-safe or unsupported dosage recommendations in MedQA, financially noncompliant transactions in FinQA, and factually incorrect or unsupported statements in TruthfulQA.”

“These suites provide breadth and coverage but often entangle perception, alignment, and reasoning in a single score.”

“Later datasets such as HybridQA [3] and FinQA [4] further introduced cross-table textual reasoning and numerical computation, broadening the scope of Table QA tasks.”

“Furthermore, financial QA datasets such as FinQA provide realistic numerical reasoning templates that guide our aggregation and answer synthesis [10].”

“Chen et al. [122] pointed out that the financial domain includes complex numerical reasoning and understanding of heterogeneous representations.”

“Financial reports contain dispersed entities, metrics, events, and numerical evidence, which are important for long-document understanding and multi-step financial analysis (Chen et al., 2021; Reddy et al., 2024).”

“This section does not attempt to provide an exhaustive survey of French evaluation datasets.”

“…our theoretical conclusions, which include: (i) Math: MATH (Hendrycks et al., 2021), CHAMP (Mao et al., 2024); (ii) Reason: Theo-remQA (Chen et al., 2023), MMLU-Redux (Gema et al., 2025), GPQA (Rein et al., 2024); (iii) QA: NaturalQues-tions (Kwiatkowski et al., 2019), FinQA (Chen et al., 2021).”

“For completeness, we tried TabSQLify(Nahid and Rafiei, 2024) on FinQA’s table questions (by running their code on our data) and obtained 70% EM, as their method decomposes table queries but doesn’t incorporate text, missing some information, whereas our full system handles both.”

“Each instance contains a self-contained question q , gold answer a , short evidence field e , verbatim source spans r ( raw_evidence ), and document provenance d through a source-file identifier and URL.”

“• FinQA [16]: A large-scale dataset focusing on numerical reasoning over hybrid data (financial texts and tables).”

“For financial domain-specific challenges, FinQA [13] provides 8,281 question-answer pairs, designed for question-answering and numerical reasoning, uniquely including step-by-step reasoning programs.”

“Addressing more complex scenarios, FinQA [4] provides over 8000 high-quality question-answer pairs curated by financial experts.”

“And FinQA (financial reasoning QA) challenges numerical and compositional reasoning skills, combining textual context with quantitative information [12].”

“To address this, Chen et al. [27] introduced FinQA, a financial QA dataset grounded in banking scenarios, while Shi et al. [25] examined LLM reliability in legal text generation.”

“Existing FinNLP benchmarks also lack instruction-following evaluation, focusing instead on core NLP tasks like sentiment analysis (FOMC [23]), numerical QA (FinQA, [24]), and summarization (SubjECTive-QA, [25], [26], [27], [28], [29]).”

“In the financial domain, datasets like FinanceBench [7], FinQA [8], and ConvFinQA [9] have been introduced, with models such as BloombergGPT [10], instruction fine-tuned models [11], and the data-centric FinGPT [12] proposed.”

“In financial QA, for example, [2] demonstrates that domain grounding alone is insufficient without faithful multi-step reasoning.”

“To strengthen the empirical rigor of the study, the retrieval-and-generation pipeline was evaluated on the FinQA dataset [12], which includes finance-specific QA pairs from corporate filings and reports.”

“Specialized benchmarks such as MedQA [16], PubMedQA [17], LegalBench [18], and finance.”

“In terms of evaluation metrics, FinQA uses Execution Accuracy (EA) and Program Accuracy (PA) as its evaluation indicators [2]; TAT-QA adopts Exact Match (EM) and a Numeracy-focused F1 score as its core metrics for evaluating numerical understanding [13].”

“Finally, in lieu of task-aligned benchmarks, we use LLM-based evaluation on SQuAD (Rajpurkar et al., 2016) and FinQA (Chen et al., 2021), where three independent LLM judges score semantic accuracy and completeness of the predicted extractions.”

“NLP has become integral in financial automation tasks such as question answering (QA), sentiment analysis, headline classification, and summarization [2], [3], [4].”

“4.1 and test SMCS in three new OOD datasets, including Hu-manEval (Chen et al., 2021a), FinQA (Chen et al., 2021b), and LiveMathBench (Liu et al., 2025).”

“C. Quality Control a) Annotation Guideline: To ensure the quality of QA annotations, we first investigate QA pairs in published datasets [15], [16], [18], [19], [20] containing complex long-distance numerical reasoning questions that at least one step in the program relies on the calculation…”

“QA tasks are addressed by FinQA, which contains 8,281 QA pairs from S&P 500 annual reports [10] with ConvFinQA extending the dataset through conversational data [11].”

“When it comes to the finance domain, the performance of open-source LLMs on the FinQA dataset presents an interesting perspective on how these models handle tasks involving complex financial reasoning.”

“FinQA [17], ConvFinQA [18] and TAT-QA [19] emphasize numerical reasoning and hybrid table-text data.”

“…processing network designs specialised processing paths for different types of features such as fundamental, technical, macroeconomic and portfolio, which significantly improves the quality of feature representation and enhances the model’s ability to understand complex financial data [8-10].”

“• Finance : FinQA (8,000 financial QA pairs) [15], ECTSUM (500 earnings call transcripts) [16].”

“For the dataset, we used FinQA [1] and the Combined dataset [5].”

“…was for demonstration, which was rendered from the source HTML file. often concentrating on specific aspects like table-based question answering [7, 3, 8, 9, 10, 11] or information retrieval [12, 13, 1], and fail to represent the full spectrum of challenges that professionals encounter when…”

“An example of this that has seen wide adoption is FinQA by Chen et al. (2021), a large-scale dataset with question-answer pairs on financial reports, written by industry experts.”

“Simple percentages were generated with a focus on numerical data (Chen et al. , 2021).”

“This dataset contains observation data of multiple variables at different time points, usually with hourly or daily intervals [9].”

“† denotes the result reported in FinQA [3].”

“Recent research has focused on addressing the challenge of numerical reasoning over Table-and-Text hybrid data (Chen et al., 2021), which is a rich area for applications such as finan-cial analysis.”

“Regarding the embedding models, we employ text-embedding-3-large (OpenAI, 2024) for the FinanceReasoning, FinQA, and TAT-QA datasets.”

“…expensive human annotation, are typically limited to a domain ( e.g. financial) or language ( e.g. SQL), or are automatically constructed with LLMs (Chen et al., 2021b; Nan et al., 2022a; Kweon et al., 2023; Chen, 2023; Wu et al., 2025), which limits their application in unseen datasets at scale.”

“In contrast, FinQA shows a substantially lower score, suggesting that its domain-specific financial questions are less represented in the model’s training data.”

“…al., 2023; Webersinke et al., 2021; Araci, 2019; Luukkonen et al., 2023), question answering (Luccioni et al., 2020; Zhao et al., 2022; Xie et al., 2023; Chen et al., 2021; Zhu et al., 2021; Deng et al., 2022; Wu et al., 2025), and claim extraction for greenwashing detection (Mahdavi et al., 2024).”

“…diverse, meaningful numerical reasoning QA pairs from financial documents usually requires substantial manual effort, as seen in datasets like FinQA (Chen et al., 2021) and TAT-QA (Zhu et al., 2021), where financial experts spend considerable time crafting the multi-step reasoning questions.”

“…most prior benchmarks focus on clean or mildly structured tables (Pasupat and Liang, 2015; Iyyer et al., 2017; Chen et al., 2019; Nan et al., 2022; Chen et al., 2021; Katsis et al., 2022; Zhong et al., 2017; Yu et al., 2018; Li et al., 2023) and cover only a limited subset of real-world types (He…”

“Because evidence-sufficiency calibration is especially consequential in high-stakes specialized domains, we plan to extend ESB to medical (PubMedQA [31], BioASQ [32], MedQA), legal (CUAD [33]), financial (FinQA [34]), and education-oriented QA, as well as multilingual settings.”

“Such silent failure modes render these systems unreliable for critical applications in finance (Chen et al., 2021) and healthcare (Akhtar et al., 2022), where auditability is paramount.”

“For FINQA’s numerical-oriented tasks, we employ Exact Match (EM) accuracy, while using F1 scores for all other datasets.”

“Novelty of EviFiVQA Existing financial visual question answering (VQA) datasets, such as FinQA [21] and FinTabNetQA [15], have significantly advanced the field by introducing complex numerical reasoning and table-based question-answering tasks.”

“Method step GPT-4 experts Datasets used PoT [6] PE FinQA, TAT-QA, TabMWP EEDP [26] PE FinQA, TAT-QA, TabMWP Chameleon [19] PE TabMWP ToRA [11] FT TabMWP Husky [17] FT FinQA, TabMWP TAT-LLM [37] FT FinQA, TAT-QA FinQANet [7] FT FinQA TagOP [36] FT TAT-QA”

“ConvFinQA “Inthecontextofthisseriesofinterconnectedfinance-relatedqueriesandtheadditionalinformation providedbythepretext,tabledata,andposttextfromacompany’sfinancialfilings, pleaseprovidearesponsetothefinalquestion.”

“Dataset Split Example Domain Input Output GSM8K Cobbe et al. (2021) Test 1318 MWP Question Number AQuA Ling et al. (2017) Test 253 MWP Question Option SVAMP Patel et al. (2021) Test 1000 MWP Question Number MultiArith Roy & Roth (2015) Test 600 MWP Question Number TabMWP Lu et al. (2022) Test 7861 MWP Table + Question…”

“We source data from open-book Question-Answer (QA) datasets (CovidQA [27], PubmedQA [15], HotpotQA [42], MS Marco [29], CUAD [12], EManual [28], TechQA [3], FinQA [5], TAT-QA [47], ExpertQA [26], HAGRID [16]), as well one that was specifically adapted for RAG (DelucionQA [34]).”

“…et al., 2018], MS Marco [Nguyen et al., 2017], CUAD [Hendrycks et al., 2021a], EManual [Nandy et al., 2021], TechQA [Castelli et al., 2020], FinQA [Chen et al., 2021], TAT-QA [Zhu et al., 2021], ExpertQA [Malaviya et al., 2024], HAGRID [Kamalloo et al., 2023]), as well as one that was…”

“Reddy et al. (2024) extend FinQA (Chen et al., 2021b) to financial documents with an average of 123k words.”