FinQA: A Dataset of Numerical Reasoning over Financial Data

“For all models, we use the Adam optimizer (Kingma and Ba, 2015).”

“Retriever The full page of the financial report can go beyond 2,000 tokens, which cannot be coped with the current popular QA models (Devlin et al., 2019).”

“, 2019) and RoBERTa (Liu et al., 2019), our proposed approach outperforms all other baselines and achieves an execution accuracy of 65.”

“3703 scales linearly with sequence length, like the Longformer (Beltagy et al., 2020).”

“Financial QA is more challenging than classic QA (Rajpurkar et al., 2018; Yang et al., 2018) because it requires the system to spot relevant information across heterogeneous sources, such as tables and unstructured texts, and then create a numerical reasoning path to connect all the information.”

“Financial QA is more challenging than classic QA (Rajpurkar et al., 2018; Yang et al., 2018) because it requires the system to spot relevant information across heterogeneous sources, such as tables and unstructured texts, and then create a numerical reasoning path to connect all the information.”

“This way the model is able to learn the program structures as patterns from very large-scale data (~40k for MathQA), but may fail on learning the reasoning paths.”

“The performance of using FinBert is no better than BERT-large, mostly because its pre-training corpus is limited (~30M words from news articles).”

“Another kind of QA datasets related to ours is the math word problem datasets, like MaWPS (Koncel-Kedziorski et al., 2016), MathQA (Amini et al.”

“Recent progress includes FinGPT [25], BloombergGPT [57] with domain-adapted tokenization, and FinQA [5] featuring numerical reasoning.”

“…incorrect entity references, temporal inconsistencies, and relation mismatches. onomy, we introduce tagged hallucinations into answers from FinQA (Chen et al., 2021) and TAT-QA (Zhu et al., 2021) - two benchmark datasets requiring multi-step numerical reasoning over financial tables and text.”

“This involves using the analyst agent and the calculator agent to get the final answer (see Section 3.3 for Table 1: Accuracy of different approaches for TATQA and FinQA.”

“RAG and Applications: RAG systems combine LLMs with ex-ternal retrieval to improve factual accuracy by leveraging domain-specific information [15, 17], particularly useful in high-stakes domains like finance [6, 31].”

“Financial NLP Financial NLP is a rapidly evolving research area, with a number of advances achieved in a number of tasks such as sentiment analysis (Shah et al., 2023; Malo et al., 2014), question answering (Chen et al., 2021b, 2022), and financial text summarization (Mukherjee et al., 2022).”

“Fin-22549”

“While grounded reasoning datasets exist in the unimodal literature (Chen et al., 2021; Zhang et al., 2021) and in open-domain VQA (Zhu et al., 2016; Pont-Tuset et al., 2020), such datasets are yet to be popularized in multi-modal document understanding.”

“Efficient Supporting Fact Annotation Previous work (Chen et al., 2020a, 2021) required annotators to manually write down the column and row indices of all relevant table cells (i.e., supporting fact), which is time-consuming and might introduce typos.”

“Additionally, the finance domain frequently employs tables to represent data (Zhu et al., 2021; Chen et al., 2021; Zhao et al., 2022; Li et al., 2022b,a; Zhao et al., 2023b,d), which adds another layer of complexity to the knowledge-intensive problem-solving.”

“FinQA [Chen et al., 2021] comprises 8K+ finance QA instances; the task is numerical reasoning over financial context, and we score answer correctness with math-verify [Kydlíˇcek et al., 2025] under a 0.001 tolerance.”

“…we use held-out benchmarks spanning all six domains: MATH-500 [19], AIME [2] (math); HumanEval [7], MBPP [4] (codegen); zebra puzzles [32], ARC-AGI [12, 13] (logic); CodeI/O [27], CruxEval [16] (simulation); HiTab [10], MultiHierTT [63], FinQA [9] (table); GPQA-Diamond [46], SuperGPQA [14] Models.”

“FinQA [31] evaluates numerical reasoning over financial reports, while DocFinQA [32] studies long-context reasoning over document-level financial data.”

“This shows that the policy model’s own activations encode a compact signal about prompt difficulty and policy-specific uncertainty, which can be leveraged for value estimation at negligible cost.”

“Concurrently, English financial NLP benchmarks such as FiQA [13], TAT-QA [37], FinQA [1], ConvFinQA [2], FinanceMath [35], and FinChain [31] have supported progress in sentiment analysis, text–table reasoning, numerical computation, and verifiable reasoning.”

“The core contribution of SAPLMA(Azaria and Mitchell, 2023) posits that LLMs often possess a form of self-awareness , in the sense that even when generating hallucinated or incorrect outputs, their hidden states typically retain information about answer correctness.”

“FinQA [1] tests arithmetic calculations given table excerpts.”

“…(Pasupat and Liang, 2015; Iyyer et al., 2017; Chen et al., 2019); later efforts incorporated tasks requiring complex mathematical reasoning (Chen et al., 2021; Kat-sis et al., 2021); and recent benchmarks extend to multimodal settings that integrate tables with other modalities (Talmor et…”

“In the financial domain, analysts leverage expert knowledge to extract meaningful insights from diverse financial data, facilitating logical and informed decision-making, like reasoning process [4, 10, 15].”

“In practice, tables appear in both structured formats with rigid schemas and executable SQL queries (Zhong et al., 2017; Li et al., 2023; Wu et al., 2025a), and semi-structured formats where columns are irregular and cells contain free text (Chen et al., 2021; 2020; Singh et al., 2025).”

“In financial domain, benchmarks like FLUE [37], CFLEB [30], and DISC-FinLLM [9] evaluate NLP capabilities, while FinEval [53] ,investorbench [21] and FinQA [10] focus on financial analysis and knowledge.”

“Context Understanding Benchmarks The development of robust and comprehensive benchmarks for evaluating long context understanding has gained increasing attention in recent research (Li et al., 2024; Liu et al., 2024; Bai et al., 2023b; Zhang et al., 2024c; Wu et al., 2025; Chen et al., 2020, 2021).”

“While these datasets are not finance-specific, they have potential applications in finance.”

“Recently, tasks combined with tabular data and unstructured text have gained traction (Chen et al., 2021; Watson et al., 2024; Chen et al., 2022; Zhu et al., 2021; Chen et al., 2024a; Zhao et al., 2022; Liu et al., 2023a; Deng et al., 2022).”

“…[16]) (iii) Complexity due to ambiguity in questions (AmbigQA [36]) and (iv) complexity due to reasoning over different modalities / heterogeneous sources such as table and text (TAT-QA [67], OTT-QA [32]) and (v) complexity due to numerical reasoning over heterogeneous sources (FinQA [6]).”

“Researchers have applied FinNLP on various languages, extending beyond English (Chen et al., 2021, 2020a) to encompass Japanese (Kannan and Seki, 2023), Chinese (Tseng et al., 2023) and multiple European languages like Danish, Spanish, Turkish, etc. (Jør-gensen et al., 2023).”

“Among the hybrid question-answering datasets, HybridQA (Chen et al., 2020) and OT-TQA (Chen et al., 2021a) are based on Wikipedia data, and TATQA (Zhu et al., 2021), FinQA (Chen et al., 2021b) and MultiHiertt (Zhao et al., 2022) on financial data.”

“The source of complexity in complex QA tasks can be due to a complex question structure [27, 49], the need to perform comparisons or aggregations [19, 29], mathematical operations [42], or having to extract from complex objects like tables and text [7].”

“…(Chen et al., 2020c): consists of diversified sentences across symbolic operations (such as max, min, etc.); (Chen et al., 2020d; Zhu et al., 2021; Chen et al., 2021; Zhao et al., 2022a) introduce table and text based QA tasks that primarily involve numerical and table reasoning along with…”

“In this paper, we propose a DeBERTa v3 large-based (He et al., 2020) model for financial question and answer task in FinQA dataset (Chen et al., 2021).”

“As shown in Table 1, unlike prior work [14, 23, 24] that frames tagging as flat multi-class classification and focuses solely on narrative text, FinTagging requires models to jointly extract financial facts and align them with the US-GAAP taxonomy across both unstructured text and structured tables.”

“Bench-marks such as FinQA [Chen et al., 2021], TAT-QA [Zhu et al., 2021], and FinanceBench [Islam The most comprehensive effort to date is FinBen [Xie et al., 2024], which aggregates 42 datasets across 24 financial NLP tasks organized into seven categories.”

“We employ the FinQA dataset [6], which requires the agent to retrieve and process complex financial reports to answer expert-level queries.”

“Short gaps of up to five consecutive trading days are forward-filled using the most recent available observation.”

“FinQA (Chen et al., 2021) × × × ✓ × ConvFinQA (Chen et al., 2022) ✓ × × ✓ × FiQA-SA (Maia et al., 2018) × × ✓ ✓ × FPB Malo et al. (2014) × × ✓ ✓ × ACL18 (Xu and Cohen, 2018) (Li et al., 2025) ✓ ✓ × × ✓ Fintrade (Xie et al., 2024) Table 1: Comparison with existing Financial Benchmarks.”

“In the financial domain, FinQA [29] provides expert-annotated question-program pairs over earnings reports, TAT-QA [30] focuses on numerical operations over hybrid tabular-textual contexts, and ConvFinQA [31] extends FinQA to multi-turn reasoning.”

“For QA tasks, we adopt four benchmark datasets: WikiTQ (Pasupat & Liang, 2015), TabMWP (Lu et al., 2022), FinQA (Chen et al., 2021), and TAT-QA (Zhu et al., 2021).”

“Existing benchmarking datasets such as FinQA (Chen et al., 2022a), ConvFinQA (Chen et al., 2022b), and TAT-QA (Zhu et al., 2021) focus primarily on numerical reasoning over financial reports and tables.”

“(7) FinQA : a dataset requiring models to perform complex numerical reasoning and calculations over unstructured text and tables found in financial reports [3].”

“…into rows and columns, serving as a fundamental medium for conveying relational data across numerous domains such as financial reports, scientific articles, medical records and government documents (Chen et al., 2021; Akhtar et al., 2022; Cheng et al., 2022; Si et al., 2023; Li et al., 2024).”

“FinQA (Chen et al., 2021) extracts tables from financial market reports and provides a dataset for numerical calculation (Su et al., 2024; Loukas et al., 2022; Khang et al., 2024) tasks.”

“Finally, activating the Misleading Reward (MR) completes the MIRAGE pipeline (Row 6).”

“A common approach to tackle numerical reasoning problem is the retriever-generator question-answering framework, initially proposed by Chen et al. (2021b).”

“WeevaluateVeritasFiacross4financialQAdatasets: FinanceBench reference chunks from financial filings; FinQA [8], from which we use In-House Company Dataset with multi-entity (e.g., Lotus, Zeekr)”

“For instance, datasets such as (Chen et al., 2021, 2024; Zhu et al., 2021; Lai et al., 2024; Reddy et al., 2024; Yuan et al., 2024) focus on question answering over financial reports.”

“To evaluate our retrieval capabilities, we use the FinQA dataset (Chen et al., 2021), a benchmark designed for financial question answering.”

“Recent studies have reported strong performance for the decision-making of these systems in domains such as healthcare [15, 16], finance [17, 18], and strategic games [19–22].”

“Our task (b) targets the unknown-context setting, requiring retrieval from mixed text-tables before answering. formance answering numerical and free-form question-answering (QA) tasks when appropriate documents are provided (Nan et al., 2021; Chen et al., 2021, 2022; Zhu et al., 2021, 2022).”

“Benchmarks like HybridQA, TabFact, ToTTo, WikiTQ, and others (Chen et al., 2020, 2019; Parikh et al., 2020; Aly et al., 2021; Chen et al., 2021; Pasupat & Liang, 2015) focus on simple fact extractions and He et al. (2024b); Sui et al. (2024) cover more advanced analysis that still rely heavily on…”

“First, we primarily evaluated the inference capabilities of LLMs on qualitative information of earning call transcripts and did not dive into the mathematical reasoning, which is crucial in financial NLP (Chen et al., 2021; 2022; Krumdick et al., 2024; Srivas-tava et al., 2024).”

“However, QA-focused approaches like FinQA [20], TAT-QA [21], and MULTIHIERTT [22] only analyze specific sections within HLDs.”

“Datasets We apply E NCORE on five datasets with various settings: FinQA (Chen et al., 2021), ConvFinQA (Chen et al., 2022b), TAT-QA (Zhu et al., 2021a), MathQA (Amini et al., 2019) and DROP (Dua et al., 2019), which cover different types of evidence and answers.”

“…2021; Pasupat and Liang, 2015a; Abbas et al., 2016; Sun et al., 2016; Chen et al., 2021a, 2020b; Lin et al., 2020; Zayats et al., 2021; Oguz et al., 2022, and others) and table-to-text generation (Zhang et al., 2020c; Parikh et al., 2020; Nan et al., 2021; Yoran et al., 2022; Chen et al., 2021b).”

“answering [2], [3], [4], dialog [5], report generation [6], [7], [8], [9], and biographical description [10], demonstrating the great potential of table-to-text generation for use in extensive real-world scenarios.”

“Some representative instances are question answering [2]–[4], dialog [5], report generation [6]–[8], and biographical description [9], demonstrating the great potential of table-to-text generation to use in extensive real-world scenarios.”

“The effectiveness of the generator of FinQANet has been verified in Chen et al. (2021) by many comparative experiments, and Unilm also has achieved superior performance on Natural language understanding and generation tasks.”

“Existing work includes question answering (Liu et al., 2020), numerical reasoning on financial reports (Chen et al., 2021) and port-folio selection (Liang et al., 2021), amongst others.”

“First, the tables cannot always be processed by relational databases since recent MRC datasets often utilize web-crawled semi-structured tables without clearly defined relations (Zhu et al., 2021; Zhao et al., 2022b; Chen et al., 2021).”

“Response-level and financial QA benchmarks evaluate whether an advisor’s answer is accurate, helpful, or compliant, but they stop at the generated response and do not follow the user’s subsequent decision trajectory (Wu et al., 2023; Islam et al., 2023; Chen et al., 2021; Xie et al., 2024).”

“FinQA [7] and TAT-QA [8] focus on numerical and tabular reasoning over financial reports, while ConvFinQA adds a conversational setting in which systems must track evidence across turns [9].”

“FinQA (Chen et al., 2021) and ConvFinQA (Chen et al., 2022) pose numerical questions over single financial reports and their tabular content, with the latter requiring conversational multi-turn reasoning.”

“Early efforts such as FinQA (Chen et al., 2021) and ConvFinQA (Chen et al., 2022) established standards for multi-step arithmetic reasoning over financial reports, while TAT-QA (Zhu et al., 2021) introduced challenges requiring reasoning across hybrid tabular and textual data.”

“In more specialized settings, FinQA [24] targets numerical reasoning over financial documents, SheetBench [85] addresses spreadsheet manipulation, and InterCode [168] standardizes interactive coding with execution feedback, allowing agents to debug and refine scripts during task solving.”

“FinQA [Chen et al., 2021] and TAT-QA [Zhu et al., 2021] specifically address question answering over financial documents with tables, but focus primarily on public company filings rather than the complex spreadsheet models used in private markets.”

“…2025), OpenMathInstruct-2 (Toshniwal et al., 2024), Ultrachat (Ding et al., 2023), DISC-Law-SFT (Yue et al., 2023), SynthLaw (Yue et al., 2025), FinQA (Chen et al., 2021), FinCoT (Qian et al., 2025), ConvFinQA (Chen et al., 2022), SMART’s Trajectory Dataset (Yue et al., 2024a), COIG (Zhang et…”

“• Finance: We employ F IN QA (Chen et al., 2021), which requires numerical reasoning over financial reports.”

“Their effectiveness relative to semantic vector search remains unclear for complex financial queries requiring multi-hop reasoning across lengthy regulatory filings (Islam et al., 2023; Chen et al., 2021; Zhu et al., 2021).”

“Recent work on table-to-text generation includes TAPAS [41] for question answering over tables and GPT-3-based approaches for numerical reasoning [42].”

“…setup involving only retrieval questions (Katsis et al., 2022; Liu et al., 2023a; Wang et al., 2025a) while the complex setup involves numerical (Chen et al., 2021; Zhu et al., 2021; Lu et al., 2022; Tian et al., 2025a), commonsense (Zhang et al., 2023), temporal (Gupta et al., 2023;…”

“A typical RAG system comprises two main components: a retriever [12, 32, 78] that searches an external knowledge base for relevant information, and an LLM that synthesizes this retrieved context with the user query to generate the final response.”

“Existing financial evaluation benchmarks, such as FinQA (Chen et al. 2021), FinBen (Xie et al. 2024), FinTextQA (Chen et al. 2024a), and FinDER (Choi et al. 2025), are designed to assess the performance of LLMs on a diverse range of tasks.”

“For the open-source portion, extensive QA pairs were gathered from publicly available datasets, including FinCorpus [31], Finance-Instruct-500K [6], FinCUGE [17], FinQA [2], FinancialData [4] and Quant-Trading-Instruct [18].”

“Fine-tuning-based: Methods adapt models via supervised fine-tuning with curated QA/classification datasets (Ma et al., 2023; Harsha et al., 2025) or continued pretraining on domain-specific corpora (Yang et al., 2020; Lee et al., 2024; Bhatia et al., 2024; Ke et al., 2025; Liu et al., 2025).”

“• FinQA (Chen et al., 2021) The Zero-Shot prompt used for FinQA is given in Figure 5.”

“These gains further transfer to other tabular reasoning benchmarks (TAT-QA, FinQA, and TabMWP), with an average improvement of 4%(Table 12), particularly on complex tables, demonstrating the effectiveness of controllable data generation.”

“Bio-Medical-Llama-3-8B [59] Financial FINQA [60] FPB [61], Flare-cfa [62], MMLU-Financial [58] Llama-3-8B-Instruct-Finance [63] Mathematical MetaMathQA [64] GSM8K [65], MATH500 [66], MMLU-Math [58] Qwen2.”

“Notably, while efforts have been directed towards releasing datasets across prominent and expansive domains such as Medicine (Pal et al., 2022; Pampari et al., 2018), Finance (Chen et al., 2021; Zhu et al., 2021), and Legal (Zhong et al., 2019; Chen et al., 2023a), there remains an apparent scarcity of such datasets i…”

“However, existing research has predominantly focused on assessing financial LLMs’ reasoning capabilities (Chen et al., 2021, 2022) or proficiency in singular tasks (Son et al., 2023b; Malo et al., 2014; Loukas et al., 2022), leaving a critical gap in understanding their comprehension of the…”

“The TAT-QA dataset (Zhu et al., 2021) is designed for hybrid financial tabular and textual data, and the FinQA dataset (Chen et al., 2021b) targets numerical reasoning tasks in finance.”

“The most relevant works to our research are FinQA (Chen et al., 2022), TAT-QA (Zhu et al.”