FinQA: A Dataset of Numerical Reasoning over Financial Data

“For all models, we use the Adam optimizer (Kingma and Ba, 2015).”

“Retriever The full page of the financial report can go beyond 2,000 tokens, which cannot be coped with the current popular QA models (Devlin et al., 2019).”

“, 2019) and RoBERTa (Liu et al., 2019), our proposed approach outperforms all other baselines and achieves an execution accuracy of 65.”

“3703 scales linearly with sequence length, like the Longformer (Beltagy et al., 2020).”

“Financial QA is more challenging than classic QA (Rajpurkar et al., 2018; Yang et al., 2018) because it requires the system to spot relevant information across heterogeneous sources, such as tables and unstructured texts, and then create a numerical reasoning path to connect all the information.”

“Financial QA is more challenging than classic QA (Rajpurkar et al., 2018; Yang et al., 2018) because it requires the system to spot relevant information across heterogeneous sources, such as tables and unstructured texts, and then create a numerical reasoning path to connect all the information.”

“This way the model is able to learn the program structures as patterns from very large-scale data (~40k for MathQA), but may fail on learning the reasoning paths.”

“The performance of using FinBert is no better than BERT-large, mostly because its pre-training corpus is limited (~30M words from news articles).”

“Another kind of QA datasets related to ours is the math word problem datasets, like MaWPS (Koncel-Kedziorski et al., 2016), MathQA (Amini et al.”

“…every fact has known provenance and no real-world proprietary material is used in the corpus (unlike financial QA sets built from public filings [15]); model tool-output logs ( stdout_head ) can echo public SEC filing text, which is why the raw logs are excluded from the planned public deposit…”

“In the financial and ESG context, recent work by [19] explored the application of RAG systems to financial question answering, high-lighting the importance of high-quality retrieval components for domain-specific applications.”

“The independent FinQA construction shows the same qualitative boundary.”

“2.1 QA datasets (Pasupat and Liang, 2015; Zhong et al., 2017; Kweon et al., 2023) to financial datasets incorporating numerical reasoning (Chen et al., 2021; Wang et al., 2025).”

“2021) is a financial question-answering dataset over earnings reports and financial tables; answering requires programmatic reasoning (e.g., computing ratios, growth rates) over tabular data accompanied by natural-language context.”

“Document-grounded financial QA benchmarks (Islam et al., 2023; Chen et al., 2021) require retrieving from filings, but these documents are data sources for numerical values rather than natural language rules that govern computation over a database.”

“Two additional datasets, TAT-QA [7] and FinQA [8], extend the focus toward hybrid and numerical reasoning.”

“They emphasize financial question answering and retrieval [6, 7, 34], request-level understanding [4, 33], bounded state tracking [3], or targeted memory retrieval [14, 23, 30, 31].”

“Down-stream reasoning work targets question answering (Chen et al., 2021; Zhu et al., 2021; Islam et al., 2023; Choi et al., 2025), injecting expert structure through hand-authored workflows (Ni-tarach et al., 2025) or historical scaffolds (Singhal, 2025; Le, 2024), but produces closed-form…”

“Existing benchmarks concentrate on finance [3–7], and regional studies focus on predictive accuracy [9–11] rather than reasoning quality.”

“Total revenue in 2015 increased $25.7 million over 2014 due to growth in non-interest income of $11.6 million and growth in net interest income of $14.1 million comparing $634.3M in 2015 and $620.2M in 2014.”

“FinQA demonstrated the challenge of numerical reasoning over financial reports, and FinanceBench extended this problem to open-book question answering over public-company filings (Chen et al., 2021; Islam et al., 2023).”

“Evaluation has evolved in parallel, from report-grounded numerical QA in FinQA(Chen et al., 2021), TAT-QA(Zhu et al., 2021), and Con-Benchmark Primary Scope #Size C3 F2 ER RR DG OP SQ LB IT Financial QA and Reasoning Benchmarks FinQA (Chen et al., 2021) Report-based numerical QA 8,281 × × ✓ ✓ × ✓ × × × FinanceMATH (Zh…”

“Cross-Distribution Sources Finance Retrieval DocFinQA [27] + FinQA [4] FinanceBench [13] + TAT-DQA [47] + TAT-HQA [17]”

“FinQA [Chen et al., 2021] pairs financial-report questions with text, tables, and executable reasoning programs.”

“First, web search integration: when a model has web search enabled, it retrieves relevant pages from the internet at query time and incorporates them into its response, rather than relying solely on its training data [17]; this is tested in Section 5.5.”

“The corresponding evaluation surface has expanded from calculations over supplied tables and text to disclosure-grounded question answering and the rejection of plausible but unsupported claims (Chen et al., 2021; Zhu et al., 2021; Chen et al., 2022; Islam et al., 2023; Jiang et al., 2026).”

“FinQA and ConvFinQA (Chen et al., 2021) score an executed final answer within tolerance; TAT-QA scores an answer span; running F1 over every figure in a free-form generation is the outlier.”

“TAT-QA asks for calculated or extracted answers from evidence presented jointly as a table and accompanying text.”

“Financial NLP evaluation has progressed from numerical reasoning over financial reports [2] to open-book QA against primary-source documents [9] and holistic multi-task suites [21].”

“FinQA [ Chen et al. , 2021 ] and TAT-QA [ Zhu et al. , 2021 ] test multi-step reasoning over semi-structured tables, but their contexts are limited to a few thousand tokens.”

“For executive business decision making, financial benchmarks remain overwhelmingly text-centric, with nearly little visual business evidence (Chen et al. 2021; Islam et al. 2023; Koncel-Kedziorski et al. 2023; Choi et al. 2025; Kundurthy et al. 2026; Dai et al. 2026a).”

“Financial NLP benchmarks largely evaluate question answering and multimodal reasoning over tables, filings, and conversations (Chen et al. 2021, 2022; Zhu et al. 2021; Zhao et al. 2022; Islam et al. 2023; Reddy et al. 2024; Liu et al. 2025; Xie et al. 2024a, 2023; Wu et al. 2023), whereas agent…”

“We evaluate S on a set of cases, running each n times to absorb LLM stochasticity, which yields pass rate PR ( S ) scored with the unbiased pass@ k estimator [Chen et al., 2021a] and cost Cost ( S ) as the average LLM token consumption per case.”

“Earlier datasets such as FinQA and ConvFinQA show that financial question answering often requires numerical and multi-step conversational reasoning over financial documents [Chen et al. , 2021; Chen et al. , 2022].”

“Financial question-answering benchmarks further show that understanding financial reports often requires reasoning over heterogeneous textual and tabular evidence, including numerical operations and multi-step derivations (Chen et al., 2021; Zhu et al., 2021).”

“Subsequently, TATQA [ Zhu et al. , 2021 ] , FinQA [ Chen et al. , 2022a ] , and ConvFinQA [ Chen et al. , 2022b ] expanded the task scope to hybrid textual–tabular reasoning, requiring models to perform arithmetic and comparative analysis.”

“Large-scale financial LLMs broaden generative capability [5], while FinQA demonstrates numerical reasoning over reports [8].”

“FinQA (Chen et al., 2021) introduced numerical reasoning over financial tables and prose, and ConvFinQA (Chen et al., 2022) extended that to multi-turn conversational reasoning.”

“The first focuses on simple tabular reasoning with one or a few numerical answers , such as FinQA (Chen et al., 2021), TAT-QA (Zhu et al., 2021), and the practical-skill subsets of FLAME (Guo et al., 2024a) and FIRE (Zhang et al., 2026).”

“Answering such questions requires more than fluent generation: it requires locating the right evidence in long, structured documents and synthesizing responses that remain grounded in disclosed sources [4, 5, 9, 13, 14, 31, 39, 46, 47, 52].”

“…BusinessCaseBench, a discipline-spanning case benchmark with rubric-graded scoring that sits between narrow quantitative/analytical suites (Chen et al., 2022; Koncel-Kedziorski et al., 2024; Xie et al., 2024) and agent benchmarks focused on enterprise software use (Drouin et al., 2024; Xu…”

“FinQA [21] introduced a dataset of over 8,000 question-answer pairs derived from S&P 500 earnings reports, requiring multi-step numerical reasoning over both text and tables.”

“FinQA studies numerical reasoning over financial reports [9], and FinanceBench evaluates financial question answering against professional-style evidence requirements [10].”

“FinQA was selected specifically because it emphasizes compositional financial reasoning over both narrative text and tables, requiring models to identify relevant evidence, bind variables across heterogeneous formats, and perform multi-step numerical operations [1].”

“FinQA [5] and TAT-QA [6] tackle numerical reasoning over financial tables but focus on single-document extraction rather than cross-document comparison.”

“Financial RAG has been studied for numerical reasoning [26], [27] but not for multi-document synthesis in evaluation tasks.”

“FinQA (Chen et al., 2022) mines earnings reports.”

“Word-level explanations are strongest on MS MARCO, FinanceBench, and FinQA, and raw chunks are never the best BM25 unit.”

“In benchmarks such as FinQA, TAT-QA, and Fin-RATE, this means that a system must do more than retrieve semantically similar passages: it must reason over time-sensitive evidence while preserving factual precision and traceable support [2]–[4].”

“Notably, financial QA datasets such as FinQA [8] and FinLLMs [48] illustrate the challenges and opportunities of reasoning over domain-specific documents with embedded numerical content.”

“Prior to agentic evaluation, financial NLP benchmarks such as FinQA [7], TAT-QA [38], and ConvFinQA [8] established the core challenge of numerical reasoning over financial documents—extracting figures from tables and text and chaining them through arithmetic operations to answer a question.”

“…evidence retrieval and reasoning over given tables, requiring models to understand tabular structures, precisely locate relevant entries, and perform multi-step numerical computations, which is widely applied in data engineering [1]– [3], financial auditing [4] and medical records analysis [5].”

“FinQA [6] pioneered numerical reasoning over short financial reports and tables.”

“The findings provide support for multi-agent LLMs as a viable means of addressing difficult problems [4] by employing a method of overcoming larger tasks through the division of complex tasks into smaller, more manageable ones that will allow multiple agents to accomplish these tasks.”

“Finance benchmarks include FinQA [13] for numerical reasoning over financial documents, FiQA [22] for financial question answering and sentiment analysis, and a financial news classification dataset.”

“FinQA, ConvFinQA, and TAT-QA [4, 5, 14] evaluate numerical reasoning over financial reports and tables.”

“Tables are a representative form of structured data and are widely found in financial reports, statistical yearbooks, and various domain-specific knowledge bases (Liu et al., 2023a; Chen et al., 2021b).”

“FinQA and ConvFinQA emphasize numerical reasoning over financial reports [6, 17]; TAT-QA focuses on table-text reasoning [7]; FinanceBench evaluates factual grounding over financial documents [4]; and DocFinQA extends financial QA to longer documents [18].”

“Professional domains are represented by MedQA [27] and FinQA [28], and safety robustness is evaluated on WildChat [29] and JailbreakBench [30].”

“FinQA [6] is the established benchmark for numerical reasoning over financial data, comprising 8,281 instances from corporate reports in English.”

“FinQA (Chen et al., 2021).”

“We evaluate DynaGraph on three datasets: (1) StrategyQA (Geva et al., 2021) for open-domain multi-hop fact retrieval; (2) MATH (Hendrycks et al., 2021) for long-horizon logical deduction and topological resilience; and (3) FinQA (Chen et al., 2021) for cross-modal multi-expert orchestration.”

“Domain-specific benchmarks such as LegalBench [11], PubMedQA [13], and FinQA [5] evaluate specialized reasoning within individual fields, while WebArena [37] and Mind2Web [8] study task completion in grounded web environments.”

“However, existing benchmarks primarily target natural language processing (NLP) tasks [3, 4] or isolated chart comprehension [5].”

“FinanceBench [44] 90 CC-BY 4.0 FinQA [45] 8,280 MIT Money StackExchange [46] 34,035 CC-BY-SA 4.0 subtotal 42,405 Law MMLU–Law [47] 1,709 MIT Australian Legal QA [48] 2,124 CC-BY 4.0 LegalQA-v1 [49] 3,724 Apache 2.0 Law Stack Exchange [50] 25,366 CC-BY-SA 4.0 EUR-Lex-Sum (EN) [51] 1,142 CC-BY 4.0…”

“08 ) in FinQA.”

“We primarily evaluate our detector on four widely used mathematical benchmarks, including Math-500 [19], AIME-2024 [26], AIME-2025 [27], and OlympiadBench-Math [28], and later extend our evaluation to additional STEM domains, including GPQA [29], OlympiadBench-Phy [28], and FinQA [30].”

“FinQA [Chen et al., 2021] is a financial QA dataset of 8 , 281 questions grounded in real earnings reports, where each answer is produced by a short numerical reasoning program over tables and accompanying text.”

“For example, the FinQA dataset [4] evaluates the numerical reasoning capabilities of several LLMs.”

“Our domain coverage is limited to one general and one biomedical benchmark, legal (Guha et al., 2023) and financial QA (Chen et al., 2021) pose structurally different challenges that future work should address.”

“FinCausal is closely related to the development of financial question answering research, which places greater emphasis on financial reasoning and multi-step inference, at the cost of more traditional financial reporting (Chen et al., 2021).”

“This is a special case of the general dynamics (Equation (3)) with score norm ∥ s ( θ ) ∥ 2 = (1 − p ) 2 , which satisfies ∥ s ∥ 2 ∈ [(1 − δ ) 2 , 1] on p ∈ [ p 0 , δ ] — confirming the bounded score assumption.”

“Recent work has focused on numerical reasoning (Chen et al., 2021; Lu et al., 2023) and multi-step analysis (Zhang et al., 2023; Wu et al., 2025), surpassing the simpler open-domain queries of NQ-TABLES (Herzig et al., 2021; Kweon et al., 2023).”

“Furthermore, existing financial datasets and benchmarks (Z. Chen et al., 2021; X. Li et al., 2024) predominantly emphasize data extraction or price prediction rather than decision-making logic.”

“Researchers like Chen et al. also found that standard RAG completely fails at financial reasoning [11].”

“…numerical computations, hard-coded business rules, and structured data operations such as filtering, joining, aggregation, and conditional transformation (Chen et al., 2021b, 2022), which † Work done during the authors’ internships at Microsoft. makes code generation substantially more challenging.”

“FinQA requires numerical reasoning over hybrid tabular and textual evidence in financial reports, while FinanceReasoning further challenges LLMs with multi-step financial numerical reasoning across three difficulty levels.”