FinQA: A Dataset of Numerical Reasoning over Financial Data

“For all models, we use the Adam optimizer (Kingma and Ba, 2015).”

“Retriever The full page of the financial report can go beyond 2,000 tokens, which cannot be coped with the current popular QA models (Devlin et al., 2019).”

“, 2019) and RoBERTa (Liu et al., 2019), our proposed approach outperforms all other baselines and achieves an execution accuracy of 65.”

“3703 scales linearly with sequence length, like the Longformer (Beltagy et al., 2020).”

“Financial QA is more challenging than classic QA (Rajpurkar et al., 2018; Yang et al., 2018) because it requires the system to spot relevant information across heterogeneous sources, such as tables and unstructured texts, and then create a numerical reasoning path to connect all the information.”

“Financial QA is more challenging than classic QA (Rajpurkar et al., 2018; Yang et al., 2018) because it requires the system to spot relevant information across heterogeneous sources, such as tables and unstructured texts, and then create a numerical reasoning path to connect all the information.”

“This way the model is able to learn the program structures as patterns from very large-scale data (~40k for MathQA), but may fail on learning the reasoning paths.”

“The performance of using FinBert is no better than BERT-large, mostly because its pre-training corpus is limited (~30M words from news articles).”

“Another kind of QA datasets related to ours is the math word problem datasets, like MaWPS (Koncel-Kedziorski et al., 2016), MathQA (Amini et al.”

“Contract review datasets and legal language models show that clause-level supervision is useful for legal NLP [11], [17], [18], while financial QA datasets demonstrate the importance of table-text reasoning and executable numerical answers [12], [13].”

“TAT-QA (Zhu et al. 2021) and FinQA (Z. Chen et al. 2021) proposed QA datasets over tables and texts from financial reports, which require numerical reasoning to infer the answer.”

“Chen et al. developed FinQA, a dataset and benchmark specifically for financial question answering that necessitates numerical reasoning over financial reports (Chen et al., 2021).”

“Chen et al. (2021) introduced FINQA, an expert-annotated dataset containing 8,281 financial QA pairs, along with their numerical reasoning processes.”

“Following Chen (Chen et al., 2021), the baseline model is a binary classification model with a Bert backbone to extract embedding.”

“Large language model (LLM) agents are increasingly prevalent in the financial domain, driving advancements in financial text analysis and reasoning (Chen et al., 2021; Xie et al., 2023; Reddy et al., 2024), trading signal mining (Luo et al., 2026; Shi et al., 2025), and automated trading (Xiao et…”

“…2021) is a BERT-based model trained on general and financial texts; FinMA (Xie et al., 2023) fine-tunes Llama (7B/30B) on financial QA datasets (Chen et al., 2021, 2022b); InvestLM (Yang et al., 2023) adapts Llama-65B with curated financial instructions; and BloombergGPT (Wu et al., 2023), a…”

“…Cobbe et al., 2021; Patel et al., 2021; Talmor et al., 2021; Geva et al., 2021; Mirzadeh et al., 2024; Han et al., 2024; Valmeekam et al., 2024; Mehrafarin et al., 2024; Paruchuri et al., 2024; Tyagi et al., 2024a; Samadarshi et al., 2024; Shiri et al., 2024; Chen et al., 2021; Luo et al., 2023).”

“For example, Rad-Graph (Jain et al., 2021) supports clinical information extraction from radiology reports, LEDGAR (Tuggener et al., 2020) and LeDQA (Liu et al., 2024) address legal QA tasks, and FinQA (Chen et al., 2021) targets financial reasoning through expert-labeled QA pairs.”

“Second, objec-tive datasets often lack balanced linguistic and domain coverage, limiting their effectiveness in diverse contexts [3, 12].”

“Recently, several chal-lengingdatasets[2],[3],[4],[21],[22]andapproaches[22],[23], [24],[25]havebeenproposedtoadvanceresearchinthisfield.”

“answering [2], [3], [4], dialog [5], report generation [6], [7], [8], [9], and biographical description [10], demonstrating the great potential of table-to-text generation for use in extensive real-world scenarios.”

“However, existing financial benchmarks for LLM agents largely focus on static question-answering tasks (Chen et al., 2021; Zhu et al., 2021; Yin et al., 2023), which are designed to test the financial knowledge of LLMs but fail to capture the dynamics of realistic trading scenarios.”

“…HellaSwag (Zellers et al., 2019), Wino-grande (Sakaguchi et al., 2020), TruthfulQA (Lin et al., 2022), ETHICS (Hendrycks et al., 2021a), QASC (Khot et al., 2020), BioASQ (Tsatsaronis et al., 2015), FinQA (Chen et al., 2021b), CaseHOLD (Zheng et al., 2021), MusicTheoryBench (Yuan et al., 2024).”

“The domain-specific collection is built from public datasets such as FinQA (Chen et al., 2021), AILACasedocs (Bhattacharya et al., 2020) and XLSum (Hasan et al., 2021).”

“Existing financial benchmarks still leave this gap: FinQA (Chen et al., 2021), TAT-QA (Zhu et al., 2021), and ConvFinQA (Chen et al., 2022) emphasize numerical reasoning with provided evidence; FinAgentBench (Choi et al., 2025) and related settings evaluate retrieval or tool use under more…”

“We conduct our experiments across 4 financial QA datasets ( CFA L EVEL 1 and CFA L EVEL 2 [20], CMT [20], and F IN QA [2]), 10 consistency dimensions, 2 tested model families ( GPT 5.5 and C LAUDE O PUS 4.5 ), and 6 distinct generator-validator pipeline configurations, measuring the joint…”

“FinQA has numerical questions over real financial reports (Chen et al., 2021).”

“Experiments on FinQA across six commercial and open-weight base models show that adaptive verification yields more efficient Pareto frontiers than fixed verification: BudgetVerify consistently improves upon uniform M ID verification and often matches or approaches uniform H IGH verification while using fewer tokens.”

“…[67], [68], [69], [70], [71], [72], [73], [74], [75], [76], [77], to table and chart reasoning with arithmetic programs [78], [79], [80], [81], [82], [83], [84], [85], [2], [86], [87], [88], to scientific and multipage evidence aggregation [89], [90], [91], [92], [93], [94], [95], [96], to…”

“…works to F IN I NTERACT and defer further related work in Appendix D. Financial QA benchmarks such as Fi-nanceBench (Islam et al., 2023), FinQA (Chen et al., 2021), and DocFinQA (Reddy et al., 2024), together with recent agentic successors, enforce single-gold, single-turn evaluation and test…”

“Financial QA datasets such as FinQA (Chen et al., 2021), ConvFinQA (Chen et al., 2022), TAT-QA (Zhu et al., 2021), and FinanceBench (Islam et al., 2023) provide the relevant evidence context, and therefore primarily evaluate reasoning over evidence rather than the end-to-end process that discovers it.”

“…never sees: BANKING77 intent (Casanueva et al., 2020) by macro-F1, Bitext support (Bitext, 2023) by response win-rate, Glaive function-calling (Glaive AI, 2023) by executable-rate, FinQA numerical QA (Chen et al., 2021) by exact-match, and CUAD legal clauses (Hendrycks et al., 2021) by clause-F1.”

“A 240-question FinQA subset [20] yields 960 document-grounded scenarios with executable gold programs and controlled table corruption, used only as an external check.”

“Recent finance-oriented models and benchmarks evaluate sentiment, numerical reasoning, forecasting, knowledge, and decision support (Araci, 2019; Chen et al., 2021, 2022; Wu et al., 2023; Yang et al., 2023; Xie et al., 2023; Matlin et al., 2025; Klimaszewski et al., 2025; Shah et al., 2025).”

“The challenge is especially acute in specialized domains such as finance (Chen et al., 2021), legal analysis (Kulkarni et al., 2026), medicine (Sing-hal et al., 2025), and education (Alawwad et al., 2025), where critical evidence is frequently layout-dependent and visually structured (Loison et…”

“For FinQA, predictions are regarded as correct when they fall within a 1% numerical relative tolerance of the gold answer under the oracle-retrieval setting.”

“Some of the data sets that contain hybrid forms of data with both tabular and textual content include FinQA [4], Con-vFinQA [5], TATQA [26].”

“We use FinQA (Chen et Models and training protocol.”

“Cross-modal benchmarks building on earlier tabular-textual QA work (Chen et al., 2021; Zhu et al., 2021), including FinanceBench (Islam et al., 2023), FAMMA (Xue et al., 2025), and XFin-Bench (Zhang et al., 2025), find current LLMs fail many realistic financial questions, worsening as context grows…”

“…Li et al., 2024; Rein et al., 2023), yet financial reasoning over structured tabular data ( particularly in settings that require compositional depth and domain knowledge) remains comparatively under-explored (Chen et al., 2022a; Zhu et al., 2021; Zhao et al., 2022; Koncel-Kedziorski et al., 2024).”

“…over prose, tables, and numerical programs, including FinQA, TAT-QA, and MultiHiertt, while ARQA and MuDABench emphasize deterministic verification and aggregation over annual-report collections (Chen et al., 2021; Zhu et al., 2021; Zhao et al., 2022; Wang and Balloccu, 2026; Li et al., 2026).”

“MATH-500 (Hendrycks et al., 2021) requires free-form competition-mathematics answers, FinQA (Chen et al., 2021b) requires numerical reasoning over financial reports, and PubMedQA (Jin et al., 2019) requires yes/no/maybe decisions from biomedical abstracts.”

“FinQA (Chen et al., 2022) and TAT-QA (Zhu et al., 2021) require multi-step reasoning over The key limitation is retrospective analysis of given information.”

“…answers from Wikipedia; HotpotQA (Yang et al., 2018) requires multi-hop reasoning over multiple Wikipedia para-graphs; NewsQA (Trischler et al., 2017) consists of reading comprehension questions over CNN news articles; FinQA (Chen et al., 2021) involves numerical reasoning over financial reports.”

“Prior financial QA benchmarks establish numerical reasoning over text, tables, and conversational evidence (Chen et al., 2021; Zhu et al., 2021; Chen et al., 2022); our setting extends this line to long, converted reports in which relevant quantities can be distributed across pages and visual…”

“FinQA [7] comprises questions written by financial analysts over S&P 500 firms’ filing excerpts (1999–2019) and requires multi-step numeric reasoning over long contexts.”

“Conceptually, P OLY G YM is related to controlled retrieval settings in QA research (Chen et al., 2021; Yang et al., 2018) Table 5: Comparison of forecasting evaluation setups.”

“Recent work in finance QA has made progress on reasoning over hybrid report content, including tables and surrounding text [6, 26].”

“Financial benchmarks such as Pixiu [7], FinEval [8], CF-Benchmark [12], FinanceBench [10], and FinBen [13] evaluate QA, extraction, and generation across languages [11], [37], [38].”

“Specifically, RAG-Bench collects extensive QA datasets, including CovidQA (Möller et al., 2020), PubMedQA (Jin et al., 2019), HotpotQA (Yang et al., 2018), MS Marco (Bajaj et al., 2016), CUAD (Hendrycks et al., 2021), EManual (Nandy et al., 2021), TechQA (Castelli et al., 2020), FinQA (Chen et al., 2021), TAT-QA (Zhu…”

“Numerical reasoning benchmarks such as FinQA [4], ConvFinQA [5] and TAT-QA [33] pair short table-plus-text snippets with arithmetic questions; Fi-nanceBench [10] broadens this to open-book QA over 10-”

“5-72B-Instruct collapses from 73.38 (FinQA) to 14.67 (Complong), Qwen2.”

“Other resources target numerical reasoning (Chen et al., 2021b; Zhu et al., 2021), hierarchical tables (Cheng et al., 2022), long-context cell retrieval (Wang et al., 2026), and complex or multilingual table understanding (Zhu et al., 2025).”

“In the financial domain, F IN QA [254] provides 8,281 expert-annotated question–answer pairs over earnings reports in which the gold rationale is a small executable arithmetic program, while TAT-QA [255] (16,552 questions) and M ULTI H IERTT [256] (10,440 questions over multi-hierarchy tables)…”

“…et al., 2023), while financial QA benchmarks such as FinQA, TAT-QA, FinanceBench, and FAITH emphasize numerical reasoning, tabular reasoning, evidence-grounded answers, or tabular faithfulness over financial documents (Chen et al., 2021; Zhu et al., 2021; Islam et al., 2023; Zhang et al., 2025).”

“Unlike general math benchmarks, datasets such as FinQA (Chen et al., 2021), HiTab (Cheng et al., 2022), and MultiHiertt (Zhao et al., 2022) represent specialized domains where expert-annotated rationales are difficult and costly to obtain.”

“…reasoning settings where tabular evidence, contracts, holdings, and expert annotation make provenance and rubric fields central (AI, 2023; Chen et al., 2022; Zhu et al., 2021; Guha et al., 2023; Zheng et al., 2021; Hendrycks et al., 2021b; Koreeda and Manning, 2021) Safety, judges, and…”

“[5], which generates domain-specific programs from financial documents; CBR-Ren [8], which leverages case-based reasoning for program retrieval and adaptation; and PoT-GPT-4 [4], which employs Program-of-Thought prompting with GPT-4 as the backbone.”

“Single-page document QA (DocVQA (Mathew et al., 2021), InfographicVQA (Mathew et al., 2022), VisualMRC (Tanaka et al., 2021), ST-VQA (Biten et al., 2019)), chart QA (ChartQA (Masry et al., 2022), PlotQA (Methani et al., 2020)), and table QA (WikiTableQues-tions (Pasupat and Liang, 2015), TabFact (Chen et al., 2020), T…”

“…on several entities (Yang et al., 2018; Trivedi et al., 2022), symbolic operations such as arithmetic or aggregation (e.g., returning the answer that maximizes a quantity) (Dua et al., 2019; Chen et al., 2021; Zhu et al., 2021), or the composition of multiple intermediate answers (Ho et al., 2020).”

“Structured and semi-structured QA benchmarks include knowledge-graph datasets such as WebQuestions (Berant et al., 2013), ComplexWebQuestions (Tal-mor and Berant, 2018), 2WikiMultiHopQA (Ho et al., 2020), and BMW-KG (Roy et al., 2025), as well as table-oriented datasets such as WikiTable-Questions (Pasupat and Liang,…”

“In this ablation, we use the full FinQA evaluation set and fix the initial agent traces, calibrated belief state, disclosure policy, and deterministic guardrail.”

“…on 11 representative benchmarks across four categories: math (AIME-24 [54], AIME-25 [53], MATH-500 [35]), science (GPQA-Diamond [83], SuperGPQA [21], HLE [72]), tabular analysis (FinQA [15], MultiHier [125]), and web information seeking (BrowseComp [64], GAIA-103 [56], XBench-DeepSearch [107]).”

“This is particularly evident in complex numerical reason - ing tasks over tabular data, such as FinQA [45], TAT-QA [46], and DocMath-Eval [ 47 ], where factual consistency is highly sensitive to format [ 48 ].”

“Domain-specific benchmarks (Liang et al., 2023; Srivastava et al., 2023) establish the methodology of expert-graded evaluation in finance (FinanceBench (Is-lam et al., 2023), FinQA (Chen et al., 2021b), TAT-QA (Zhu et al., 2021)), economically valuable knowledge work (APEX-v1 (Vidgen et al., 2025), APEX-Agents (Vidgen…”

“Existing financial tasks (Das et al., 2024c, 2025c) and their reasoning benchmarks, such as FinQA (Chen et al., 2021), TAT-QA (Zhu et al., 2021), ConvFinQA, and their programmatic extensions CodeFinQA and CodeTAT-QA (Krumdick et al., 2024) largely focus on tabular extraction and shallow arithmetic,…”

“…(logical inference under closed-world rules), AIME 2022 to 2024 [https://huggingface.co/datasets/AI-MO/aimo-validation aime] (olympiad mathematics, symbolic), ALFRED [Shridhar et al., 2020] (procedural action-grounded planning), and FinQA [Chen et al., 2021] (tabular financial question answering).”

“We first aggregate raw records from 14 prominent public datasets ( e.g., WTQ (Pasupat and Liang, 2015), HiTab (Cheng et al., 2021), Tab-Fact (Chen et al., 2019), MMQA (Wu et al., 2025), FinQA (Chen et al., 2021), etc. ), standardizing them into a canonical format comprising the table , textual context , question , and…”

“…applications, serving as a carrier of rich informa-1 tion (Cafarella et al., 2008) across critical domains, ranging from financial analysis (Chen et al., 2021b; Zhu et al., 2021; Chen et al., 2022) and medical interpretation (Lee et al., 2022; Shi et al., 2024) to daily record management…”

“Later work adopted multimodal approaches combining text, tables, and figures (Chen et al., 2021; Singh et al., 2024).”

“For FinQA, we randomly sample 400 question pairs with similarity scores ≥ 0 .”

“More advanced NIAH benchmarks [62, 63, 64, 65, 66] introduce more challenging domains or distractors to make finding the correct “needle” harder; however, the total amount of information required to solve each problem remains small.”

“[21 , 22] More recently, FinanceBench and FinDER have also been published as human-annotated sources of question-answer pairs whose evidence is grounded in the respective knowledge bases, covering major companies that constitute highly liquid market indexes such as the American S&P500.”

“For in-domain evaluation, we select diverse benchmarks including FinQA (Chen et al. 2021b), HiTab (Cheng et al. 2022a), MultiHiertt (Zhao et al. 2022), TabFact (Chen et al. 2019), TAT-QA (Zhu et al. 2021), Fever-ous (Aly et al. 2021), and WikiTQ (Pasupat and Liang 2015).”

“For example, FinQA (Chen et al., 2021) is constructed from financial statements and requires models to answer natural language questions through reasoning and numerical computation.”

“Year FAT NR MDE RFD FA RE FinQA[14] 2021 FAT : Financial Analysis Tasks, NR : Numerical Reasoning, MDE : Multi-Dimensional Evaluation, RFD : Real Financial Data, FA : Failure Analysis, RE : Recency Evaluation.”

“TAT-QA [16] and FinQA [28] evaluate if models can perform numerical reasoning over hybrid financial data.”

“Many studies rely on clean benchmarks like FinQA [7] or TAT-QA [8], creating a false sense of security.”

“Benchmarks such as as FinQA [6], TAT-QA [7], and ConvFinQA [8] emphasize numerical reasoning, often requiring multi-step computation grounded in tables and narrative context.”

“External knowledge access enables dynamic updates and reduces factual inconsistencies [5], [6].”

“Existing benchmarks like FinQA [2] and TAT-QA [22] typically utilize short, curated contexts, while FinanceBench [6] focuses primarily on retrieval.”

“Many high-performing approaches use program sketches, tool calls, or verifiable execution traces, which improves au-ditabilityandcalibration[24,4, ?”

“As we know Confidence prefix (strong) What are some innovative rehabilitation techniques for stroke survivors?”

“A later extension, ConvFinQA, turned FinQA into a multi-turn dialogue challenge.”

“According to the FinQA paper [3], over 85% of the errors stem from: (i) lack of domain knowledge in finance; or (ii) numerical reasoning errors.”

“Significant progress has been achieved in English, driven by benchmark datasets like FinQA [1] and TAT-QA [2], which integrate textual and tabular financial data and have spurred the development of program-based reasoning models.”

“The evaluation of Vision-Language Models (VLMs) in the finance domain has seen remarkable progress, from early text-based benchmarks (Chen et al., 2021; Shah et al., 2022; Xie et al., 2024) to more recent efforts targeting financial image understanding (Kim et al., 2024; Zhu et al., 2025; Shu et…”

“Chen et al. [1] studied reasoning over financial data and showed that structured information can help models handle numeric facts.”

“On one hand, most benchmarks that explore complex reasoning, like FinQA [Chen et al., 2021], remain fundamentally text-based , ignoring the visual aspect of tables as they appear in documents.”

“While other financial datasets exist [9, 10, 28], most emphasize tabular data or long-form question answering, often derived from the traditional FinQA [10] dataset introduced in 2021.”

“Similarly, FinQA [5], FinTextQA [4], and DocFinQA [36] prioritize answer generation over retrieval performance.”

“The proposed approach has been evaluated on multiple benchmark datasets, including FinQA [1], TAT-QA [2], ConvFinQA [3], FinanceBench [4], FinQABench [5], FinDER [6], and Multi-Hiertt [7].”

“Our contributions are: • QASU, the first benchmark to isolate multiple structural • We provide a complete ablation of serialization and prompt • Demonstration that self-augmented prompting [42] is a simple”

“Early work mostly focused on rule-based methods [16, 23], while more recent studies explore multimodal financial analysis that combines text with tables and figures [7, 24].”