FinQA: A Dataset of Numerical Reasoning over Financial Data

“For all models, we use the Adam optimizer (Kingma and Ba, 2015).”

“Retriever The full page of the financial report can go beyond 2,000 tokens, which cannot be coped with the current popular QA models (Devlin et al., 2019).”

“, 2019) and RoBERTa (Liu et al., 2019), our proposed approach outperforms all other baselines and achieves an execution accuracy of 65.”

“3703 scales linearly with sequence length, like the Longformer (Beltagy et al., 2020).”

“Financial QA is more challenging than classic QA (Rajpurkar et al., 2018; Yang et al., 2018) because it requires the system to spot relevant information across heterogeneous sources, such as tables and unstructured texts, and then create a numerical reasoning path to connect all the information.”

“Financial QA is more challenging than classic QA (Rajpurkar et al., 2018; Yang et al., 2018) because it requires the system to spot relevant information across heterogeneous sources, such as tables and unstructured texts, and then create a numerical reasoning path to connect all the information.”

“This way the model is able to learn the program structures as patterns from very large-scale data (~40k for MathQA), but may fail on learning the reasoning paths.”

“The performance of using FinBert is no better than BERT-large, mostly because its pre-training corpus is limited (~30M words from news articles).”

“Another kind of QA datasets related to ours is the math word problem datasets, like MaWPS (Koncel-Kedziorski et al., 2016), MathQA (Amini et al.”

“FinQA and TAT-QA focus on numerical reasoning over financial reports and tables [2], [3]; FinanceBench evaluates open-book question answering over public-company filings [4]; FinAgentBench and FinRetrieval examine retrieval over financial documents or structured data [5], [16]; Finch evaluates…”

“Financial benchmarks cover numerical and document reasoning (Chen et al. 2021; Zhu et al. 2021; Chen et al. 2022; Islam et al. 2023), broad capability suites (Xie et al. 2023, 2024a), and professional analysis, research, tool-use, or spreadsheet tasks (BenYoash et al. 2025; Bigeard et al. 2025;…”

“T2-RAGBench includes four datasets: ConvFinQA [1], FinQA [1], TAT-DQA [23], and VQAonBD [11] that cover heterogeneous financial sources such as annual reports, regulatory filings, and financial statements.”

“Charts have dedicated benchmarks (ChartQA [18], PlotQA [22], CharXiv [32], ChartQAPro [19]), as do tables and numerical reasoning (HybridQA [3], TAT-QA [37], FinQA [5]).”

“The finance subset is constructed from Financial Sentiment Analysis [25], FinQA [7], and FinClaim [34].”

“Output Perturbation increases Agreement on FinQA relative to No Defense, suggesting that noisy responses can provide exploitable training signals.”

“FinQA (19) is held out from training and tests generalization to heterogeneous tool routing (search followed by computation).”

“TableBench (Wu et al., 2025b) consolidates tables from WTQ (Pasupat & Liang, 2015), SQA (Iyyer et al., 2017), FinQA (Chen et al., 2021), and related datasets to evaluate LLMs across 18 task subtypes.”

“We construct two failure attribution datasets as described in Section 4.2: (a) FinQA-TAS is based on the hybrid reasoning FinQA dataset (Chen et al., 2021) and (b) Spider-TAS is based on the Spider dataset comprising structured text-to-SQL tasks (Yu et al., 2018).”

“However, benchmarks like FinQA Chen et al. [2021] and TAT-QA Zhu et al. [2021] reveal that even code-generating models struggle with grounding ; frequently executing correct logic on hallucinated input variables.”

“FinQA (Chen et al., 2021a), and TAT-QA (Zhu et al., 2021) primarily focus on interpreting financial reports and statements, while FinanceMath (Zhao et al., 2024a) evaluates financial reasoning using college-level finance and economics problems.”

“Existing evaluations span general Japanese linguistic benchmarks (Kurihara et al., 2022; Someya and Oseki, 2023) and global financial benchmarks (Xie et al., 2024; Chen et al., 2022), but the former lacks finance coverage and the latter is largely English-centric.”

“SEC filings have therefore become a common testbed for financial QA with LLMs [19, 20, 7], alongside broader benchmarks in finance [31, 10].”

“RAG benchmarks (Yu et al., 2024; Fan et al., 2024) have been central to evaluating systems across question answering (Talmor et al., 2019; Chen et al., 2021), document retrieval (Wang et al., 2024; Wasserman et al., 2025), and summarization (Wang et al., 2020; Petroni et al., 2021).”

“We observe consistent performance gains in the retriever metrics (except for FinQA + Marco), but only meaningful gains in HotPotQA and FinQA.”

“However, single-turn dialogues cannot capture dialogue history understanding and adaptive response strategies, re-stricting LLM deployment.”

“TATQA [39] and FINQA [5] are financial report datasets.”

“For example, FinQA [10] and TAT-QA [49] focus on hybrid question answering over financial reports, requiring multi-step reasoning with textual and tabular data.”

“…OCRVQA [76], POIE [46], Sroie [35], CIDP [52] Infographic Cauldron [49], InfoVQA [71], InfographicGPT4v [71], Docs-diagrams [26], Slide [93], Finqa [19], Hitab [20], MultiHiertt [128], Robut-SQA [129], Robut-Wikisql [129], Robut-WTQ [129], Table Tabmwp [66], TAT-QA [131], Pubtables [91],…”

“FinQA (Chen et al., 2021) advances numerical reasoning evaluation by constructing a dataset requiring multi-step calculations and inference from financial reports.”

“Datasets To capture the diversity in our experimental evaluations, we conducted experiments on multiple benchmark datasets: FinQA(Chen et al. 2021), HotpotQA(Yang et al. 2018) and Ms Marco(Nguyen et al. 2017).”

“These benchmarks cover relevant real-world application areas, including health (Jin et al., 2019; Möller et al., 2020), finance (Chen et al., 2021; Zhu et al., 2021), customer support (Sadat et al., 2023; Nandy et al., 2021; Malaviya et al., 2023), legal (Hendrycks et al., 2021), and general…”

“Four Abilities in SK Understanding real-world scenarios (e.g., financial reports with numerous tables (Chen et al., 2021) and product knowledge graphs (Wu et al., 2024)), thus serving as a significant knowledge base for existing LLM systems (Liang et al., 2024; Wang et al., 2025).”

“Different from the previous works that focus on the solving general financial NLP tasks, such as financial sentiment analysis [32], text classification [38, 40] or question answering [3, 4], our work supports three applications for the banking services.”

“Existing financial benchmarks include FLUE [39], CFLEB [40], and DISC-FinLLM [35] for evaluating financial NLP capabilities, and FinEval [41], InvestorBench [42], and FinQA [43] for financial knowledge and analysis.”

“Fi-nanceBench [8] and FinQA [4] focus narrowly on static financial statement analysis, without evaluation of real-time queries.”

“Pre-text and Post-text refer to optional unstructured context surrounding the table, such as the data description format used in FinQA (Chen et al., 2021b). formula operator instruction represents the textual representations and usage guidelines of the representative spreadsheet formula operators…”

“Lucia Zheng, Neel Guha, Brandon R. (Rajpurkar et al., 2016) Wikipedia 400 5 CaseHold (Zheng et al., 2021) Law 400 5 FinQA (Chen et al., 2021) Finance 400 5 SciQ (Johannes Welbl, 2017) Science 400 5 TriviaQA (Joshi et al., 2017) Open-domain 400 5 AQuA (Behrendt et al., 2024) Math 400 5 PubMedQA (Jin…”

“To evaluate the generalization ability of our method, we use SQA(Iyyer et al., 2017), Wik-iTableText (Bao et al., 2018) and FinQA (Chen et al., 2021) as held-out datasets, where SQA belongs to table-based question, WikiTableText belongs to structured data to Text and FinQA requires generating python-executable math ex…”

“Moreover, the introduction of niche datasets such as FinQA or financial-phrasebank has played a major role in refining the proficiency of LLMs in processing financial texts interspersed with numerical information [4,17].”

“Other tabular datasets are constructed from financial reports including TAT-QA (Zhu et al., 2021), F IN QA (Chen et al., 2021), M ULTI H IERTT (Zhao et al., 2022), or scientific reviews (Sundar et al., 2024).”

“In recent years, question-answering tasks based on heterogeneous knowledge (HQA) have increasingly gained the attention of researchers (Chen et al., 2020b,a; Zhu et al., 2021; Talmor et al., 2021; Chen et al., 2021).”

“Datasets such as FinQA show that finance questions require multi-step numerical reasoning over financial reports [9].”

“These gains further transfer to other tabular reasoning benchmarks (TAT-QA, FinQA, and TabMWP), with an average improvement of 4%(Table 12), particularly on complex tables, demonstrating the effectiveness of controllable data generation.”

“…systems, despite having been addressed in other generation tasks such as text normalization (Zhang et al., 2019; Sproat, 2022), question-answering (Chen et al., 2021; Kim et al., 2022), and text-to-speech (Nikulásdóttir and Guðnason, 2019); tasks which also struggle to synthesize texts with…”

“FinQA (Chen et al., 2021) is based on a collection of financial reports published by U.S. companies that were released as part of FinTabNet (Zheng et al., 2021).”

“In conversational, high-consequence domains like health, finance, or e-commerce, we must be sure that any factual error introduced by an NLG system is detected (Chen et al., 2021; Di Sotto and Viviani, 2022; Khan et al., 2022).”

“Financial Datasets and Evaluation Existing datasets such as FinQA [Chen et al. , 2021], TAT-QA [Zhu et al. , 2021], and FLARE [Xie et al. , 2023] focus on question answering or numerical reasoning, but lack support for comprehensive, real-time stock analysis.”

“Apply counting, spatial, or conditional rules; answer table and geometry questions ImajevBench (mohit67890, 2026); FinQA (Chen et al., 2021); Geometry3K (Lu et al., 2021)”

“Representative tasks include retrieving numerical facts from filings, identifying relevant spans in financial reports, and extracting values from tables or textual disclosures (Chen et al., 2021; Krumdick et al., 2024).”

“FinQA is a financial numerical reasoning corpus containing roughly 8k expert-annotated question–answer pairs grounded in approximately 2.8k financial reports, where each problem requires reasoning jointly over textual and tabular evidence.”

“• T 2 -RAGBench/FinQA (Strich et al., 2026; Chen et al., 2021b): A decontextualized variant of FinQA designed for RAG evaluation, where questions about corporate earnings reports have been rewritten to be context-independent.”

“…financial LLM benchmarks have substantially expanded coverage of financial language understanding and numerical reasoning Shah et al. (2022), Chen et al. (2021, 2022), Zhu et al. (2021), broad financial knowledge, prediction, and professional applications Xie et al. (2023, 2024), Matlin et…”

“FinQA[4],ConvFinQA[5],andTAT-QA[43]focuson arithmetic reasoning over financial tables and text.”

“FinQA (Chen et al., 2021) and ConvFinQA (Chen et al., 2022) focus on multi-step arithmetic over financial report excerpts, while TAT-QA (Zhu et al., 2021) combines tabular and textual evidence.”

“Chen et al. [2021] introduced FinQA, an expert-annotated dataset of numerical reasoning questions with executable reasoning programs.”

“They are widely used across real-world domains such as finance (Chen et al., 2021), healthcare (Yan et al., 2025), and scientific reporting (Moosavi et al., 2021; Zhang et al., 2025b), making the ability to effectively perform tasks over tabular data essential.”

“A diverse agent task set of 679 instances from IWG was gathered, spanning domains such as complex data analysis (Chen et al., 2021), research report writing (Yang et al., 2018), omni task (Mishra et al., 2019).”

“FinQA [6] tests table-grounded numerical reasoning over financial reports, and MedQA-USMLE [10] tests medical multiple-choice reasoning.”

“Several benchmarks have driven progress in financial QA: FinQA (Chen et al., 2021) introduced 8,281 question-answer pairs requiring numerical reasoning over S&P 500 reports; ConvFinQA (Chen et al., 2022) extended this to multi-turn conversations; TAT-QA (Zhu et al., 2021) mixed tabular and textual…”

“Adapting LLMs to specialized tasks typically requires extensive, high-quality annotated data D = ⟨ D x , D y ⟩ , whether derived from human (Chen et al., 2021) or model-assisted (Wu et al., 2025) sources.”

“For instance, CP DFA has the highest accuracy for the Mistral model (across all datasets) and on the FinQA dataset (finance domain questions across all models) while also providing better efficiency than CP FF .”

“Financial NLP is a fast-growing field, that applies NLP to financial data to solve complex challenges and has recently drawn attention for its diverse, impactful applications [6].”

“However, general VLMs are often criticized for their “hallucination” tendencies regarding numerical data [6, 19], which is a critical failure point for financial auditing.”

“To further evaluate the robustness and transferability of TableMind++, we extend our experiments to two out-of-domain datasets: HiTab [10] and FinQA [5].”

“We first aggregated authoritative cross-domain datasets, including software development (LiveCodeBench [14], SWE-bench [16]), financial (FinQA [7], BizFinBench [20]), STEM (GSM8K [8], SciQ [27]), and Legal/Admin (LEXam [9], Pile-of-law [12]).”

“This is the focus of FinQA (Chen et al., 2021) and TAT-QA (Yu et al., 2021), the latter being larger in scale and intentionally rich in numerical reasoning challenges.”

“TableQA has evolved from foundational datasets like WTQ (Pasupat and Liang, 2015) and TabFact (Chen et al., 2019), which primarily utilize Wikipedia-sourced tables, to more specialized domains such as aviation (AIT-QA (Katsis et al., 2022)), finance (TAT-QA (Zhu et al., 2021b), FinQA (Chen et al., 2021)) and others (C…”

“Financial benchmarks such as FinEval (Guo et al., 2025), BizBench (Krumdick et al., 2024), and FinQA (Chen et al., 2022) probe financial knowledge and quantitative reasoning.”

“We conduct experiments on three question-answering (QA) datasets from distinct domains: GSM8K [6] (mathematics), FinQA [5] (finance), and LegalBench [9] (law).”

“The training data is sourced from WikiTableQuestions (WTQ) [34] for table QA, HiTab [11] for hierarchical table reasoning, and FinQA [7] for numerical reasoning over financial tables.”

“Benchmarks like FinQA [5] and FinBen [26] focus on general applied reasoning skills useful for evaluating industry-capabilities but not always central to educational learning objectives.”

“Finally, domain specificity intro-a WikiTQ (Pasupat and Liang, 2015) Wikipedia (Wikipedia) ✗ ✗ ✗ ✗ ✗ 6.3 1133.51 TabFact (Chen et al., 2019) Wikipedia ✗ ✗ ✗ ✗ ✗ 6.2 586.51 Hybrid-QA (Chen et al., 2020b) Wikipedia ✓ ✗ ✗ ✓ ✗ 15.7 372.14 ∗ OTT-QA (Chen et al., 2020a) Wikipedia ✓ ✗ ✗ ✓ ✗ 15.7 372.14 ∗ CRT-QA (Zhang et al.…”

“Evaluation on financial QA benchmarks demonstrates that domain-adapted models significantly outperform general-purpose models, highlighting the value of financial corpus exposure during training [33].”

“…including MAWPS (Koncel-Kedziorski et al., 2016), DROP (Dua et al., 2019), MathQA (Amini et al., 2019), GSM8k (Cobbe et al., 2021), FinQA (Chen et al., 2021b), SVAMP (Patel et al., 2021), MATH (Hendrycks et al., 2021), AIME (Veeraboina, 2023), and Omni-MATH (Gao et al., 2025) require…”

“In contrast, practitioners and researchers working with financial data face highly specialized challenges, such as interpreting banking records, analyzing card transactions, managing retail and sales data, or handling loan and insurance queries [1, 39].”

“Early research focused on discrete reasoning (Jin et al., 2022), with works such as TAT-QA (Zhu et al., 2021), FinQA (Chen et al., 2021), and MVGE (Ma et al., 2017) exploring methods like fine-tuning, pre-training, and in-context learning.”

“Benchmarks like FinQA [4] and FinBen [24] focus on general applied reasoning skills useful for evaluating industry-capabilities but not always central to educational learning objectives.”

“• FinQA [39] is a dataset of QA pairs, where solving the problems require extracting relevant information from long financial reports.”

“Notably, financial QA datasets such as FinQA [8] and FinLLMs [45] illustrate the challenges and opportunities of reasoning over domain-specific documents with embedded numerical content.”

“2%accuracy and supports 100 concurrent queries, enhancing scalability, accuracy, compliance, and accessibility (Chen et al., 2021).”

“…datasets can be categorized into two types: those that use a raw table as direct input for model analysis (Pa-supat and Liang, 2015; Hwang et al., 2019), and those that use a table with additional annotated information (Zhu et al., 2021; Nguyen et al., 2023; Jauhar et al., 2016; Chen et al., 2021).”

“In English, FINQA by Chen et al. (2021) contains 8,281 question-answering pairs, with their numerical reasoning, from the earnings reports of S&P 500 companies.”

“Existing finance-focused benchmarks primarily assess LLMs on their ability to answer expert-level questions, covering areas such as financial natural language understanding [5, 6, 7], numerical reasoning [8, 9], and professional certification tests [10, 11].”

“Current approaches fall into two principal categories: financial question-answering datasets like FiQA [5] and FinQA [2] that lack retrieval integration capabilities, or comprehensive financial benchmarks such as FinBen [10] and PIXIU [9] that primarily focus on static model capabilities rather…”

“For instance, TAT-QA (Zhu et al., 2021) combines tables and text for numerical reasoning, while FinQA (Chen et al., 2022c) offers 8,281 expert-annotated QA pairs requiring math operations like addition and comparison.”

“This is the imperfect model’s inherent error, note that the accuracy for remote-only method on MultiHiertt/FinQA dataset is 66.4% and 73.7%.”

“FinQA [62] consist of tables from the finance domain but specifically exclude tables with complex nested structures.”

“To evaluate the performance of OkraLong comprehensively, we conduct experiments on six long-text query-answering datasets, spanning various domains and multiple query types: (1) FINQA (Chen et al., 2021) is a finanicl numerical reasoning dataset, constructed from the public earnings reports of S&P 500 companies.”

“To evaluate the model capabilities on the TATQA task, several datasets are proposed (Li et al., 2021; Chen et al., 2021; Zhao et al., 2024b).”

“We investigate this by contributing: • HiFi-KPI: An iXBRL-based dataset with 1.65M paragraphs and 4.5M entities, and a smaller HiFi-KPI-Lite with four expert-defined labels.”

“These proposed benchmarks cover a diverse set of domains, including Wikipedia tables (Parikh et al., 2020), financial tables (Chen et al., 2021b), scientific tables (Moosavi et al., 2021), which serve as invaluable sources for developing and testing general table understanding models.”

“Method step GPT-4 experts Datasets used PoT [6] PE FinQA, TAT-QA, TabMWP EEDP [26] PE FinQA, TAT-QA, TabMWP Chameleon [19] PE TabMWP ToRA [11] FT TabMWP Husky [17] FT FinQA, TabMWP TAT-LLM [37] FT FinQA, TAT-QA FinQANet [7] FT FinQA TagOP [36] FT TAT-QA”

“For the creation of the FinQA dataset [14] in the finance domain, the process starts by filtering the FinTab-Net dataset of earnings reports to remove complex nested structures.”

“This observation is consistent with the FinMA paper (Xie et al., 2023), which notes that these models were trained on a combination of instruction datasets, including FinQA.”

“The FinQA dataset (Chen et al., 2021) [3] has emerged as a top-tier publicly accessible dataset featuring more than 8,000 question-answer pairs carefully assembled by finance experts, broadening the range of assessments in financial question-answering.”

“While beneficial in education, these methods may not translate well to professional contexts where questions are complex and require domain expertise.”