FinQA: A Dataset of Numerical Reasoning over Financial Data

“For all models, we use the Adam optimizer (Kingma and Ba, 2015).”

“Retriever The full page of the financial report can go beyond 2,000 tokens, which cannot be coped with the current popular QA models (Devlin et al., 2019).”

“, 2019) and RoBERTa (Liu et al., 2019), our proposed approach outperforms all other baselines and achieves an execution accuracy of 65.”

“3703 scales linearly with sequence length, like the Longformer (Beltagy et al., 2020).”

“Financial QA is more challenging than classic QA (Rajpurkar et al., 2018; Yang et al., 2018) because it requires the system to spot relevant information across heterogeneous sources, such as tables and unstructured texts, and then create a numerical reasoning path to connect all the information.”

“Financial QA is more challenging than classic QA (Rajpurkar et al., 2018; Yang et al., 2018) because it requires the system to spot relevant information across heterogeneous sources, such as tables and unstructured texts, and then create a numerical reasoning path to connect all the information.”

“This way the model is able to learn the program structures as patterns from very large-scale data (~40k for MathQA), but may fail on learning the reasoning paths.”

“The performance of using FinBert is no better than BERT-large, mostly because its pre-training corpus is limited (~30M words from news articles).”

“Another kind of QA datasets related to ours is the math word problem datasets, like MaWPS (Koncel-Kedziorski et al., 2016), MathQA (Amini et al.”

“Unlike open-domain QA, financial QA demands a higher level of precision, disambiguation, and technical understanding, which makes it uniquely challenging and error-prone [6, 7, 41, 59, 62].”

“Question Type (Task) English FINQA (Chen et al., 2021) Question Answering TAT-QA (Zhu et al., 2021) Question Answering BizBench (Koncel-Kedziorski et al., 2023) Quantitative Reasoning FINANCEBENCH (Islam et al., 2023) Question Answering FLUE (Shah et al., 2022) Sentiment Classification (Malo et…”

“Compare to previous multi-hop QA only focuses on textual data, FinQA (Chen et al., 2021), which extracted from financial reports, includes more kinds of information, such as table and number, and therefore some specific numerical operation is needed.”

“To assess generalization of To address RQ1, we conduct a comprehensive evaluation and summarize the key findings from the results presented in Table 2 as follows:”

“FinQA (from DuEE-fin) (Lu et al., 2023) Chinese Financial news bulletin event quiz dataset.”

“The task utilizes the FinQA dataset (Chen et al., 2021), which is designed to evaluate complex question answering and reasoning over both natural language and structured tables in the financial domain.”

“We also include large open QA datasets like FinQA (Chen et al., 2021).”

“Existing finance datasets, including TAT-QA (Zhu et al., 2021), FinQA (Chen et al., 2021), Multi-Hiertt (Zhao et al., 2022), PACIFIC (Deng et al., 2022) and ConvFinQA (Chen et al., 2022), focus on quantity extraction and basic numerical reasoning tasks when provided with company’s financial reports.”

“Recent studies have explored the effectiveness of various models and approaches in comprehending financial documents through Question-Answering (QA) tasks [3, 4].”

“These questions are much more challenging than existing financial QA tasks such as FinQA(Chen et al., 2021) or TAT-QA (Zhu et al., 2021).”

“Correct Option : B Current research delves into exploring mathematical INT [190], Feit-Thompson [64], and IsarStep [92]) to develop novel machine theorem-proving strategies.”

“We used the results of Long-former and NeRd reported in Chen et al. (2021b).”

“Moreover, as emphasised by Chen et al. (2021), the complexity of financial data is further increased by its heterogeneous nature, including both structured and unstructured data sources.”

“The second is the numerical reasoning question, which usually aims to use the contents of tables and texts for numerical calculation (Zhu et al., 2021; Chen et al., 2021).”

“…QA benchmarks have been proposed to evaluate models’ performance in answering questions based on financial documents, with a particular focus on numerical reasoning (Chen et al., 2021; Zhu et al., 2021; Zhao et al., 2022; Chen et al., 2022b; Koncel-Kedziorski et al., 2024; Zhao et al., 2024b,a).”

“Comparison A3: Rafer Johnson 2020b,a; Zhu et al., 2021; Chen et al., 2021; Zhao et al., 2022; Wang et al., 2022a).”

“In code reasoning, benchmarks such as HumanEval (Chen et al., 2021a) and MBPP (Austin et al., 2021) challenge LLMs to generate correct and functional code by reasoning through programming tasks.”

“• We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.”

“These tasks include performing arithmetic operations and understanding numerical relationships, which add a layer of complexity and sophistication to the benchmarks [36, 37].”

“…understanding and knowledge acquisition (Wang et al., 2019; Hendrycks et al., 2021a; Zhou et al., 2023; Wang et al., 2024), code generation (Chen et al., 2021a; Liu et al., 2023; Jimenez et al., 2024), mathematical reasoning (Cobbe et al., 2021; Hendrycks et al., 2021b), tool use (Yan et…”

“For verifying numerical claims, we also use FinQA-RoBERTa-Large [20] fine-tuned using the financial QA dataset, FinQA [4].”

“We test our TAT-LLM on three popular tabular and textual QA benchmarks: FinQA [7], TAT-QA [54] and TAT-DQA [53].”

“We conduct experiments on two challenging datasets: FinQA [15], and MathQA [19].”

“FinQA (Chen et al., 2021) is a recent work similar to ours, aiming to address the task of numerical reasoning over financial data.”

“Further evaluation results with complementary metrics, including BERTScore (Zhang et al., 2020) and numeric-consistency checks (Chen et al., 2021a), are provided in Table 9 (Appendix H.12).”

“Early financial NLP benchmarks [5, 6, 22, 33, 38, 42] such as Fin-Ben [41] expanded evaluation beyond sentiment and QA to include retrieval augmented tasks and multi-step decision-making problems, yet they remained largely static and English-centric.”

“- investor_name = sample(investors) - project_name = sample(projects) - principal = range(1000, 5000) - rate = uniform(2, 10) - time = range(1, 5)”

“…growing emphasis on developing IR benchmarks in specialized domains, such as law (Xiao et al., 2019; Li et al., 2023), finance (Jangid et al., 2018; Chen et al., 2021; Zhao et al., 2024b), scientific literature (Wu et al., 2024; Ajith et al., 2024), and healthcare (Roberts et al., 2020; Tamine and…”

“Reranking includes a set of financial datasets that have the ranking of retrieved documents to user queries such as FinQA2018-Rerank (Chen et al., 2021).”

“ConvFinQA builds upon FinQA by introducing multi-turn conversations for question-answering tasks using earnings reports.”

“…in areas like question answering (Dua et al., 2019; Andor et al., 2019; Ran et al., 2019; Herzig et al., 2020; Chen et al., 2020a; Yin et al., 2020; Chen et al., 2021) and math word problems (MWPs) solving (Xie and Sun, 2019b; Amini et al., 2019; Koncel-Kedziorski et al., 2016; Hendrycks et al.,…”

“To further evaluate the robustness and transferability of TableMind, we extend our experiments to two out-of-domain datasets: HiTab [7] and FinQA [2].”

“In contrast, we focus on fine-tuning small language models for financial question answering problems (Chen et al., 2021b, 2022; Zhu et al., 2021).”

“This isolates the core retrieval bottleneck in financial QA: locating grounded evidence within long documents before generation (Zhu et al., 2021; Chen et al., 2021).”

“Existing methods often use exact match (Zhu et al., 2021), F1-score (Chen et al., 2021b), or n-gram-matching (Parikh et al., 2020; Nan et al., 2022) metrics.”

“Homomor-phic encryption similarly faces high inference, memory, and communication costs.”

“FinQA [61] requires multi-step numerical reasoning over financial reports—e.g., computing growth rates or combining multiple cells with arithmetic operators.”

“To test the individual modules and end-to-end pipeline, we conducted experiments using two datasets: the S&P 500 companies’ earnings reports [32] and the FinQA dataset [7].”

“Numerical reasoning over text is an essential skill for a neural model to help analyze rich numerical information from large-scale textual data (Chen et al., 2021).”

“…(2) Countdown , where the model needs to construct a target number from given inputs using basic arithmetic operators (Pan et al., 2025); (3) MATH , focused on mathematical problem solving (Hendrycks et al., 2021); and (4) FinQA , financial reasoning over tabular data and text (Chen et al., 2021).”

“Detecting errors across the hierarchical and structured financial documents requires understanding semantics, hierarchical relations, and numerical logic.”

“…Answer EM,F1 MultiHiertt (Zhao et al., 2022) Spreadsheet Table Answer EM,F1 OTT-QA (Chen et al., 2021a) Semi-Structured Table Answer EM,F1 FinQA (Chen et al., 2021b) Spreadsheet Table Answer Acc FeTaQA (Nan et al., 2022) Spreadsheet Table Sentence Answer BLEU TAT-QA (Nan et al., 2022)…”

“HybridQA (Chen et al., 2020b) derived from Wikipedia emphasises span lookup, while TAT-QA (Zhu et al., 2021), FinQA (Chen et al., 2021), and DocMath-Eval (Zhao et al., 2024) address numerical reasoning in the financial domain.”

“HiTab (HIT) (Cheng et al., 2022), TabMWP (TMP) (Lu et al., 2023b) and FinQA (FQA) (Chen et al., 2021) datasets focus on numerical reasoning reasoning.”

“Based on the HybridQA, Chen et al. (2021a) further pro-pose the OTT-QA dataset, which requires the sys-tem to retrieve relevant tables and text for the given questions.”

“Applying these models in the finance domain (Xie et al., 2023; Wu et al., 2023a; Yang et al., 2023) presents additional challenges in terms of interpreting hybrid data (Zhu et al., 2021) and utilizing domain-specific expertise (Chen et al., 2021; Zhao et al., 2024).”

“More recent work extends financial QA toward numerical and long-form reasoning over reports, as in FinQA (Chen et al., 2021) and TAT-QA (Zhu et al., 2021), and toward multi-task settings that integrate retrieval and multi-step reasoning, including PIXIU (Xie et al., 2023), FinBen (Xie et al., 2024), EMS (Hu et al., 20…”

“Notable examples include FinQA (Chen et al., 2021b), FinTextQA (Chen et al., 2024), and FinDER (Choi et al., 2025).”

“We begin by constructing a high-quality reasoning dataset, DianJin-R1-Data, using three major sources: CFLUE (Zhu et al., 2024), FinQA (Chen et al., 2021), and our proprietary compliance dataset for the task of Chinese Compliance Check (CCC).”

“Besides, several datasets are developed to evaluate LLMs in various financial NLP tasks, such as sentiment analysis (Malo et al., 2014; FiQA), text classification (Sinha and Khandait, 2020; Shah et al., 2023) or question answering (Chen et al., 2021, 2022).”

“…, github_repos ) [4, 14], local business and reviews ( googlelocal , yelp ) [18, 27, 46], music ( music_brainz_20k ) [38, 39], financial markets ( stockindex , stockmarket ) [32], medical research pancancer_atlas patents The crmarenapro dataset and its queries are drawn from the CR-MArena”

“…2024), selected to cover (Islam et al., 2023) Real-world questions written by non-experts TATQA(Zhu et al., 2021) Basic arithmetic questions FinQA(Chen et al., 2021) Complex arithmetic questions ConvFinQA(Chen et al., 2022) Questions asking for specific value from table values MultiHiertt(Zhao…”

“…and limited table contexts; Open-Domain DatasetssuchasOTTQA[3]andNQ-tables[15],designedfor broader generalization across various tables; and Hybrid Datasets like TAT-QA[46] and FinQA[4], which combine tables with surrounding text and require models to handle both structured and unstructured data.”

“In this domain, widely recognized datasets include FinQA [4] and TAT-QA [5].”

“like Wikidata or DBpedia [25, 47]; (iii) they span only two sources, like tables and text [10, 12, 50], or text and knowledge bases [25, 38, 47]; (iv) they explore only one domain like finance [12, 50], geography [24], or e-commerce [34]; and (v) their questions are only in conversational form with implicit intent, un…”

“Domain-specific evaluation includes FinQA [52] for financial reasoning, MedQA [53] for medical knowledge assessment, LegalQA [54] for legal understanding, and SciQ [55] for scientific reasoning.”

“Benchmarks like ChartQA [24] and ChartX [38] have begun exploring visual reasoning over scientific figures.”

“…lookups or aggregations; fact-checking queries (TabFact (Chen et al., 2019)) that require claim verification; numerical reasoning queries (FinQA (Chen et al., 2021b)) necessitating multi-step calculations; and multi-modal queries (FinQA and OTT-QA (Chen et al., 2021a)) that demand reasoning…”

“…significantly curtails misclassification and enhances factual grounding, but key challenges persist: numeric reasoning remains difficult, factual “hallucinations” can occur in extended text generation, and certain real-time analyses require continuous updates (Chen et al., 2022; Ji et al., 2023).”

“• ConvFinQA : The ConvFinQA dataset (Chen et al., 2022) extends FinQA by including conversational aspects, making the question-answering process more complex.”

“Other tabular datasets are constructed from financial reports including TAT-QA [60], F IN QA [8], M ULTI H IERTT [59], or arXiv iTBLS [46].”

“A substantial body of work has developed benchmarks for table understanding, spanning question answering over structured tables, fact verification against tabular evidence, and numerical reasoning over hybrid tabular-textual data [31, 32, 33, 34, 35, 36, 37, 20].”

“…in NLP and multimodal research, as structured data in the form of tables is pervasive across diverse domains such as scientific research (Van Breugel and Van Der Schaar, 2024; Li et al., 2024a), finance (Chen et al., 2021; Katsis et al., 2022), and education (Lu et al., 2023; Kang et al., 2025).”

“Works such as (Chen et al., 2020; Zhu et al., 2021; Chen et al., 2021; Zhao et al., 2022) integrate paragraph context alongside tables for enhanced tabular question answering.”

“As LLM capabilities advanced, more domain-specific benchmarks emerged, extending evaluation to fields like science reasoning (ScienceQA) (Lu et al., 2022), financial reasoning (FinQA) (Chen et al., 2022b), and theorem comprehension (The-oremQA) (Chen et al., 2023b).”

“Further evaluation results with complementary metrics, including BERTScore (Zhang et al., 2020) and numeric-consistency checks (Chen et al., 2021a), are provided in Table 9 (Appendix H.12).”

“Applying these models in the finance domain (Xie et al., 2023; Wu et al., 2023a; Yang et al., 2023b) presents additional challenges in terms of interpreting hybrid data (Zhu et al., 2021) and utilizing domain-specific expertise (Chen et al., 2021; Zhao et al., 2023a).”

“The development of Large Language Models (LLMs) has brought significant benefits to the financial domain across a range of applications (Chen et al., 2021; Zhu et al., 2021; Zhao et al., 2022; Yang et al., 2023; Xie et al., 2023; Gruver et al., 2023; Nie et al., 2024; Zhao et al., 2024b; Peng et…”

“Early efforts include FiQA (Maia et al., 2018) for sentiment analysis and opinionated QA, while FinQA (Chen et al., 2021) and its conversational extension ConvFinQA (Chen et al., 2022) offer more realistic, multi-turn interactions.”

“Question Answering using sensor data Question Answering has attracted extensive interests in the field of Natural Language Processing (NLP), with a wide range of datasets designed to address various language tasks [6, 9, 11, 20, 21, 23, 27, 40, 55, 58].”

“For finanical Q&A task, the FinQA (Chen et al., 2021) dataset focuses on multi-step numerical reasoning through financial reports.”

“• FinQA [5]: We use the publicly available train split as part of the training set or generating tool-integrated solution trajectories, and the test split as part of the evaluation set.”

“Authors also show that the n-gram contamination detection method from Llama-2 Touvron et al. (2023b) is not effective at detecting contaminated data points, calling for more robust methods.”

“Examples include FinanceBench [12], OpenFinData [19], and report-focused datasets such as FinQA [4] and TAT-QA [33].”

“FinQA (Chen et al., 2021) ✗ ✗ ✗ ✗ ConvFinQA (Chen et al., 2022) ✗ ✗ ✗ ✗ FLUE (Shah et al., 2022) ✗ ✗ ✗ ✗ FinEval (Zhang et al., 2023) ✗ ✗ ✗ ✗ BizFinBench (Bigeard et al., 2025) ✗ ✗ ✗ ✗ FinAgentBench (Bigeard et al., 2025) ✗ ✗ ✗ ✗ FinSearchComp (Hu et al., 2025) ✓ ✗ ✗ ✗ INVESTORBENCH (Li et al.,…”

“We use as prompts the long-answer sub-tasks of W ATER B ENCH : 200 questions from the ELI-5 dataset (Fan et al., 2019), 200 questions from the F INANCE -QA dataset (Chen et al., 2021), and 200 questions from the A LPACA -”

“Datasets like FinQA [5] and TAT-QA [30] test joint reasoning over tables and text, while others focus on navigating complex hierarchical structures [9, 23, 27].”

“In this section, we detail our process for constructing open-domain QA pairs by leveraging three established financial QA benchmarks: FinQA (Chen et al., 2021), Financebench (Islam et al., 2023), and SEC-QA (Lai et al., 2024).”

“FinQA provides numerical reasoning challenges over financial reports with hybrid text-table evidence (Chen et al., 2021).”

“TAT-QA (Zhu et al., 2021), FinQA (Chen et al., 2021), and DocMath-Eval (Zhao et al., 2024b) primarily address the numerical reasoning task in the financial domain.”

“We fully translated datasets for text-centric tasks like FinSA and FinHC, while translating only the validation sets for the more intricate FinSP and FinQA tasks.”

“We use a DSL similar to (Chen et al., 2021b) with two differences: we don’t have table operators and instead we have a max and min operator.”

“For FinQA, Chen et al. (2021) propose program accuracy where the model is expected to generate a ‘program’ along with the answer.”

“Nowadays, there are relevant research efforts to apply LLMs to fintech, such as sentiment analysis [134], news classification [135], and question answering [136].”