FinQA: A Dataset of Numerical Reasoning over Financial Data

“For all models, we use the Adam optimizer (Kingma and Ba, 2015).”

“Retriever The full page of the financial report can go beyond 2,000 tokens, which cannot be coped with the current popular QA models (Devlin et al., 2019).”

“, 2019) and RoBERTa (Liu et al., 2019), our proposed approach outperforms all other baselines and achieves an execution accuracy of 65.”

“3703 scales linearly with sequence length, like the Longformer (Beltagy et al., 2020).”

“Financial QA is more challenging than classic QA (Rajpurkar et al., 2018; Yang et al., 2018) because it requires the system to spot relevant information across heterogeneous sources, such as tables and unstructured texts, and then create a numerical reasoning path to connect all the information.”

“Financial QA is more challenging than classic QA (Rajpurkar et al., 2018; Yang et al., 2018) because it requires the system to spot relevant information across heterogeneous sources, such as tables and unstructured texts, and then create a numerical reasoning path to connect all the information.”

“This way the model is able to learn the program structures as patterns from very large-scale data (~40k for MathQA), but may fail on learning the reasoning paths.”

“The performance of using FinBert is no better than BERT-large, mostly because its pre-training corpus is limited (~30M words from news articles).”

“Another kind of QA datasets related to ours is the math word problem datasets, like MaWPS (Koncel-Kedziorski et al., 2016), MathQA (Amini et al.”

“On the other hand, Table QA benchmarks such as FinQA (Chen et al., 2021) and HeteQA (Yu et al., 2025) test complex reasoning but always provide a complete, structured table upfront.”

“• Financial Question Answering FinQA [1] introduced 8,281 numerical reasoning pairs with program supervision but limited scope to single documents.”

“Even advanced LLMs, such as GPT-4o (Hurst et al., 2024) and Llama3 (Dubey et al., 2024), exhibit substantial gaps compared to human experts in domain-specific question answering (QA), particularly in tasks involving quantitative reasoning, like financial analysis (Chen et al., 2021).”

“To strengthen the empirical rigor of the study, the retrieval-and-generation pipeline was evaluated on the FinQA dataset [10].”

“FinQA FinQA [9] is a question-answering dataset in the financial domain.”

“…domain multiply choices questions (MedMCQ (Pal et al., 2022), 3-shots, abbreviated as MMCQ), finance domain table question-answering (Fin-TQA (Chen et al., 2021), 8-shots, abbreviated as FTQA), social media domain sentiment analysis (TSA (Kharde and Sonawane, 2016), 3-shots), and e-commerce…”

“We evaluate SRM across a diverse range of decision-making scenarios, including mathematical reasoning on GSM8K (Cobbe et al., 2021), reasoning and planning in BlocksWorld (Valmeekam et al., 2023), and financial numeric reasoning on FinQA (Chen et al., 2021).”

“Moving beyond natural language understanding, benchmarks such as MMLU(Hendrycks et al., 2020) and BIG-bench (Srivastava et al., 2022) have been developed to evaluate general knowledge and reasoning abilities. domain-specific benchmarks have emerged across various fields, including PubMedQA (Jin et al., 2019) in medici…”

“The figures show that interval lengths are largest in FinQA, followed by Medical, and smallest in MMLU.”

“Other datasets, such as BioNumQA 29 , NQuAD 30 , FinQA 31 , DROP 32 , or datasets of math word problems require models to perform numerical reasoning and mathematical operations, but do not focus on detailed annotation of quantities and their context.”

“Key datasets include FinQA [1] and TAT-QA [25], emphasizing hybrid question answering from tables and text.”

“Referring to [26], [27], [7], [28], this study uses numerical reasoning that is commonly used, namely Reasoning results are given column names in the format "max + column name k".”

“While several benchmarks exist for evaluating QA models on tabular data (Chen et al., 2021; Nan et al., 2022; Osés Grijalba et al., 2024), much less attention has been given to assessing models’ abilities on structured lists.”

“This approach is inspired by numerous studies on number comprehension and reasoning within financial texts (Zhu et al., 2021; Chen et al., 2021; Nan et al., 2022; Yang et al., 2022; Shi et al., 2023), which collectively suggest that numerical information plays a critical role in understanding…”

“[22] show that numerical reasoning across tables and text is feasible (pp. 5–7).”

“They offer a data-driven and dynamic approach to comprehending complex financial information, aiding in financial forecasting, risk assessment, and informed investment decisions [22].”

“Recently, more and more domain-specific datasets are introduced in the NLP community to enrich fact-driven datasets in financial tasks, such as financial sentiment analysis [8], numerical reasoning [9] and multilingual topic classification [10].”

“Specifically, on FinQA, it outperforms the optimal Qwen2.”

“By limiting inputs to short, self-contained segments, existing financial QA benchmarks (Chen et al., 2021; Zhu et al., 2021) strip away deep hierarchy, artificially simplifying the task and masking the inability to explicate implicit cues.”

“The QA family – FinQA, ConvFinQA, TAT-QA, FinanceBench, DocFinQA – pairs each item with a gold span, number, or reasoning program, and grading reduces to matching it.”

“Similarly, hybrid or text-table QA datasets such as HybridQA (Chen et al., 2020b), OTT-QA (Chen et al., 2020a), and FinQA (Chen et al., 2021) target text and table question answering, but are not designed for the explicit extraction of triples with a grounding distributed over both text and tables.”

“Finally, we add special knowledge taks: time-related question from TimeQA (Chen et al., 2021), and financial reasoning FinQA (Chen et al., 2022b) and ConvFinQA (Chen et al., 2022a).”

“…systems capable of answering questions over hybrid data from real sources, such as financial reports and Wikipedia, which combine tables and text (Wenhu Chen, 2021; Chen et al., 2020, 2021; Li et al., 2022; Eisenschlos et al., 2021; Krichene et al., 2021; Li et al., 2021; Zhao et al., 2022; Zhu…”

“More specifically, (Zhu et al., 2021) and (Chen et al., 2021) proposed TAT-QA and FinQA for numerical calculation question-answering on hybrid data in the financial domain.”

“As an example, our analysis of the FinQANet model (Chen et al., 2021b) output showed that if the model generates an incorrect operator, it is about 30% more likely to commit other errors in the following”

“(Chen et al., 2021; Cheng et al., 2021) created datasets containing text and numerical tables aligned with them which are used for question answering with quatitative reasoning.”

“FinEval (Guo et al., 2025b) FinQA (Chen et al., 2021) ConvFinQA (Chen et al., 2022) FinMaster (Jiang et al., 2025) FinBen (Xie et al., 2024) FinanceReasoning (Tang et al., 2025 As the first large-scale financial evaluation benchmark to cover authentic business data from both Chinese and U.S. equity markets while integ…”

“Tabular QA benchmarks — WikiTableQuestions [Pasupat and Liang, 2015], TableBench [Wu et al., 2025], BIRD-SQL [Li et al., 2024a], FinQA [Chen et al., 2021] — all report accuracy-based metrics without systematic calibration analysis.”

“Financial benchmarks include FinQA, ConvFinQA, FinanceBench, FinBen, and Fi-nanceAgent [20, 21, 19, 22, 23].”

“Financial QA benchmarks ask about modern corporate filings, such as earnings reports [4] and full public filings [8], which come with machine‑readable text.”

“FinQA studies numerical reasoning over supplied reports, while FinAgentBench evaluates financial document retrieval (Chen et al., 2021; Choi et al., 2025).”

“FinQA evaluates numerical reasoning over financial documents [7]; TAT-QA combines tables and associated text [39], and ConvFinQA extends numerical reasoning to conversational chains [8].”

“Web-based applications, such as financial analysis platforms [5, 30], government information portals, academic search engines [7], and online technical support [3, 11], increasingly rely on understanding long and visually rich documents.”

“Financial benchmarks cover numerical question answering [2], research reports [13, 14, 43] and search [10].”

“…answers are checkable, and the underlying data is abundant and structured), and the field has moved from domain-specific models and static task suites [5, 6, 9, 21–23] to agentic evaluation in which the model must retrieve its own evidence through search and tools [3, 4, 8, 11, 12, 15, 26, 27].”

“FinQA [4] and TAT-QA [37] test numerical reasoning over financial text and tables; ConvFinQA [5] extends this setting to conversational reasoning.”

“A Extended Related Work Numerical QA over filings began with expert-annotated report excerpts (Chen et al., 2021; Zhu et al., 2021; Chen et al., 2022).”

“FinQA supplies expert-authored questions and reasoning programs; FinanceBench supplies evidence-grounded financial QA (Chen et al., 2021; Islam et al., 2023).”

“…as are the BEIR [84] sub-corpora built from Natural Questions [96] and HotpotQA [97] (CC BY-SA 4.0) and the chunked sub-corpora from PubMedQA [85], FinQA [86] and TAT-QA [87] (MIT; CC BY 4.0 on the Hugging Face mirrors) and CUAD [88] (CC BY 4.0), each with its document ids and an attribution file.”

“By contrast, FinQA and HotpotQA show higher re-1 User prompt.”

“FinRAGBench-V and PHANTOM provide evidence-or hallucination-oriented labels, while FinQA and TAT-QA provide correctness labels that can support UQ evaluation [7, 30, 67, 69].”

“…PDFs (Cabinet Office, FSA, MOF); Phase 2 dis-tils chain-of-thought mathematical reasoning from Qwen3-30B-A3B-Thinking-2507 (Qwen Team, 2025b); Phase 3 injects financial-domain knowledge via text-only financial QA (Chen et al., 2021; 2022; Zhu et al., 2021) and visual QA from government PDFs.”

“Chen et al. (2021) make this connection central to FinQA, an English-language dataset of expert-written questions over financial reports accompanied by reasoning programmes.”

“Prior work has addressed adjacent problems, including QA over financial reports and tables (Chen et al., 2021; Zhu et al., 2021; Chen et al., 2022; Zhao et al., 2022), time-series QA (Kong et al., 2025; Jing et al., 2026; Xie et al., 2024; Han et al., 2026; Chen et al., 2026; Nie et al., 2026; Xie…”

“…and datasets evaluate information extraction, numerical reasoning, question answering, forecasting, retrieval, and decision-oriented tasks over financial text, including FinQA, TAT-QA, PIXIU, FinBen, and open-book financial QA over public-company evidence in FinanceBench [4, 9, 32, 33, 37].”

“Finance uses the reasoning-annotated FinQA/ConvFinQA release [12, 13].”

“…our analysis in realistic RAG workloads, we collect publicly available question-answering datasets from four domains where RAG is actively deployed: finance (FinQA [9], TAT-QA [10]), biomedical (CovidQA [4]), legal (CuAD [11], PrivacyQA [12]), and general knowledge (TriviaQA [13], HotpotQA [45]).”

“The additional datasets—HiTab, FinQA, As shown in Figure 5(a), perception errors (recognition, locating, and color) account for about two fifths of the errors of Qwen3-VL-8B (167 of 405).”

“We evaluate this setting on MedQA [Jin et al., 2021], CaseHOLD [Zheng et al., 2021], and FinQA [Chen et al., 2021].”

“CaseHOLD and LexGLUE test legal interpretation [21, 22], whereas FinQA requires calculations grounded in financial reports [23].”

“Financial analysis is an important application of Cowork Agents [6, 7].”

“For thinking variants, we cap generations to 4,096 new tokens on all datasets except BioASQ, StreamingQA, and FinQA, as longer reasoning trajectories are almost always repetitive or unparseable.”

“A third task is FinQA, numerical reasoning over financial tables and text, where a calculator-tool agent answers questions under the same protocol (Chen et al., 2021).”

“…functions (Table 14), correctness, format/structure, tool-use, retrieval-grounding, numeric-match, and task-specific variants for SQL and FinQA [23], each returning list[float] , one score per completion; Appendix C catalogs all 26. combine_rewards(reward_funcs, weights) accepts callables or…”

“We construct a continual-learning sequence using ToolUse (Tang et al., 2023), FinQA (Chen et al., 2021), and SciKE-Bio (Feng et al., 2024).”

“FinQA contains financial reports combining prose passages and tables (Chen et al., 2021).”

“FinQA supplies numerical reasoning questions over financial reports [1].”

“Symbolic representations have also been used to make LLM reasoning more explicit, structured, and inspectable across domains [3, 37, 42, 43].”

“FinQA established financial numerical reasoning over heterogeneous reports as a distinct challenge [3], while LoRA made parameter-efficient specialization practical for large pretrained models [4].”

“5 ◦ C for a temperature, following benchmarks for numerical question answering [50, 51].”

“This structured representation supports data analysis across domains such as healthcare (Johnson et al., 2023), finance (Chen et al., 2021), and materials design (Choudhary et al., 2020).”

“Agent benchmarks supply interactive environments, mostly for software, web, and desktop tasks [11, 31, 32, 35], and healthcare and finance benchmarks mostly test knowledge, question answering, or bounded tool use [1–3, 9, 10, 12].”

“M ATH QA (Yu et al., 2024), ARC-C HALLENGE (Clark et al., 2018), O PEN B OOK QA (Mihaylov et al., 2018), MMLU (Hendrycks et al., 2021), T RIVIA QA (Joshi et al., 2017), BBH (BIG-bench authors, 2023), F IN QA (Chen et al., 2021), and L EGAL B ENCH (Guha et al., 2023).”

“We also include FinQA(Chen et al., 2022) 2024)), ranging from those with a few million parameters to those with several billion.”

“Financial benchmarks span numerical reasoning and question answering (Chen et al., 2021; Islam et al., 2023; Xie et al., 2024).”

“…LLM benchmarks test knowledge and numerical reasoning: broad suites such as FinBen, PIXIU, and FinEval (Xie et al., 2024, 2023; Guo et al., 2025), numerical and table reasoning (Chen et al., 2021, 2022; Zhu et al., 2021; Krumdick et al., 2024), and deposit and loan arithmetic (Lee et al., 2026).”

“…et al., 2024b) and the ChatQA2 training data (Xu et al., 2024b), supplemented by TriviaQA (Joshi et al., 2017) and multi-hop and numerical-reasoning QA datasets including HotpotQA, 2WikiMultihopQA, MuSiQue, and FinQA (Yang et al., 2018; Ho et al., 2020; Trivedi et al., 2022; Chen et al., 2021).”

“A major reason for this limitation is that the majority of current financial datasets are heavily skewed toward a single modality, such as pure text or numerical time-series data (Chen et al., 2021; Lai et al., 2025; Shah et al., 2023; Dong et al., 2024).”

“Financial QA and numerical reasoning benchmarks show that strong language models can struggle to combine narrative evidence, tables, and arithmetic consistently (Zhu et al., 2021; Chen et al., 2021, 2022; Reddy et al., 2024).”

“Examples range from document-based question answering (Zhou et al., 2026) and fact verification (Yao et al., 2023) to clinical reasoning (Tang et al., 2024; Wang et al., 2026) and business reasoning (Chen et al., 2021).”

“FinQA targets numerical reasoning over reports [8]; PIXIU and FinBen cover broad financial understanding and decision tasks [36, 37]; and FinTradeBench combines fundamental information with trading signals [1].”

“…block transcribes the official FinQA grammar: six arithmetic operations, four table-aggregation operations, const_* constants, and #N step references from Chen et al. [2], and predicted programs are executed by a transcription of the benchmark’s MIT-licensed official evaluation script.”

“Banking77 (Casanueva et al., 2020) evaluates single-turn intent classification, while FinBen (Xie et al., 2024a) and FinQA (Chen et al., 2021b) emphasize financial knowledge and document reasoning.”

“CUAD [17] annotates contract clauses for review tasks; LexGLUE [18] assembles legal understanding benchmarks; FinQA [19] targets numerical reasoning over financial filings.”

“Region-level grounding benchmarks (OCRBench v2 [24], CC-OCR [25], MMDocBench [26]) and financial QA suites (TAT-DQA [27], FinQA [28], DocMath-Eval [29]) focus on single-step localization or free-form numerical QA.”

“Domain benchmarks move evaluation into specialized settings, including legal reasoning [Guha et al., 2023], broad finance [Xie et al., 2024], filing-grounded QA [Islam et al., 2023], and financial reasoning over heterogeneous evidence [Chen et al., 2021b, Zhu et al., 2021, Chen et al., 2022].”