The DOI matches its DataCite record; 12 of the 18 references it cites were found in a registry or an index.
Solving General Arithmetic Word Problems
Open accessAbstract
This paper presents a novel approach to automatically solving arithmetic word problems. This is the first algorithmic approach that can handle arithmetic problems with multiple steps and operations, without depending on additional annotations or predefined templates. We develop a theory for expression trees that can be used to represent and evaluate the target arithmetic expressions; we use it to uniquely decompose the target arithmetic problem to multiple classification problems; we then compose an expression tree, combining these with world knowledge through a constrained inference framework. Our classifiers gain from the use of {\em quantity schemas} that supports better extraction of features. Experimental results show that our method outperforms existing systems, achieving state of the art performance on benchmark datasets of arithmetic word problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Table 6 shows that our optimized prompts also outperform baseline prompts with different scorer LLMs on these two benchmarks.”
From Large Language Models as Optimizers (2023) · cites this paper as Roy & Roth, 2016
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning to Automatically Solve Algebra Word Problems
Kushman, N., Zettlemoyer, L., Barzilay, R., & Artzi, Y. (2014) · Annual Meeting of the Association for Computational Linguistics · doi:10.3115/v1/p14-1026 · Cited by 400 · cited 6×
“(Hos-seini et al., 2014) mentions the result of running the system of (Kushman et al., 2014) on AI2 dataset, and we report their result here.”
BackgroundMethodInfluentialThe Importance of Syntactic Parsing and Inference in Semantic Role Labeling
Punyakanok, V., Roth, D., & Yih, W.-T. (2008) · Computational Linguistics · doi:10.1162/coli.2008.34.2.257 · Cited by 399 · cited 1×
“This has been successfully used to incorporate world knowledge at inference time, as well as getting around the need for large amounts of jointly annotated data for structured prediction (Roth and Yih, 2005; Punyakanok et al., 2005; Punyakanok et al., 2008; Clarke and Lapata, 2006; Barzilay and Lapata, 2006; Roy et al…”
MethodSelf-citationLearning to Solve Arithmetic Word Problems with Verb Categorization
Hosseini, M. J., Hajishirzi, H., Etzioni, O., & Kushman, N. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1058 · Cited by 324 · cited 8×
“(Hosseini et al., 2014) mentions the result of running the system of (Kushman et al.”
BackgroundMethodResultInfluentialInteger linear programming inference for conditional random fields
Roth, D., & Yih, W.-T. (2005) · International Conference on Machine Learning · doi:10.1145/1102351.1102444 · Cited by 214 · cited 2×
“This has been successfully used to incorporate world knowledge at inference time, as well as getting around the need for large amounts of jointly annotated data for structured prediction (Roth and Yih, 2005; Punyakanok et al., 2005; Punyakanok et al., 2008; Clarke and Lapata, 2006; Barzilay and Lapata, 2006; Roy et al…”
BackgroundMethodSelf-citationModeling Biological Processes for Reading Comprehension
Berant, J., Srikumar, V., Chen, P., Linden, A. V., Harding, B., Huang, B., Clark, P. E., & Manning, C. D. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1159 · Cited by 205 · cited 1×
“The system described in (Berant et al., 2014) attempts to automatically answer biology questions, by extracting the structure of biological processes from text.”
BackgroundReasoning about Quantities in Natural Language
Roy, S., Vieira, T., & Roth, D. (2015) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00118 · Cited by 133 · cited 6×
“For the IL dataset, we acquired the system of (Roy et al., 2015) from the authors, and ran it with the same fold information.”
BackgroundMethodInfluentialSelf-citationStructured learning with constrained conditional models
Chang, M.-W., Ratinov, L., & Roth, D. (2012) · Machine Learning · doi:10.1007/s10994-012-5296-5 · Cited by 117 · cited 4×
“In particular, we use the L + I scheme of CCMs, which predicts structured output by independently learning several simple components, combining them at inference time.”
MethodInfluentialSelf-citationDiagram Understanding in Geometry Questions
Seo, M. J., Hajishirzi, H., Farhadi, A., & Etzioni, O. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v28i1.9146 · Cited by 105 · cited 1×
“There has also been efforts to solve geometry questions by jointly understanding diagrams and associated text (Seo et al., 2014).”
BackgroundElementary School Science and Math Tests as a Driver for AI: Take the Aristo Challenge!
Clark, P. (2015) · AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i2.19066 · Cited by 99 · cited 1×
“Our approach is related to work in understanding and solving elementary school standardized tests (Clark, 2015).”
MethodAggregation via Set Partitioning for Natural Language Generation
Barzilay, R., & Lapata, M. (2006) · North American Chapter of the Association for Computational Linguistics · doi:10.3115/1220835.1220881 · Cited by 94
VisKE: Visual knowledge extraction and question answering by visual verification of relation phrases
Sadeghi, F., Divvala, S., & Farhadi, A. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298752 · Cited by 89 · cited 1×
“A recent work (Sadeghi et al., 2015) tries to answer science questions by visually verifying relations from images.”
BackgroundConstraint-Based Sentence Compression: An Integer Programming Approach
Clarke, J., & Lapata, M. (2006) · Annual Meeting of the Association for Computational Linguistics · doi:10.3115/1273073.1273092 · Cited by 77
Research works printed but not linked: 3
- Y. Goldberg and M. Elhadad. 2010. An efficient algo- rithm for easy-first non-directional dependency pars- ing. In Human Language Technologies: The 2010 Annual Conference of the North American Chap- ter of the Association for Computational Linguistics, pages 742-750, Los Angeles, California, June.
- V. Punyakanok, D. Roth, and W. Yih. 2005. The ne- cessity of syntactic parsing for semantic role label- ing. In Proc. of the International Joint Conference on Artificial Intelligence (IJCAI), pages 1117-1123.
- D. Roth and W. Yih. 2004. A linear programming formulation for global inference in natural language tasks. In Hwee Tou Ng and Ellen Riloff, editors, Proc. of the Conference on Computational Natural Language Learning (CoNLL), pages 1-8. Associa- tion for Computational Linguistics.
Cited by
Sorted by citations
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
“• Math Word Problem Repository (Koncel-Kedziorski et al., 2016): AddSub (Hosseini et al., 2014): https://www.cs.washington.edu/nlp/arithmetic ; MultiArith (Roy and Roth, 2015), license: CC BY 4.0.”
InfluentialLarge Language Models Are Human-Level Prompt Engineers
Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., & Ba, J. (2022) · arXiv · doi:10.48550/arxiv.2211.01910 · Cited by 330
“Known as Zero-Shot-CoT, this prompting strategy improves the zero-shot performance of InstructGPT on MultiArith (Roy & Roth, 2016) from 17.7 to 78.7 and improves performance on GSM8K(Cobbe et al., 2021) from 10.4 to 40.7.”
MethodInfluentialA Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
“Mathematical Reasoning MATH [284], GSM8k [244], SVAMP [245], MultiArith [328], ASDiv [246], MathQA [329], AQUA-RAT [330], MAWPS [331], DROP [332], NaturalProofs [333], PISA [334], miniF2F [335], ProofNet [336]”
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language ModelsPreprint
Wang, L., Xu, W., Lan, Y.-H., Hu, Z.-Q., Lan, Y.-S., Lee, R. K.-W., & Lim, E.-P. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2305.04091 · Cited by 937
“The proposed method is evaluated on the ten benchmark datasets from three categories of reasoning problems: Arithmetic Reasoning: (1) the GSM8K (Cobbe et al., 2021) dataset of high quality linguistically diverse grade school math word problems created by human problem writers, (2) the SVAMP (Patel et al., 2021) benchm…”
BackgroundInfluentialIs ChatGPT a General-Purpose Natural Language Processing Task Solver?
Qin, C., Zhang, A., Zhang, Z., Chen, J., Yasunaga, M., & Yang, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.85 · Cited by 869
“5 with 20 different datasets covering 7 representative task categories: reasoning (MultiArith (Roy and Roth, 2015), GSM8K (Cobbe et al., 2021), AddSub (Hos-seini et al., 2014), AQUA-RAT (Ling et al., 2017), SingleEq (Koncel-Kedziorski et al., 2015), SVAMP (Patel et al., 2021), CSQA (Talmor et al.,…”
Background
Metadata
- Authors
- Roy, S., & Roth, D.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Corresponding author
- Roth, Dan
- ORCID
- Roth, Dan
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- Pages
- 1743-1752
- arXiv
- 1608.01413
- OpenAlex
- W2251935656
- Cited by
- 275
- References
- 18
- Citations by year
- 2026: 16 · 2025: 35 · 2024: 33 · 2023: 49 · 2022: 28 · 2021: 25 · 2020: 20 · 2019: 21 · 2018: 22 · 2017: 12 · 2016: 12
- FWCI
- 9.02
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Defense Advanced Research Projects Agency (FA8750-13-2-0008, FA8750)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Mathematics, Computing, and Information Processing
- Keywords
- arithmetic word problems, automated problem solving, expression trees, constrained inference, world knowledge, feature extraction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
