The DOI matches its DataCite record; 20 of the 20 references it cites were found in a registry or an index.
Using Large Language Model to Solve and Explain Physics Word Problems Approaching Human Level
Open accessAbstract
Our work demonstrates that large language model (LLM) pre-trained on texts can not only solve pure math word problems, but also physics word problems, whose solution requires calculation and inference based on prior physical knowledge. We collect and annotate the first physics word problem dataset-PhysQA, which contains over 1000 junior high school physics word problems (covering Kinematics, Mass&Density, Mechanics, Heat, Electricity). Then we use OpenAI' s GPT3.5 to generate the answer of these problems and found that GPT3.5 could automatically solve 49.3% of the problems through zero-shot learning and 73.2% through few-shot learning. This result demonstrates that by using similar problems and their answers as prompt, LLM could solve elementary physics word problems approaching human level performance. In addition to solving problems, GPT3.5 can also summarize the knowledge or topics covered by the problems, provide relevant explanations, and generate new physics word problems based on the input. Our work is the first research to focus on the automatic solving, explanation, and generation of physics word problems across various types and scenarios, and we achieve an acceptable and state-of-the-art accuracy. This underscores the potential of LLMs for further applications in secondary education.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Second, prompting methods like CoT encourage models to reason explicitly, reducing reliance on shallow text-based pattern-matching and enabling discovery of more nuanced causal and spatial relationships (Wei et al., 2022b; Ding et al., 2023).”
From Large Language Model Reasoning Failures (2026) · cites this paper as Ding et al., 2023
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.18223 · Cited by 2,090
Are NLP Models really able to Solve Simple Math Word Problems?
Patel, A., Bhattamishra, S., & Goyal, N. (2021) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2103.07191 · Cited by 1,423
MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms
Amini, A., Gabriel, S., Lin, S., Koncel-Kedziorski, R., Choi, Y., & Hajishirzi, H. (2019) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n19-1245 · Cited by 956
Solving General Arithmetic Word Problems
Roy, S., & Roth, D. (2016) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1202 · Cited by 702
Deep Neural Solver for Math Word Problems
Wang, Y., Liu, X., & Shi, S. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1088 · Cited by 347
MathPrompter: Mathematical Reasoning using Large Language ModelsPreprint
Imani, S., Du, L., & Shrivastava, H. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2303.05398 · Cited by 312
Measuring Massive Multitask Language Understanding
Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., & Steinhardt, J. (2020) · arXiv · doi:10.48550/arxiv.2009.03300 · Cited by 291
Solving General Arithmetic Word Problems
Roy, S., & Roth, D. (2016) · arXiv · doi:10.48550/arxiv.1608.01413 · Cited by 275 · cited 1×
“As one of the most focused areas in natural language processing, automatic solution of Math word problem (MWP) reflects the capability of artificial intelligence (Roy and Roth, 2015; Zhang et al., 2019; Patel et al., 2021).”
BackgroundGeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning
Chen, J., Tang, J., Qin, J., Liang, X., Liu, L., Xing, E. P., & Lin, L. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.46 · Cited by 244
Parsing Algebraic Word Problems into Equations
Koncel-Kedziorski, R., Hajishirzi, H., Sabharwal, A., Etzioni, O., & Ang, S. D. (2015) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00160 · Cited by 242
A Goal-Driven Tree-Structured Neural Model for Math Word Problems
Xie, Z., & Sun, S. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/736 · Cited by 203
How well do Computers Solve Math Word Problems? Large-Scale Dataset Construction and Evaluation
Huang, D.-Q., Shi, S., Lin, C.-Y., Yin, J., & Ma, W.-Y. (2016) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p16-1084 · Cited by 158
Translating a Math Word Problem to a Expression Tree
Wang, L., Wang, Y., Cai, D., Zhang, D., & Liu, X. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1132 · Cited by 156
An Overview of Problem Solving Studies in Physics Education
İnce, E. (2018) · Journal of Education and Learning · doi:10.5539/jel.v7n4p191 · Cited by 153
The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers
Zhang, D., Wang, L., Zhang, L., Dai, B. T., & Shen, H. T. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2914054 · Cited by 149
MathDQN: Solving Arithmetic Word Problems via Deep Reinforcement Learning
Wang, L., Zhang, D., Gao, L., Song, J., Guo, L. Z., & Shen, H. T. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11981 · Cited by 124
Lu, L., Jin, P., & Karniadakis, G. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.03193 · Cited by 79
LogicSolver: Towards Interpretable Math Word Problem Solving with Logical Prompt-enhanced Learning
Yang, Z., Qin, J., Chen, J., Lin, L., & Liang, X. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.1 · Cited by 32
Neural Sequence Modeling in Physical Language Understanding
Bleiweiss, A. (2019) · International Joint Conference on Computational Intelligence · doi:10.5220/0008071104640472
Research works printed but not linked: 4
- Savitha Sam Abraham and Deepak Khemani. 2016. Hy- brid of qualitative and quantitative knowledge models for solving physics word problems. In The Twenty- Ninth International Flairs Conference.
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901.
- Iddo Drori, Sarah Zhang, Reece Shuttleworth, Leonard Tang, Albert Lu, Elizabeth Ke, Kevin Liu, Linda Chen, Sunny Tran, Newman Cheng, et al. 2022. A neural network solves, explains, and generates uni- versity math problems by program synthesis and few- shot learning at human level. Proceedings of the Na- tional Academy of Sciences, 119(32):e2123433119.
- Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirec- tional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171-4186.
Cited by
Sorted by citations
Large Language Model Reasoning Failures
Song, P., Han, P., & Goodman, N. (2026) · arXiv · doi:10.48550/arxiv.2602.06176 · Cited by 3
Metadata
- Authors
- Ding, J., Cen, Y., & Wei, X.
- Affiliations
- Columbia University (US)
- Fudan University (CN)
- ORCID
- Wei, Xinyuan
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2309.08182
- OpenAlex
- W4386841311
- Cited by
- 4
- References
- 20
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Fudan University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- large language models, GPT-3.5, zero-shot learning, few-shot learning, automated problem solving, problem generation, junior high school physics, secondary education, physics education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
