“Mathematical word problem generation from commonsense knowledge graph and equations,”
Open accessAbstract
There is an increasing interest in the use of mathematical word problem (MWP) generation in educational assessment.Different from standard natural question generation, MWP generation needs to maintain the underlying mathematical operations between quantities and variables, while at the same time ensuring the relevance between the output and the given topic.To address above problem, we develop an end-to-end neural model to generate diverse MWPs in real-world scenarios from commonsense knowledge graph and equations.The proposed model (1) learns both representations from edge-enhanced Levi graphs of symbolic equations and commonsense knowledge; (2) automatically fuses equation and commonsense knowledge information via a self-planning module when generating the MWPs.Experiments on an educational gold-standard set and a large-scale generated MWP set show that our approach is superior on the MWP generation task, and it outperforms the SOTA models in terms of both automatic evaluation metrics, i.e., BLEU-4, ROUGE-L, Self-BLEU, and human evaluation metrics, i.e., equation relevance, topic relevance, and language coherence.To encourage reproducible results, we make our code and MWP dataset public available at https:// github.com/tal-ai/MaKE_EMNLP2021.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Subsequent research on automatic MWP generation has focused on improving topic relevance, expression relevance, language coherence, as well as completeness and validity of the generated problems.
“Subsequent research along this direction has focused on improving topic relevance, expression relevance, language coherence, as well as completeness and validity of the generated problems using a spectrum of approaches (Liu et al., 2021; Wang et al., 2021; Wu et al., 2022).”
From Proactive and reactive engagement of artificial intelligence methods for education: a review (2023) · cites this paper as Liu et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
(2004) · Cited by 9,718 · see citing works
Kudo, T., & Richardson, J. (2018) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-2012 · Cited by 4,454 · see citing works
Meteor Universal: Language Specific Translation Evaluation for Any Target Language
Denkowski, M., & Lavie, A. (2014) · WMT@ACL · doi:10.3115/v1/w14-3348 · Cited by 2,108 · see citing works
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
Bengio, S., Vinyals, O., Jaitly, N., & Shazeer, N. (2015) · arXiv · doi:10.48550/arxiv.1506.03099 · Cited by 1,621 · see citing works
Unified Language Model Pre-training for Natural Language Understanding and Generation
Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., Gao, J., Zhou, M., & Hon, H.-W. (2019) · arXiv · doi:10.48550/arxiv.1905.03197 · Cited by 1,225 · see citing works
MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms
Amini, A., Gabriel, S., Lin, S., Koncel-Kedziorski, R., Choi, Y., & Hajishirzi, H. (2019) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n19-1245 · Cited by 956 · see citing works
Zhao, T., Zhao, R., & Eskenazi, M. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1061 · Cited by 736 · see citing works
MAWPS: A Math Word Problem Repository
Koncel-Kedziorski, R., Roy, S., Amini, A., Kushman, N., & Hajishirzi, H. (2016) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n16-1136 · Cited by 551 · see citing works
A Systematic Review of Automatic Question Generation for Educational Purposes
Kurdi, G., Leo, J., Parsia, B., Sattler, U., & Al-Emari, S. (2020) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-019-00186-y · Cited by 514 · see citing works
Texygen: A Benchmarking Platform for Text Generation Models
Zhu, Y., Lu, S., Zheng, L., Guo, J., Zhang, W., Wang, J., & Yu, Y. (2018) · The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval · doi:10.1145/3209978.3210080 · Cited by 408 · see citing works
Graph-to-Sequence Learning using Gated Graph Neural Networks
Beck, D., Haffari, G., & Cohn, T. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1026 · Cited by 380 · see citing works
Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing
Fu, H., Li, C., Liu, X., Gao, J., Çelikyılmaz, A., & Carin, L. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1021 · Cited by 272 · see citing works
Generating Natural Questions About an Image
Mostafazadeh, N., Misra, I., Devlin, J., Mitchell, M., He, X., & Vanderwende, L. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1170 · Cited by 270 · see citing works
Answer-focused and Position-aware Neural Question Generation
Sun, X., Liu, J., Lyu, Y., He, W. P., Ma, Y., & Wang, S. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1427 · Cited by 185 · see citing works
Visual Question Generation as Dual Task of Visual Question Answering
Li, Y., Duan, N., Zhou, B., Chu, X., Ouyang, W., Wang, X., & Zhou, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00640 · Cited by 171 · see citing works
Texygen: A Benchmarking Platform for Text Generation ModelsPreprint
Zhu, Y., Lu, S., Zheng, L., Guo, J., Zhang, W., Wang, J., & Yu, Y. H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.01886 · Cited by 168 · see citing works
Zhao, T., Zhao, R., & Eskénazi, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.10960 · Cited by 164 · see citing works
Machine Comprehension by Text-to-Text Neural Question Generation
Yuan, X., Wang, T., Gülçehre, Ç., Sordoni, A., Bachman, P., Zhang, S., Subramanian, S., & Trischler, A. (2017) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w17-2603 · Cited by 158 · see citing works
Cyclical Annealing Schedule: A Simple Approach to Mitigating KL VanishingPreprint
Fu, H., Li, C., Liu, X., Gao, J., Çelikyılmaz, A., & Carin, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.10145 · Cited by 153 · see citing works
MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based FormalismsPreprint
Amini, A., Gabriel, S., Lin, P., Koncel-Kedziorski, R., Choi, Y., & Hajishirzi, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13319 · Cited by 137 · see citing works
Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering
Zhang, S., & Bansal, M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1253 · Cited by 115 · see citing works
Recent Advances in Neural Question GenerationPreprint
Pan, L., Lei, W., Chua, T., & Kan, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.08949 · Cited by 79 · see citing works
Solving Verbal Problems: Results and Implications from National Assessment
Carpenter, T. P., Corbitt, M. K., Kepner, H. S., et al. (1980) · The Arithmetic Teacher · doi:10.5951/at.28.1.0008 · Cited by 65 · see citing works
Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherence
Guan, J., Mao, X., Fan, C., Liu, Z., Ding, W., & Huang, M. (2021) · doi:10.18653/v1/2021.acl-long.499 · Cited by 64 · see citing works
Self-citationZhou, Q. S., Yang, N., Wei, F., & Zhou, M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11915 · Cited by 56 · see citing works
Question Generation from SQL Queries Improves Neural Semantic Parsing
Guo, D., Sun, Y., Tang, D., Duan, N., Yin, J. P., Chi, H., Cao, J., Chen, P., & Zhou, M. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1188 · Cited by 56 · see citing works
Teaching Machines to Ask Questions
Yao, K., Zhang, L., Luo, T., Tao, L., & Wu, Y. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/632 · Cited by 44 · see citing works
Personalized mathematical word problem generation
Polozov, O., O’Rourke, E., Smith, A. M., Zettlemoyer, L., Gulwani, S., & Popović, Z. (2015) · Cited by 42 · see citing works
Gated Graph Convolutional Recurrent Neural Networks
Ruiz, L., Gama, F., & Ribeiro, A. (2019) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2019.8902995 · Cited by 37 · see citing works
Story Problem Formats: Drawn versus Verbal versus Telegraphic
Moyer, J. C., Sowder, L., Threadgill-Sowder, J. A., et al. (1984) · Journal for Research in Mathematics Education · doi:10.5951/jresematheduc.15.5.0342 · Cited by 36 · see citing works
Story Problem Formats: Drawn versus Verbal versus Telegraphic
Moyer, J. C., Sowder, L., Threadgill-Sowder, J. A., et al. (1984) · Journal for Research in Mathematics Education · doi:10.2307/748424 · Cited by 35 · see citing works
OpenMEVA: A Benchmark for Evaluating Open-ended Story Generation Metrics
Guan, J., Zhang, Z., Feng, Z., Liu, Z., Ding, W., Mao, X., Fan, C., & Huang, M. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.500 · Cited by 33 · see citing works
Self-citationAutomatic Item Generation via Frame Semantics: Natural Language Generation of Math Word Problems.
Deane, P., & Sheehan, K. M. (2003) · Cited by 31 · see citing works
“A theme-rewriting approach for generating algebra word problems,”
Koncel-Kedziorski, R., Konstas, I., Zettlemoyer, L., & Hajishirzi, H. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1168 · Cited by 30 · see citing works
Towards Generating Math Word Problems from Equations and Topics
Zhou, Q. S., & Huang, D. (2019) · International Conference on Natural Language Generation · doi:10.18653/v1/w19-8661 · Cited by 30 · see citing works
Generating Multi-hop Reasoning Questions to Improve Machine Reading Comprehension
Yu, J., Quan, X., Su, Q., et al. (2020) · doi:10.1145/3366423.3380114 · Cited by 24 · see citing works
Generating Mathematical Word Problems
Williams, S. (2011) · Open Research Online (The Open University) · Cited by 17 · see citing works
Personalized Multimodal Feedback Generation in Education
Liu, H., Liu, Z., Wu, Z., & Tang, J. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.166 · Cited by 15 · see citing works
Self-citationAddressing Semantic Drift in Question Generation for Semi-Supervised Question AnsweringPreprint
Zhang, S., & Bansal, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06356 · Cited by 14 · see citing works
Representation Learning from Limited Educational Data with Crowdsourced Labels
Wang, W., Xu, G., Ding, W., et al. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.3017122 · Cited by 14 · see citing works
Self-citationDimensionally guided synthesis of mathematical word problems
Wang, K., & Su, Z. (2016) · International Joint Conference on Artificial Intelligence · Cited by 12 · see citing works
Exploring the Needs and Interests of Fifth Graders for Personalized Math Word Problem Generation
Moon, D., Mack, N. A., & Gilbert, J. E. (2019) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3311927.3325309 · Cited by 10 · see citing works
Automatic Generation of Amharic Math Word Problem and Equation
Bekele, A. A. (2020) · Journal of Computer and Communications · doi:10.4236/jcc.2020.88006 · Cited by 9 · see citing works
Liyanage, V. P., & Ranathunga, S. (2020) · Language Resources and Evaluation · Cited by 8 · see citing works
Cetintas, S., Si, L., Xin, Y. P., Zhang, D., Park, J. Y., & Tzur, R. (2010) · Journal of Educational Data Mining · doi:10.5281/zenodo.3554741 · Cited by 6 · see citing works
Long Text Generation by Modeling Sentence-Level and Discourse-Level CoherencePreprint
Guan, J., Mao, X., Fan, C., Liu, Z., Ding, W., & Huang, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.08963 · Cited by 4 · see citing works
Self-citationOpenMEVA: A Benchmark for Evaluating Open-ended Story Generation MetricsPreprint
Guan, J., Zhang, Z., Feng, Z., Liu, Z., Ding, W., Mao, X., Fan, C., & Huang, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.08920 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- DeKita G Moon-Rembert and Juan E Gilbert. 2019. Illmatics: A web-based math word problem gener- ator for students' distal and proximal interests. In E-Learn: World Conference on E-Learning in Cor- porate, Government, Healthcare, and Higher Educa- tion, pages 842-848. Association for the Advance- ment of Computing in Education.
- Guowei Xu, Wenbiao Ding, Jiliang Tang, Songfan Yang, Gale Yan Huang, and Zitao Liu. 2019. Learning ef- fective embeddings from crowdsourced labels: An educational case study. In 2019 IEEE 35th Interna- tional Conference on Data Engineering, pages 1922- 1927. IEEE.
- Qingyu Zhou and Danqing Huang. 2019. Towards gen- erating math word problems from equations and top- ics. In Proceedings of the 12th International Confer- ence on Natural Language Generation, INLG 2019, Tokyo, Japan, October 29 -November 1, 2019, pages 494-503.
Cited by
Sorted by citations
Proactive and reactive engagement of artificial intelligence methods for education: a review
Mallik, S., & Gangopadhyay, A. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1151391 · Cited by 139 · see citing works
“Math word problem generation with mathematical consistency and problem context constraints,”
Wang, Z., Lan, A., & Baraniuk, R. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.484 · Cited by 39 · see citing works
- Authors
- Liu, T., Fang, Q., Ding, W., Li, H., Wu, Z., & Liu, Z.
- Venue
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
- Published
- 2021
- Pages
- 4225-4240
- OpenAlex
- W3092766609
- Cited by
- 29
- References
- 52
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 5 · 2023: 10 · 2022: 3 · 2021: 3
- FWCI
- 2.43
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Beijing Nova Program
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- commonsense knowledge graph, symbolic equations, knowledge fusion, topic relevance, language coherence, educational assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
