Reinforced Dynamic Reasoning for Conversational Question Generation
Open accessAbstract
This paper investigates a new task named Conversational Question Generation (CQG) which is to generate a question based on a passage and a conversation history (i.e., previous turns of question-answer pairs).CQG is a crucial task for developing intelligent agents that can drive question-answering style conversations or test user understanding of a given passage.Towards that end, we propose a new approach named Reinforced Dynamic Reasoning (ReDR) network, which is based on the general encoder-decoder framework but incorporates a reasoning procedure in a dynamic manner to better understand what has been asked and what to ask next about the passage.To encourage producing meaningful questions, we leverage a popular question answering (QA) model to provide feedback and fine-tune the question generator using a reinforcement learning mechanism.Empirical results on the recently released CoQA dataset demonstrate the effectiveness of our method in comparison with various baselines and model variants.Moreover, to show the applicability of our method, we also apply it to create multiturn question-answering conversations for passages in SQuAD. * Work done while visiting the Ohio State University.Shelly is in second grade.She is a new student at her school.Shelly's family has lived in many different places.Shelly was born in Florida.Her family moved to Tennessee when she was two years old.When she was four years old, they moved to Texas.They moved from there to Arizona, where they now live.Q1: What grade is Shelly in ?A1: second R1: Shelly is in second grade.Q2: Was she a
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The [95] model also tested on SQuAD for multi-turn question-answer conversations and outperformed several baselines on automatic and human evaluation metrics.
“The experimental analysis proves that the model gives better results than several compared baselines using automatic and human-based evaluation metrics.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [95]
An encoder-decoder architecture with dynamic reasoning via reinforcement learning in [95] generates the next question from previous questions on a CoQA passage.
“An encoder–decoder-based architecture employing a dynamic reasoning technique using reinforcement learning is explored in [95].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [95]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
(2014) · Cited by 33,419 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
(2004) · Cited by 9,718 · see citing works
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · see citing works
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 7,936 · see citing works
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · see citing works
Get To The Point: Summarization with Pointer-Generator Networks
See, A., Liu, P. J., & Manning, C. D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1099 · Cited by 3,775 · see citing works
Bidirectional Attention Flow for Machine ComprehensionPreprint
Seo, M., Kembhavi, A., Farhadi, A., & Hajishirzi, H. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1611.01603 · Cited by 2,322 · see citing works
A Diversity-Promoting Objective Function for Neural Conversation Models
Li, J., Galley, M., Brockett, C., Gao, J., & Dolan, B. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1014 · Cited by 1,981 · see citing works
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Serban, I. V., Sordoni, A., Bengio, Y., Courville, A. C., & Pineau, J. (2016) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v30i1.9883 · Cited by 1,639 · see citing works
Reading Wikipedia to answer open-domain questions
Chen, D., Fisch, A., Weston, J., & Bordes, A. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1171 · Cited by 1,546 · see citing works
Incorporating Copying Mechanism in Sequence-to-Sequence Learning
Gu, J., Lu, Z., Li, H., & Li, V. O. K. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1154 · Cited by 1,472 · see citing works
Self-citationVinyals, O., & Le, Q. (2015) · arXiv · doi:10.48550/arxiv.1506.05869 · Cited by 1,400 · see citing works
Deep Reinforcement Learning for Dialogue Generation
Li, J., Monroe, W., Ritter, A., Jurafsky, D., Galley, M., & Gao, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1127 · Cited by 1,221 · see citing works
CoQA: A Conversational Question Answering Challenge
Reddy, S., Chen, D., & Manning, C. D. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00266 · Cited by 975 · see citing works
Dynamic Coattention Networks For Question AnsweringPreprint
Xiong, C., Zhong, V., & Socher, R. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1611.01604 · Cited by 827 · see citing works
A Network-based End-to-End Trainable Task-oriented Dialogue System
Wen, T.-H., Vandyke, D., Mrkšić, N., Gašić, M., Rojas-Barahona, L. M., Su, P.-H., Ultes, S., & Young, S. J. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-1042 · Cited by 807 · see citing works
A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
Sordoni, A., Galley, M., Auli, M., Brockett, C., Ji, Y., Mitchell, M., Nie, J., Gao, J., & Dolan, B. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1020 · Cited by 768 · see citing works
QuAC: Question Answering in Context
Choi, E., He, H., Iyyer, M., Yatskar, M., Yih, W.-T., Choi, Y., Liang, P., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1241 · Cited by 714 · cited 5× · see citing works
“Recently released large-scale datasets (Reddy et al., 2018; Choi et al., 2018) enabled studying it in the textual setting where the information source used to answer questions is a given passage, and they inspired many significant work (Zhu et al., 2018; Huang et al., 2018; Yatskar, 2018).”
BackgroundMethodInfluentialLearning to Ask: Neural Question Generation for Reading Comprehension
Du, X., Shao, J., & Cardie, C. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1123 · Cited by 696 · cited 8× · see citing works
“NQG (Du et al., 2017) is a strong attention-based neural network approach for question generation task.”
BackgroundMethodInfluentialData-Driven Response Generation in Social Media
Ritter, A., Cherry, C. A., & Dolan, W. B. (2011) · Cited by 566 · see citing works
Good Question! Statistical Ranking for Question Generation
Heilman, M., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473390 · Cited by 366 · see citing works
Neural Question Generation from Text: A Preliminary Study
Zhou, Q. S., Yang, N., Wei, F., Tan, C., Bao, H., & Zhou, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-73618-1_56 · Cited by 353 · see citing works
Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus
Serban, I. V., García-Durán, A., Gulcehre, C., Ahn, S., Chandar, S., Courville, A., & Bengio, Y. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1056 · Cited by 280 · see citing works
Generating Natural Questions About an Image
Mostafazadeh, N., Misra, I., Devlin, J., Mitchell, M., He, X., & Vanderwende, L. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1170 · Cited by 270 · see citing works
Computer-aided generation of multiple-choice tests
Mitkov, R., & Ha, L. A. (2003) · doi:10.3115/1118894.1118897 · Cited by 258 · see citing works
Search-based Neural Structured Learning for Sequential Question Answering
Iyyer, M., Yih, W.-T., & Chang, M. (2017) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/p17-1167 · Cited by 219 · see citing works
A Systematic Comparison of Smoothing Techniques for Sentence-Level BLEU
Chen, B., & Cherry, C. (2014) · Proceedings of the Ninth Workshop on Statistical Machine Translation · doi:10.3115/v1/w14-3346 · Cited by 191 · see citing works
Saha, A., Pahuja, V., Khapra, M. M., Sankaranarayanan, K., & Chandar, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11332 · Cited by 179 · see citing works
Harvesting Paragraph-level Question-Answer Pairs from Wikipedia
Du, X., & Cardie, C. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1177 · Cited by 168 · see citing works
Leveraging Context Information for Natural Question Generation
Song, L., Wang, Z., Hamza, W., Zhang, Y., & Gildea, D. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-2090 · Cited by 148 · see citing works
SDNet: Contextualized Attention-based Deep Network for Conversational Question AnsweringPreprint
Zhu, C., Zeng, M., & Huang, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.03593 · Cited by 118 · see citing works
Neural Models for Key Phrase Extraction and Question Generation
Subramanian, S., Wang, T., Yuan, X., Zhang, S., Trischler, A., & Bengio, Y. (2018) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/w18-2609 · Cited by 97 · see citing works
The first question generation shared task evaluation challenge
Rus, V., Brendan Wyse, A., Piwek, P., Lintean, M. C., Stoyanchev, S., & Moldovan, C. (2010) · Open Research Online (The Open University) · Cited by 95 · see citing works
Neural Response Generation via GAN with an Approximate Embedding Layer
Xu, Z. X., Liu, B., Wang, B., Sun, C., Wang, X., Wang, Z., & Qi, C. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1065 · Cited by 92 · see citing works
FlowQA: Grasping Flow in History for Conversational Machine ComprehensionPreprint
Huang, H.-Y., Choi, E., & Yih, W.-T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.06683 · Cited by 63 · see citing works
QuAC : Question Answering in ContextPreprint
Choi, E., He, H., Iyyer, M., Yatskar, M., Yih, W.-T., Choi, Y., Liang, P., & Zettlemoyer, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07036 · Cited by 58 · see citing works
MEMEN: Multi-layer Embedding with Memory Networks for Machine ComprehensionPreprint
Pan, B., Li, H., Zhao, Z., Cao, B., Cai, D., & He, X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.09098 · Cited by 58 · see citing works
Self-citationA Qualitative Comparison of CoQA, SQuAD 2.0 and QuAC
Yatskar, M. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1241 · Cited by 56 · see citing works
Question Generation from SQL Queries Improves Neural Semantic Parsing
Guo, D., Sun, Y., Tang, D., Duan, N., Yin, J. P., Chi, H., Cao, J., Chen, P., & Zhou, M. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1188 · Cited by 56 · see citing works
Automatically Generating Questions from Queries for Community-based Question Answering
Zhao, S., Wang, H., Li, C., Liu, T., & Guan, Y. (2011) · Cited by 37 · see citing works
Neural Models for Key Phrase Detection and Question GenerationPreprint
Subramanian, S., Wang, T., Yuan, X., Zhang, S., Bengio, Y., & Trischler, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.04560 · Cited by 32 · see citing works
A Qualitative Comparison of CoQA, SQuAD 2.0 and QuACPreprint
Yatskar, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.10735 · Cited by 31 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, and Bill Dolan. 2018. Generating informative and diverse conversational responses via adversarial information maximization. In Advances in Neural Information Processing Sys- tems, pages 1815-1825.
Cited by
Sorted by citations
Mulla, N., & Gharpure, P. (2023) · Progress in Artificial Intelligence · doi:10.1007/s13748-023-00295-9 · Cited by 132 · see citing works
ChainCQG: Flow-Aware Conversational Question Generation
Gu, J., Mirshekari, M., Yu, Z., & Sisto, A. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.177 · Cited by 28 · see citing works
Qi, P., Zhang, Y., & Manning, C. D. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.3 · Cited by 23 · see citing works
- Authors
- Pan, B., Li, H., Yao, Z., Cai, D., & Sun, H.
- Affiliations
- Zhejiang University (CN)
- The Ohio State University (US)
- Alibaba Group (China) (CN)
- Corresponding author
- Cai, Deng
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 2114-2124
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2952939310
- Cited by
- 40
- References
- 45
- Citations by year
- 2025: 3 · 2024: 2 · 2023: 8 · 2022: 7 · 2021: 5 · 2020: 13 · 2019: 2
- FWCI
- 2.58
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1815674)
- National Natural Science Foundation of China (61751307)
- Army Research Office (W911NF-17, W911NF-17-1-0412, W911NF)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- reinforcement learning, question answering systems, conversation history, SQuAD, intelligent agents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
