The DOI matches its Crossref record; 41 of the 42 references it cites were found in a registry or an index.
Learning to Ask: Neural Question Generation for Reading Comprehension
Open accessAbstract
We study automatic question generation for sentences from text passages in reading comprehension.We introduce an attention-based sequence learning model for the task and investigate the effect of encoding sentence-vs.paragraph-level information.In contrast to all previous work, our model does not rely on hand-crafted rules or a sophisticated NLP pipeline; it is instead trainable end-to-end via sequenceto-sequence learning.Automatic evaluation results show that our system significantly outperforms the state-of-the-art rule-based system.In human evaluations, questions generated by our system are also rated as being more natural (i.e., grammaticality, fluency) and as more difficult to answer (in terms of syntactic and lexical divergence from the original text and reasoning needed to answer).
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The authors performed both automatic evaluations using BLEU, METEOR, and ROUGE as well as human-based evaluation and found that their model outperformed the baseline model in [32] that they considered.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [32]
In the citing paper
“Other approaches make use of naturalness [32, 35] and difficulty [35].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [32]
In the citing paper
“However, the questions generated lack to a certain extent in terms of semantics and relevance as reported in most studies through human evaluation [32, 35, 36, 39, 56, 61].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [32]
In the citing paper
“While ITSs have traditionally been defined and constructed by hand, recent approaches have applied ML techniques such as multi-armed bandit techniques to adaptively personalize sequences of learning activities [135], LSTMs to generate questions to evaluate language comprehension [205], and RL to improve the strategies used within the ITS [365,441].”
From Tackling climate change with machine learning (2023) · cites this paper as [205]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“W s and W t are parameters to be learned. here, LSTM is the Long Short-Term Memory (LSTM) network (Hochreiter and Schmidhuber, 1997).”
BackgroundGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 2×
“300d pre-trained embeddings (Pennington et al., 2014) for initialization.”
MethodLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 2×
“We model the conditional probability us-ing RNN encoder-decoder architecture (Bahdanau et al., 2015; Cho et al., 2014), and adopt the global attention mechanism (Luong et al., 2015a) to make the model focus on certain elements of the input when generating each word during decoding.”
MethodBLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 2×
“The package includes BLEU 1, BLEU 2, BLEU 3, BLEU 4 (Papineni et al., 2002), METEOR (Denkowski and Lavie, 2014) and ROUGE L (Lin, 2004) evaluation scripts.”
MethodSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 4×
“More specifically, inspired by the recent success in neural machine translation (Sutskever et al., 2014; Bahdanau et al., 2015), summarization (Rush et al.”
BackgroundInfluentialNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 4×
“We model the conditional probability us-ing RNN encoder-decoder architecture (Bahdanau et al., 2015; Cho et al., 2014), and adopt the global attention mechanism (Luong et al., 2015a) to make the model focus on certain elements of the input when generating each word during decoding.”
BackgroundMethodInfluentialBinary Codes capable of currecting deletions, insertions, and reversals
Levenshtein, V. I. (1965) · Soviet physics. Doklady · Cited by 10,387
(2004) · Cited by 9,718
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · cited 3×
“We model the conditional probability us-ing RNN encoder-decoder architecture (Bahdanau et al., 2015; Cho et al., 2014), and adopt the global attention mechanism (Luong et al., 2015a) to make the model focus on certain elements of the input when generating each word during decoding.”
MethodShow, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · cited 2×
“, 2016), and image caption generation (Xu et al., 2015), we tackle question generation using a conditional neural language model with a global attention mechanism (Luong et al.”
Method
Research works printed but not linked: 2
- Thang Luong, Hieu Pham, and Christopher D. Manning. 2015a. Effective approaches to attentionbased neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, Lisbon, Portugal, pages 14121421. http://aclweb.org/anthology/D15-1166.
- Ruslan Mitkov and Le An Ha. 2003. Computeraided generation of multiple-choice tests. In Jill Burstein and Claudia Leacock, editors, Proceedings of the HLT-NAACL 03 Workshop on Building Educational Applications Using Natural Language Processing . pages 17-22. http://www.aclweb.org/anthology/W03-0203.pdf.
Cited by
Sorted by citations
Unified Language Model Pre-training for Natural Language Understanding and Generation
Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., Gao, J., Zhou, M., & Hon, H.-W. (2019) · arXiv · doi:10.48550/arxiv.1905.03197 · Cited by 1,225
GLM: General Language Model Pretraining with Autoregressive Blank Infilling
Du, Z., Qian, Y., Liu, X., Ding, M., Qiu, J., Yang, Z., & Tang, J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.26 · Cited by 1,068
“Considering the available baseline results, we use the Gigaword dataset (Rush et al., 2015) for abstractive summarization and the SQuAD 1.1 dataset (Rajpurkar et al., 2016) for question generation (Du et al., 2017) as the benchmarks for models pretrained on BookCorpus and Wikipedia.”
MethodInfluentialEvent Extraction by Answering (Almost) Natural Questions
Du, X., & Cardie, C. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.49 · Cited by 459
Self-citationProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training
Qi, W., Yan, Y., Gong, Y., Liu, D., Duan, N., Chen, J., Zhang, R., & Zhou, M. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.217 · Cited by 459
(2020) · doi:10.18653/v1/2020.acl-main.450 · Cited by 356
Advances and challenges in conversational recommender systems: A survey
Gao, C., Lei, W., He, X., de Rijke, M., & Chua, T. (2021) · AI Open · doi:10.1016/j.aiopen.2021.06.002 · Cited by 390
“Other metrics for measuring fluency [15, 123, 42], consistency [48, 86], readability [85], and informativeness [67] can also be used in evaluating the natural language output of CRS models.”
MethodParagraph-level Neural Question Generation with Maxout Pointer and Gated Self-attention Networks
Zhao, Y., Ni, X., Ding, Y., & Ke, Q. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1424 · Cited by 282
“(Du et al., 2017) pioneered the work of automatic QG using an end-to-end trainable sequence to sequence neural model.”
BackgroundMethodResultInfluentialQuestion Generation for Question Answering
Duan, N., Tang, D., Chen, P., & Zhou, M. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1090 · Cited by 280
“We first perform a vanilla sequence-to-sequence method (Du et al., 2017) using the original training sets of these three datasets, and show QG results in Table 3, where BLEU 4 score is used as the metric.”
MethodA Survey of Controllable Text Generation Using Transformer-based Pre-trained Language Models
Zhang, H., Song, H., Li, S., Zhou, M., & Song, D. (2024) · ACM Computing Surveys · doi:10.1145/3617680 · Cited by 317
“…it is essential to evaluate the quality of generated text in general aspects such as (1) fluency —how fluent the language in the output text is [12, 31], (2) factuality —to what extent the generated text reflects the facts described in the context [46, 146], (3) grammar : —how grammatically…”
BackgroundJust Ask: Learning to Answer Questions from Millions of Narrated Videos
Yang, A., Miech, A., Šivic, J., Laptev, I., & Schmid, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00171 · Cited by 232
Synthetic QA Corpora Generation with Roundtrip Consistency
Alberti, C., Andor, D., Pitler, E., Devlin, J., & Collins, M. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1620 · Cited by 221
“Question generation is a well-studied task in its own right (Heilman and Smith, 2010; Du et al., 2017; Du and Cardie, 2018).”
BackgroundTackling climate change with machine learning
Rolnick, D., Donti, P. L., Kaack, L. H., Kochanski, K., Lacoste, A., Sankaran, K., Ross, A. S., Milojevic-Dupont, N., Jaques, N., Waldman-Brown, A., et al. (2023) · ACM Computing Surveys · doi:10.1145/3485128 · Cited by 336
Answer-focused and Position-aware Neural Question Generation
Sun, X., Liu, J., Lyu, Y., He, W. P., Ma, Y., & Wang, S. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1427 · Cited by 186
“Recent neural question generation approaches (Du et al., 2017; Zhou et al., 2017) employ sequence-to-sequence model that takes an answer and its context as input and outputs a relevant question.”
BackgroundUnsupervised Commonsense Question Answering with Self-Talk
Shwartz, V., West, P., Bras, R. J. L., Bhagavatula, C., & Choi, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.373 · Cited by 215
Improving Neural Question Generation Using Answer Separation
Kim, Y., Lee, H., Shin, J., & Jung, K. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016602 · Cited by 179
Harvesting Paragraph-level Question-Answer Pairs from Wikipedia
Du, X., & Cardie, C. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1177 · Cited by 168
“Compared to models that only take into account sentence-level information (Heilman and Smith, 2010; Du et al., 2017; Zhou et al., 2017), we find that the linguistic knowledge introduced by the coreference representation aids question generation significantly, producing models that outperform the current state-of-thear…”
BackgroundMethodInfluentialSelf-citationZhang, Z., Xu, Y., Wang, Y., Yao, B., Ritchie, D., Wu, T., Yu, M., Wang, D., & Li, T. J.-J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517479 · Cited by 174
“, [14, 15, 65, 73, 77]) models that can generate “more accurate” questions and answers.”
BackgroundA Recurrent BERT-based Model for Question Generation
Chan, Y.-H., & Fan, Y.-C. (2019) · Proceedings of the 2nd Workshop on Machine Reading for Question Answering · doi:10.18653/v1/d19-5821 · Cited by 165
“(Du et al., 2017) pioneered the work of automatic QG tasks using an end-to-end trainable seq2seq neural model.”
BackgroundMethodInfluentialThe GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Gehrmann, S., Adewumi, T., Aggarwal, K., Ammanamanchi, P. S., Aremu, A., Bosselut, A., Chandu, K. R., Clinciu, M.-A., Das, D., Dhole, K., et al. (2021) · Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021) · doi:10.18653/v1/2021.gem-1.10 · Cited by 159
Question Answering and Question Generation as Dual TasksPreprint
Tang, D., Duan, N., Qin, T., Yan, Z., & Zhou, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02027 · Cited by 155
“Recent studies also investigate question generation for the reading comprehension task (Du, Shao, and Cardie 2017; Zhou et al. 2017).”
BackgroundLeveraging Context Information for Natural Question Generation
Song, L., Wang, Z., Hamza, W., Zhang, Y., & Gildea, D. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-2090 · Cited by 146
“In Table 2, we analyze the importance of each matching strategy by performing an ablation experiment on the devset according to the data split of Du et al. (2017).”
BackgroundMethodInfluentialRetrieving and Reading: A Comprehensive Survey on Open-domain Question AnsweringPreprint
Zhu, F., Lei, W., Wang, C., Zheng, J., Poria, S., & Chua, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.00774 · Cited by 343
“In the past few years, research on automatic question generation from text passages has received growing attention [150], [151], [152].”
BackgroundPAQ: 65 Million Probably-Asked Questions and What You Can Do With Them
Lewis, P., Wu, Y., Liu, L., Minervini, P., Küttler, H., Piktus, A., Stenetorp, P., & Riedel, S. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00415 · Cited by 171
“, 2019), generative modelling for contextual QA (Lewis and Fan, 2018), as well as being studied in its own right (Du et al., 2017; Hosking and Riedel, 2019).”
BackgroundUnsupervised Question Answering by Cloze Translation
Lewis, P., Denoyer, L., & Riedel, S. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1484 · Cited by 156
“Also, whilst supervised models exist for this task, they require the type of annotation unavailable in this setting (Du et al. 2017; Du and Cardie 2018; Hosking and Riedel 2019, inter alia ).”
BackgroundMethodSoft Layer-Specific Multi-Task Summarization with Entailment and Question Generation
Guo, H., Pasunuru, R., & Bansal, M. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1064 · Cited by 151
“Table 7 compares the performance of our model w.r.t. the state-of-the-art Du et al. (2017).”
BackgroundMethodInfluentialYe, H., Jiang, X., Luo, Z., & Chao, W. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1168 · Cited by 159
“We use a bidirectional LSTM (Hochreiter and Schmidhuber, 1997) as encoder and use another LSTM as decoder similar to Du et al. (2017). Decoder.”
MethodIf I Hear You Correctly: Building and Evaluating Interview Chatbots with Active Listening Skills
Xiao, Z., Zhou, M. X., Chen, W., Yang, H., & Chi, C. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376131 · Cited by 156
“Although there is research on question it is often done in a static context [1, 12] or without a conversation goal [56].”
BackgroundNatural Language Generation Using Deep Learning to Support MOOC Learners
Li, C., & Xing, W. (2021) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-020-00235-x · Cited by 133
“Researchers have used NLG for a series of tasks such as conversation building, question answering, and reading comprehension (Benamara and Saint-Dizier 2004; Dale 2016; Du et al. 2017).”
MethodA Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education
Doughty, J., Wan, Z., Bompelli, A., Qayum, J., Wang, T., Zhang, J., Zheng, Y., Doyle, A., Sridhar, P., Agarwal, A., et al. (2024) · Proceedings of the 26th Australasian Computing Education Conference · doi:10.1145/3636243.3636256 · Cited by 132
“Using the fine-tuned BERT classifier [24] (Section 4 has details), each LO was categorized into one of the six levels of the revised Bloom’s taxonomy [19].”
MethodProactive and reactive engagement of artificial intelligence methods for education: a review
Mallik, S., & Gangopadhyay, A. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1151391 · Cited by 139
“, 2016), question generation was later framed as a sequence-to-sequence learning problem that directly maps a sentence (or the entire passage containing the sentence) to a question (Du et al., 2017; Zhao et al., 2018; Kim et al., 2019), and can thus be trained in an end-to-end manner (Pan et al.”
BackgroundMulla, N., & Gharpure, P. (2023) · Progress in Artificial Intelligence · doi:10.1007/s13748-023-00295-9 · Cited by 132
“The authors have attempted to generate questions based on a paragraph of text for machine-reading comprehension in [32].”
BackgroundMethodInfluentialTraining Question Answering Models From Synthetic Data
Puri, R., Spring, R., Shoeybi, M., Patwary, M. A., & Catanzaro, B. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.468 · Cited by 161
Automatic question generation and answer assessment: a survey
Das, B., Majumder, M., Phadikar, S., & Sekh, A. A. (2021) · Research and Practice in Technology Enhanced Learning · doi:10.1186/s41039-021-00151-1 · Cited by 123
Biomedical Question Answering: A Survey of Approaches and Challenges
Jin, Q., Yuan, Z. P., Xiong, G., Yu, Q., Ying, H., Tan, C., Chen, M., Huang, S., Liu, X., & Yu, S. (2022) · ACM Computing Surveys · doi:10.1145/3490238 · Cited by 145
End-to-end generation of multiple-choice questions using text-to-text transfer transformer models
Rodriguez-Torrealba, R., Garcia-Lopez, E., & Garcia-Cabot, A. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118258 · Cited by 118
Generative Data Augmentation for Commonsense Reasoning
Yang, Y., Malaviya, C., Fernandez, J., Swayamdipta, S., Bras, R. J. L., Wang, J., Bhagavatula, C., Choi, Y., & Downey, D. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.90 · Cited by 156
“A candidate solution that has shown promise in other tasks, such as reading comprehension, is to augment a human-authored training set with a large set of synthetically-generated examples (Zhou et al., 2017; Du et al., 2017; Zhao et al., 2018a).”
BackgroundReinforcement Learning Based Graph-to-Sequence Model for Natural Question GenerationPreprint
Chen, Y., Wu, L., & Zaki, M. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.04942 · Cited by 135
“Existing NN based approaches (Du et al., 2017; Zhou et al., 2017; Song et al., 2018a; Kumar et al., 2018a) rely on the Seq2Seq model with attention, copy or coverage mechanisms.”
MethodTowards a Better Metric for Evaluating Question Generation Systems
Nema, P., & Khapra, M. M. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1429 · Cited by 106
“…problem and create even more training data, there is growing interest in developing techniques that can automatically generate questions from a given source, say a document (Du et al., 2017; Du and Cardie, 2017), knowledge base (Reddy et al., 2017; Serban et al., 2016), or image (Li et al., 2017) .”
BackgroundAddressing Semantic Drift in Question Generation for Semi-Supervised Question Answering
Zhang, S., & Bansal, M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1253 · Cited by 121
“, 2014) for QG tasks, taking answer sentence as input and outputting the question (Du et al., 2017; Zhou et al., 2017), which proved to be better than rulebased methods.”
BackgroundMethodInfluentialLearning to Generate Questions by LearningWhat not to Generate
Liu, B., Zhao, M. M., Niu, D., Lai, K., He, Y., Wei, H., & Xu, Y. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313737 · Cited by 105
“One of the key applications of question generation is to automatically produce question-answer pairs to enhance machine reading comprehension systems [8, 34, 35, 38].”
BackgroundMethodInfluentialIdentifying Where to Focus in Reading Comprehension for Neural Question Generation
Du, X., & Cardie, C. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1219 · Cited by 100
“Question Generation Similar to Du et al. (2017), we implement the sentence-level question generator with an attention-based sequence-to-sequence learning framework (Sutskever et al., 2014; Bahdanau et al., 2015), to map a sentence in the reading comprehension article to natural questions.”
BackgroundMethodResultInfluentialSelf-citationNeural Models for Key Phrase Extraction and Question Generation
Subramanian, S., Wang, T., Yuan, X., Zhang, S., Trischler, A., & Bengio, Y. (2018) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/w18-2609 · Cited by 100
“For the latter, Du et al. (2017) used a sequence-to-sequence model with an attention mechanism derived from the encoder states.”
MethodOne Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrases
Yuan, X., Wang, T., Meng, R., Thaker, K. M., Brusilovsky, P. L., He, D., & Trischler, A. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.710 · Cited by 89
“Paulus et al. (2017); Zhou et al. (2017) applied Seq2Seq models on summary generation tasks, while Du et al. (2017); Yuan et al. (2017) generated questions conditioned on documents and answers from machine comprehension datasets.”
MethodA Review on Question Generation from Natural Language Text
Zhang, R., Guo, J., Chen, L., Fan, Y., & Cheng, X. (2021) · ACM Transactions on Information Systems · doi:10.1145/3468889 · Cited by 102
“For example, Du et al. [57] presented a fully data-driven neural networks approach to automatically generate questions for MRC. Gao et al. [70] presented a novel setting, i.e., difficulty controllable QG for RC, where the generated RC questions should satisfy the specified difficulty as much as…”
BackgroundMethodInfluentialA Joint Model for Question Answering and Question GenerationPreprint
Wang, T., Yuan, X., & Trischler, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.01450 · Cited by 103
“Alongside QA, question generation has also gained increased popularity (Du et al., 2017; Yuan et al., 2017).”
BackgroundMethodToward Subgraph-Guided Knowledge Graph Question Generation With Graph Neural Networks
Chen, Y., Wu, L., & Zaki, M. J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3264519 · Cited by 106
“The results of L2A reported here are taken from [23].”
BackgroundMethodInfluentialZero-Shot Question Generation from Knowledge Graphs for Unseen Predicates and Entity Types
Elsahar, H., Gravier, C., & Laforest, F. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1020 · Cited by 93
“Information retrieval has been used before as base-line for QG from text input (Rush et al., 2015; Du et al., 2017).”
MethodInformation Maximizing Visual Question Generation
Krishna, R., Bernstein, M. S., & Fei-Fei, L. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00211 · Cited by 91
“In NLP, a few methods have attempted to automatically generate questions from knowledge bases using rule based [50] or deep learning based systems [12].”
Towards Human-Like Educational Question Generation with Large Language Models
Wang, Z., Valdez, J., Mallick, D. B., & Baraniuk, R. G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-11644-5_13 · Cited by 98
LearningQ: A Large-Scale Dataset for Educational Question Generation
Chen, G., Yang, J., Hauff, C., & Houben, G.-J. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.14987 · Cited by 83
“In accordance with the original work (Sutskever, Vinyals, and Le 2014; Du, Shao, and Cardie 2017), Bi-LSTM is used for the encoder and LSTM for the decoder.”
MethodResultInfluentialSemantic Graphs for Generating Deep Questions
Pan, L., Xie, Y., Feng, Y., Chua, T., & Kan, M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.135 · Cited by 92
Recent Advances in Neural Question GenerationPreprint
Pan, L., Lei, W., Chua, T., & Kan, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.08949 · Cited by 124
“To address this, they propose to replace the answer into a special token for passage encoding, and a separate RNN is used to encode the answer.”
BackgroundMethodInfluentialSelf-Training for Jointly Learning to Ask and Answer Questions
Sachan, M., & Xing, E. P. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1058 · Cited by 72
“However, recent papers build on neural network approaches such as seq2seq (Du et al., 2017; Tang et al., 2017; Zhou et al., 2017), CNNs and RNNs (Duan et al., 2017) for QG.”
BackgroundMethodInfluentialLearning to Ask Questions in Open-domain Conversational Systems with Typed Decoders
Wang, Y., Liu, C., Huang, M., & Nie, L. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1204 · Cited by 72
“Deciding what to ask and how is an indicator of machine understanding (Mostafazadeh et al., 2016), as demonstrated in machine comprehension (Du et al., 2017; Zhou et al., 2017b; Yuan et al., 2017) and question answering (Tang et al., 2017; Wang et al., 2017).”
BackgroundInfluentialSelf-Attention Architectures for Answer-Agnostic Neural Question Generation
Scialom, T., Piwowarski, B., & Staiano, J. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1604 · Cited by 70
“Three professional English speakers were asked to evaluate, the questions generated by: a) Transformer base , b) our best performing model, and c) the state-of-the-art model by Du et al. (2017) 6 .”
BackgroundMethodInfluentialWang, Z., Lan, A., Nie, W., Waters, A. E., Grimaldi, P. J., & Baraniuk, R. G. (2018) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3231644.3231654 · Cited by 77
“…question neural network-based question generation model using the attention mechanism, LSTM + attention + linguistic features , the model in [7] augmented with the same linguistic features that we use in our model, and QG-Net without features , QG-Net with the linguistic features removed.”
BackgroundMethodInfluentialGraph Neural Networks for Natural Language Processing: A SurveyPreprint
Wu, L., Chen, Y., Shen, K., Guo, X., Gao, H., Li, S., Pei, J., & Long, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.06090 · Cited by 70
Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation
Bartolo, M., Thrush, T., Jia, R., Riedel, S., Stenetorp, P., & Kiela, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.696 · Cited by 94
“Many approaches have been proposed to generate question-answer pairs given a passage (Du et al., 2017; Du and Cardie, 2018; Zhao et al., 2018; Lewis and Fan, 2019; Alberti et al., 2019; Puri et al., 2020; Lewis et al., 2021b).”
BackgroundCan a Student Large Language Model Perform as Well as Its Teacher?
Gholami, S., & Omar, M. (2024) · Advances in medical technologies and clinical practice book series · doi:10.4018/979-8-3693-1906-2.ch007 · Cited by 65
Generative Question Answering: Learning to Answer the Whole Question.
Lewis, M., & Fan, A. (2018) · International Conference on Learning Representations · Cited by 60
A Dataset and Baselines for Visual Question Answering on Art
García, N., Ye, C., Liu, Z., Hu, Q., Otani, M., Chu, C., Nakashima, Y., & Mitamura, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-66096-3_8 · Cited by 68
Recent progress in leveraging deep learning methods for question answering
Hao, T., Li, X., He, Y., Wang, F. L., & Qu, Y. (2022) · Neural Computing and Applications · doi:10.1007/s00521-021-06748-3 · Cited by 66
Difficulty Controllable Generation of Reading Comprehension Questions
Gao, Y., Bing, L., Chen, W., Lyu, M. R.-T., & King, I. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/690 · Cited by 83
“We employ BLEU (B), METEOR (MET) and ROUGE-L (R-L) scores by following [Du et al. , 2017].”
BackgroundMethodInfluentialDifficulty-Controllable Multi-hop Question Generation from Knowledge Graphs
Kumar, V., Hua, Y., Ramakrishnan, G., Qi, G., Gao, L., & Li, Y.-F. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-30793-6_22 · Cited by 66
“Instead, we use a state-of-the-art natural language QG model, Learning to Ask [9] (referred to as L2A hereinafter), as a baseline model for comparison.”
MethodInfluentialLee, D. B., Lee, S., Jeong, W. T., Kim, D., & Hwang, S. J. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.20 · Cited by 69
Semantic Answer Similarity for Evaluating Question Answering Models
Risch, J., Möller, T., Gutsch, J., & Pietsch, M. (2021) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/2021.mrqa-1.15 · Cited by 66
End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems
Shakeri, S., dos Santos, C. N., Zhu, H., Ng, P., Nan, F., Wang, Z., Nallapati, R. M., & Xiang, B. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.439 · Cited by 88
Scalable Educational Question Generation with Pre-trained Language Models
Bulathwela, S., Muse, H., & Yılmaz, E. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-36272-9_27 · Cited by 80
Cross-Lingual Training for Automatic Question Generation
Kumar, V., Joshi, N., Mukherjee, A., Ramakrishnan, G., & Jyothi, P. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1481 · Cited by 61
“Du et al. (2017) propose a sequence to sequence model for automatic question generation from English text.”
BackgroundMethodVariational Attention for Sequence-to-Sequence ModelsPreprint
Bahuleyan, H., Mou, L., Vechtomova, O., & Poupart, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.08207 · Cited by 81
“Their resulting models are denoted as VED+VAttn-0 and VED+VAttn-¯ h .”
BackgroundMethodInfluentialLet’s Ask Again: Refine Network for Automatic Question Generation
Nema, P., Mohankumar, A. K., Khapra, M. M., Srinivasan, B. V., & Ravindran, B. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1326 · Cited by 60
“Current models for AQG are based on the encode-attend-decode paradigm and they either generate questions from the passage alone (Du and Cardie, 2017; Du et al., 2017; Yao et al., 2018) or from the passage and a given answer (in which case the generated question must result in the given answer).”
MethodImproving Question Generation with Sentence-Level Semantic Matching and Answer Position Inferring
Ma, X., Zhu, Q., Zhou, Y., & Li, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6366 · Cited by 63
Capturing Greater Context for Question Generation
Tuan, L. A., Shah, D., & Barzilay, R. (2020) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v34i05.6440 · Cited by 59
On the Importance of Diversity in Question Generation for QA
Sultan, M. A., Chandel, S., Astudillo, R. F., & Castelli, V. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.500 · Cited by 62
“For fine-tuning we use the train set of the SQ u AD 1 split by Du et al. (2017).”
MethodAnalysis of LLMs for educational question classification and generation
Faraby, S. A., Romadhony, A., & Adiwijaya, A. (2024) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2024.100298 · Cited by 79
Question Generation from SQL Queries Improves Neural Semantic Parsing
Guo, D., Sun, Y., Tang, D., Duan, N., Yin, J. P., Chi, H., Cao, J., Chen, P., & Zhou, M. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1188 · Cited by 61
“Recent studies in machine reading comprehension generate questions from an answer span and its context from the document (Du et al., 2017; Golub et al., 2017).”
BackgroundAdversarial Domain Adaptation for Machine Reading Comprehension
Wang, H., Gan, Z., Liu, X., Liu, J., Gao, J., & Wang, H. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1254 · Cited by 65
“We plan to study if recent question generation methods (Du et al., 2017; Duan et al., 2017; Sun et al., 2018; Benmalek et al., 2019) could further help to close the performance gap in future work.”
BackgroundImproving Question Generation With to the Point Context
Li, J., Gao, Y., Bing, L., King, I., & Lyu, M. R.-T. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1317 · Cited by 51
“We evaluate with all commonly-used metrics in question generation (Du et al., 2017): BLEU-”
BackgroundMethodInfluentialLearning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT WorldsPreprint
Klein, T., & Nabi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.02365 · Cited by 68
“Although, question generation (QG) (Du et al., 2017; Serban et al., 2016; Pan et al., 2019; Kim et al., 2019) enjoys a bit of niche existence, it has a plethora of potential applications such as improving the training of QA systems, and help in the creation of material in the educational domain…”
BackgroundAutomating Reading Comprehension by Generating Question and Answer Pairs
Kumar, V., Boorla, K., Meena, Y. K., Ramakrishnan, G., & Li, Y.-F. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93040-4_27 · Cited by 49
Li, T. J.-J., & Riva, O. (2018) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3210240.3210339 · Cited by 53
“Due to the scarcity of domain-specific conversation datasets, it also cannot rely on a pure machine-learned approach [24, 69, 84, 86, 87].”
BackgroundTeaching Machines to Ask Questions
Yao, K., Zhang, L., Luo, T., Tao, L., & Wu, Y. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/632 · Cited by 44
“More recently, neural networks based question generation models aim at training an end-to-end system to generate natural language questions from text without human-designed transformation and generation rules [Zhou et al. , 2017; Yuan et al. , 2017; Du et al. , 2017].”
BackgroundSyn-QG: Syntactic and Shallow Semantic Rules for Question Generation
Dhole, K., & Manning, C. D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.69 · Cited by 48
Towards Visual-Prompt Temporal Answer Grounding in Instructional Video
Shutao Li, A., Li, B., Sun, B., & Weng, Y. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3411045 · Cited by 55
“RaNet[53] represent the relation-aware network, which formulates temporal language grounding in the video inspired by reading comprehension [54].”
EQG-RACE: Examination-Type Question Generation
Jia, X., Zhou, W., Sun, X., & Wu, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17553 · Cited by 48
A Unified Query-based Generative Model for Question Generation and Question AnsweringPreprint
Song, L., Wang, Z., & Hamza, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.01058 · Cited by 55
“Here, we follow Du, Shao, and Cardie (2017) and Zhou et al. (2017) to conduct experiments on the accessible part as our entire dataset.”
MethodResultInfluentialGenerating Question-Answer Hierarchies
Krishna, K., & Iyyer, M. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1224 · Cited by 44
“The proliferation of reading comprehension datasets like SQuAD (Rajpurkar et al., 2016, 2018) has enabled state-of-the-art neural question generation systems (Du et al., 2017; Kim et al., 2018).”
BackgroundMethodInfluentialLearning to Ask Unanswerable Questions for Machine Reading Comprehension
Zhu, H., Dong, L., Wei, F., Wang, W., Qin, B., & Liu, T. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1415 · Cited by 48
“Du et al. (2017) show that neural models based on the encoder-decoder framework can generate significantly better questions than rule-based systems (Heilman and Smith, 2010).”
BackgroundDeep Learning Based Question Generation Using T5 Transformer
Grover, K., Kaur, K., Tiwari, K., Rupali, & Kumar, P. (2021) · Communications in computer and information science · doi:10.1007/978-981-16-0401-0_18 · Cited by 50
Learning to Collaborate for Question Answering and Asking
Tang, D., Duan, N., Yan, Z., Zhang, Z., Sun, Y., Liu, S., Lv, Y., & Zhou, M. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1141 · Cited by 45
“On one hand, the Seq2Seq based QG model is trained by maximizing the literal similarity between the generated sentence and the ground truth sentence with maximum-likelihood estimation objective function (Du et al., 2017).”
MethodUnsupervised Multi-hop Question Answering by Question Generation
Pan, L., Chen, W., Xiong, W., Kan, M., & Wang, W. Y. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.469 · Cited by 55
Machine Reading Comprehension: The Role of Contextualized Language Models and BeyondPreprint
Zhang, Z., Zhao, H., & Wang, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.06249 · Cited by 63
Towards Generalized Methods for Automatic Question Generation in Educational Domains
Nguyen, H. A., Bhat, S., Moore, S., Bier, N., & Stamper, J. C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16290-9_20 · Cited by 47
“Forhumanevaluation,arecentmetaanalysisfound thatoverhalfofthereviewedresearchinvolvedcriteriarelatedtogrammar,fluency,topic relevance,andnaturalness[9,13,47],mostfrequentlyonanumericalscale[4].”
Automated Question and Answer Generation from Texts using Text-to-Text Transformers
Goyal, R., Kumar, P., & Singh, V. P. (2023) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-023-07840-7 · Cited by 46
“Most existing studies [19, 22] use an encoder–decoder framework with an attention mechanism [23, 24].”
A Multi-Agent Communication Framework for Question-Worthy Phrase Extraction and Question Generation
Wang, S., Wei, Z., Fan, Z., Liu, Y., & Huang, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017168 · Cited by 42
“…by the success of the encoder-decoder structure in machine translation (Bah-danau, Cho, and Bengio 2015) and image captioning (Xu et al. 2015), Du et al. (2017) propose to apply sequence-to-sequence model with attention mechanism for question generation without human designed rules, which is…”
BackgroundMethodIntegrating External Event Knowledge for Script Learning
Lv, S., Zhu, F., & Hu, S. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.27 · Cited by 44
Quiz-Style Question Generation for News Stories
Lelkes, A. D., Tran, V. Q., & Yu, C. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449892 · Cited by 49
“While early work utilized rule-based transformations and an over-generate and rank approach [16], recent mainstream works rely on neural-network based sequence-to-sequence methods, in particular encoder-decoder models with attention [12, 51].”
The Turking Test: Can Language Models Understand Instructions?Preprint
Efrat, A., & Levy, O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11982 · Cited by 109
“While there is prior work on learning to ask reading comprehension questions from thousands of examples (Du et al., 2017), our goal here is to test how well a model performs when given the same information as human annotators.”
BackgroundReinforced Dynamic Reasoning for Conversational Question Generation
Pan, B., Li, H., Yao, Z., Cai, D., & Sun, H. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1203 · Cited by 40
“NQG (Du et al., 2017) is a strong attention-based neural network approach for question generation task.”
BackgroundMethodInfluentialSimplifying Paragraph-Level Question Generation via Transformer Language Models
Lopez, L. E., Cruz, D. K., Cruz, J. C. B., & Cheng, C. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-89363-7_25 · Cited by 41
“The most prevalent technique for question generation studies is the usage of a sequence-to-sequence (Seq2Seq) model [4,3,20,2] in addition to a variety of other features and mechanisms.”
BackgroundMethodInfluential
Metadata
- Authors
- Du, X., Shao, J., & Cardie, C.
- Affiliations
- Cornell University (US)
- Shanghai Jiao Tong University (CN)
- Corresponding author
- Cardie, Claire
- Venue
- Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2017
- Pages
- 1342-1352
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2962717047
- Cited by
- 696
- References
- 42
- Citations by year
- 2026: 25 · 2025: 36 · 2024: 54 · 2023: 95 · 2022: 101 · 2021: 120 · 2020: 111 · 2019: 90 · 2018: 56 · 2017: 8
- FWCI
- 33.06
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- automatic question generation, reading comprehension, end-to-end learning, question difficulty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
