Identifying Where to Focus in Reading Comprehension for Neural Question Generation
Open accessAbstract
A first step in the task of automatically generating questions for testing reading comprehension is to identify questionworthy sentences, i.e. sentences in a text passage that humans find it worthwhile to ask questions about.We propose a hierarchical neural sentence-level sequence tagging model for this task, which existing approaches to question generation have ignored.The approach is fully data-driven -with no sophisticated NLP pipelines or any hand-crafted rules/features -and compares favorably to a number of baselines when evaluated on the SQuAD data set.When incorporated into an existing neural question generation system, the resulting end-to-end system achieves stateof-the-art performance for paragraph-level question generation for reading comprehension.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In [12, 94], three crowd-workers rate questions on a 1-to-5 scale for fluency and relevance.
“An approach in which three crowd-workers are used for rating questions based on a scale of 1 to 5 (5 being good) on two parameters fluency and relevance is employed in [12, 94].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · see citing works
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831 · see citing works
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · see citing works
Natural Language Processing (almost) from ScratchPreprint
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 6,882 · see citing works
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878 · see citing works
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · see citing works
Meteor Universal: Language Specific Translation Evaluation for Any Target Language
Denkowski, M., & Lavie, A. (2014) · WMT@ACL · doi:10.3115/v1/w14-3348 · Cited by 2,108 · see citing works
A Fast and Accurate Dependency Parser using Neural Networks
Chen, D., & Manning, C. D. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1082 · Cited by 1,810 · see citing works
Microsoft COCO Captions: Data Collection and Evaluation Server
Chen, X., Fang, H., Lin, T.-Y., Vedantam, R., Gupta, S., Dollar, P., & Zitnick, C. L. (2015) · arXiv · doi:10.48550/arxiv.1504.00325 · Cited by 1,614 · see citing works
Learning classifiers from only positive and unlabeled data
Elkan, C., & Noto, K. (2008) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1401890.1401920 · Cited by 1,060 · see citing works
Neural Summarization by Extracting Sentences and Words
Cheng, J., & Lapata, M. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1046 · Cited by 802 · see citing works
Learning to Ask: Neural Question Generation for Reading Comprehension
Du, X., Shao, J., & Cardie, C. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1123 · Cited by 696 · cited 7× · see citing works
“Question Generation Similar to Du et al. (2017), we implement the sentence-level question generator with an attention-based sequence-to-sequence learning framework (Sutskever et al., 2014; Bahdanau et al., 2015), to map a sentence in the reading comprehension article to natural questions.”
BackgroundMethodResultInfluentialSelf-citationLearning Character-level Representations for Part-of-Speech Tagging
dos Santos, C., & Zadrozny, B. (2014) · Cited by 519 · see citing works
Good Question! Statistical Ranking for Question Generation
Heilman, M., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473390 · Cited by 366 · see citing works
Neural Question Generation from Text: A Preliminary Study
Zhou, Q. S., Yang, N., Wei, F., Tan, C., Bao, H., & Zhou, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-73618-1_56 · Cited by 353 · see citing works
Learning with Positive and Unlabeled Examples Using Weighted Logistic Regression
Lee, W. S., & Liu, B. (2003) · Cited by 339 · see citing works
Jointly Learning to Extract and Compress
Berg-Kirkpatrick, T., Gillick, D., & Klein, D. (2011) · Cited by 225 · see citing works
Topical Coherence for Graph-based Extractive Summarization
Parveen, D., Ramsl, H.-M., & Strube, M. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1226 · Cited by 139 · see citing works
Language independent extractive summarization
Mihalcea, R. F. (2005) · doi:10.3115/1225753.1225766 · Cited by 128 · see citing works
The first question generation shared task evaluation challenge
Rus, V., Brendan Wyse, A., Piwek, P., Lintean, M. C., Stoyanchev, S., & Moldovan, C. (2010) · Open Research Online (The Open University) · Cited by 95 · see citing works
Towards a Unified Approach to Simultaneous Single-Document and Multi-Document Summarizations
Wan, X. (2013) · Cited by 89 · see citing works
Question Generation from Paragraphs at UPenn: QGSTEC System Description
Mannem, P., Prasad, R., & Joshi, A. K. (2010) · Cited by 68 · see citing works
Neural Question Generation from Text: A Preliminary StudyPreprint
Zhou, Q. S., Yang, N., Wei, F., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.01792 · Cited by 35 · see citing works
Detecting (Un)Important Content for Single-Document News Summarization
Yang, Y., Bao, F. S., & Nenkova, A. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-2112 · Cited by 16 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Michael Denkowski and Alon Lavie. 2014. Me- teor universal: Language specific translation eval- uation for any target language. In Proceed- ings of the Ninth Workshop on Statistical Machine Translation. Association for Computational Lin- guistics, Baltimore, Maryland, USA, pages 376- 380. http://www.aclweb.org/anthology/W14-3348.
Cited by
Sorted by citations
Harvesting Paragraph-level Question-Answer Pairs from Wikipedia
Du, X., & Cardie, C. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1177 · Cited by 168 · see citing works
Self-citationMulla, N., & Gharpure, P. (2023) · Progress in Artificial Intelligence · doi:10.1007/s13748-023-00295-9 · Cited by 132 · see citing works
Towards a Better Metric for Evaluating Question Generation Systems
Nema, P., & Khapra, M. M. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1429 · Cited by 106 · see citing works
LearningQ: A Large-Scale Dataset for Educational Question Generation
Chen, G., Yang, J., Hauff, C., & Houben, G.-J. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.14987 · Cited by 83 · see citing works
Generating Question-Answer Hierarchies
Krishna, K., & Iyyer, M. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1224 · Cited by 44 · see citing works
A Semantic Role-based Approach to Open-Domain Automatic Question Generation
Flor, M., & Riordan, B. (2018) · Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w18-0530 · Cited by 39 · see citing works
ChainCQG: Flow-Aware Conversational Question Generation
Gu, J., Mirshekari, M., Yu, Z., & Sisto, A. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.177 · Cited by 28 · see citing works
- Authors
- Du, X., & Cardie, C.
- Affiliations
- Cornell University (US)
- Corresponding author
- Cardie, Claire
- ORCID
- Cardie, Claire
- Venue
- Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
- Published
- 2017
- Pages
- 2067-2073
- OpenAlex
- W2757715585
- Cited by
- 100
- References
- 28
- Citations by year
- 2026: 2 · 2025: 7 · 2024: 5 · 2023: 18 · 2022: 16 · 2021: 12 · 2020: 16 · 2019: 16 · 2018: 8
- FWCI
- 4.99
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- reading comprehension, SQuAD
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
