NewsQA: A Machine Comprehension Dataset
Open accessAbstract
We present NewsQA, a challenging machine comprehension dataset of over 100,000 human-generated question-answer pairs. Crowdworkers supply questions and answers based on a set of over 10,000 news articles from CNN, with answers consisting of spans of text in the articles. We collect this dataset through a four-stage process designed to solicit exploratory questions that require reasoning. Analysis confirms that NewsQA demands abilities beyond simple word matching and recognizing textual entailment. We measure human performance on the dataset and compare it to several strong neural models. The performance gap between humans and machines (13.3% F1) indicates that significant progress can be made on NewsQA through future research. The dataset is freely available online.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1× · see citing works
“Existing datasets vary in size, difficulty, and collection methodology; however, as pointed out by Rajpurkar et al. (2016), most suffer from one of two shortcomings: those that are designed explicitly to test comprehension (Richardson et al.”
BackgroundGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 2× · see citing works
“Word embeddings are initialized using GloVe vectors (Pennington et al., 2014) pre-trained on the 840-billion Common Crawl corpus.”
BackgroundMethodBLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 1× · see citing works
“BLEU is a precision-based metric popular in machine translation that uses a weighted average of variable length phrase matches ( n -grams) against the reference sentence (Papineni et al., 2002).”
MethodSQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 3× · see citing works
“Existing datasets vary in size, difficulty, and collection methodology; however, as pointed out by Rajpurkar et al. (2016), most suffer from one of two shortcomings: those that are designed explicitly to test comprehension (Richardson et al., 2013) are too small for training data-intensive deep…”
BackgroundMethodInfluentialCIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · cited 2× · see citing works
“ase matches (n-grams) against the reference sentence (Papineni et al., 2002). CIDEr was designed to correlate better with human judgements of sentence similarity, and uses tf-idf scores over n-grams (Vedantam et al., 2015). As given in Table 4, humans averaged 0.694 F1 on NewsQA. The human EM scores are relatively low…”
MethodInfluentialOn the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310 · see citing works
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,779 · see citing works
Liu, C., Lowe, R., Serban, I. V., Noseworthy, M., Charlin, L., & Pineau, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1230 · Cited by 1,262 · see citing works
Exact solutions to the nonlinear dynamics of learning in deep linear neural networksPreprint
Saxe, A., McClelland, J. L., & Ganguli, S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.6120 · Cited by 1,010 · see citing works
MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
Richardson, M., Burges, C., & Renshaw, E. (2013) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1020 · Cited by 778 · cited 5× · see citing works
“Like Richardson et al. (2013) we want to encourage reasoning in comprehension models.”
BackgroundInfluentialA Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
Chen, D., Bolton, J., & Manning, C. D. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1223 · Cited by 494 · cited 8× · see citing works
“ES The forms of reasoning required to solve NewsQA directly influence the abilities that models will learn from the dataset. We stratified reasoning types using a variation on the taxonomy presented by Chen et al. (2016) in their analysis of the CNN/Daily Mail dataset. Types are as follows, in ascending order of difficul…”
BackgroundMethodInfluentialThe Goldilocks Principle: Reading Children's Books with Explicit Memory RepresentationsPreprint
Hill, F., Bordes, A., Chopra, S., & Weston, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.02301 · Cited by 478 · cited 3× · see citing works
“…2013) are too small for training data-intensive deep learning models, while those that are sufficiently large for deep learning (Hermann et al., 2015; Hill et al., 2016; Bajgar et al., 2016) are generated synthetically, yielding questions that are not posed in natural language and that may not…”
BackgroundMethodInfluentialMachine Comprehension Using Match-LSTM and Answer PointerPreprint
Wang, S., & Jiang, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.07905 · Cited by 424 · cited 3× · see citing works
“The first neural model is the match-LSTM (mLSTM) system of Wang & Jiang (2016b).”
MethodLearning Natural Language Inference with LSTM
Wang, S., & Jiang, J. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1170 · Cited by 403 · cited 4× · see citing works
“The first neural model is the match-LSTM (mLSTM) system of Wang & Jiang (2016b).”
MethodInfluentialText Understanding with the Attention Sum Reader Network
Kadlec, R., Schmid, M., Bajgar, O., & Kleindienst, J. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1086 · Cited by 299 · cited 4× · see citing works
“As such, determining the correct answer relies mostly on recognizing textual entailment between the article and the question.”
BackgroundInfluentialMachine Comprehension with Syntax, Frames, and Semantics
Wang, H., Bansal, M., Gimpel, K., & McAllester, D. (2015) · doi:10.3115/v1/p15-2115 · Cited by 81 · cited 1× · see citing works
“Nevertheless, recent comprehension models have performed well on MCTest (Sachan et al., 2015; Wang et al., 2015), including a highly structured neural model (Trischler et al., 2016a).”
MethodNatural Language Comprehension with the EpiReader
Trischler, A., Ye, Z., Yuan, X., Bachman, P., Sordoni, A., & Suleman, K. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1013 · Cited by 69 · cited 3× · see citing works
“Nevertheless, recent comprehension models have performed well on MCTest (Sachan et al., 2015; Wang et al., 2015), including a highly structured neural model (Trischler et al., 2016a).”
BackgroundMethodSelf-citationFastQA: A Simple and Efficient Neural Architecture for Question Answering.Preprint
Weissenborn, D., Wiese, G. K., & Seiffe, L. (2017) · arXiv (Cornell University) · Cited by 59 · see citing works
Learning Answer-Entailing Structures for Machine Comprehension
Sachan, M., Dubey, K. A., Xing, E. P., & Richardson, M. (2015) · doi:10.3115/v1/p15-1024 · Cited by 55 · cited 3× · see citing works
“Nevertheless, recent comprehension models have performed well on MCTest (Sachan et al., 2015; Wang et al., 2015), including a highly structured neural model (Trischler et al.”
MethodEmbracing data abundance: BookTest Dataset for Reading ComprehensionPreprint
Bajgar, O., Kadlec, R., & Kleindienst, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.00956 · Cited by 52 · cited 4× · see citing works
“…for training data-intensive deep learning models, while those that are sufficiently large for deep learning (Hermann et al., 2015; Hill et al., 2016; Bajgar et al., 2016) are generated synthetically, yielding questions that are not posed in natural language and that may not test comprehension…”
BackgroundInfluentialIterative Alternating Neural Attention for Machine ReadingPreprint
Sordoni, A., Bachman, P., Trischler, A., & Bengio, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.02245 · Cited by 38 · cited 1× · see citing works
“However, Chen et al. (2016) demonstrated that the task requires only limited reasoning and, in fact, performance of the strongest models (Kadlec et al., 2016; Trischler et al., 2016b; Sordoni et al., 2016) nearly matches that of humans.”
BackgroundSelf-citationA Parallel-Hierarchical Model for Machine Comprehension on Sparse Data
Trischler, A., Ye, Z., Yuan, X., He, J., & Bachman, P. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1041 · Cited by 34 · cited 3× · see citing works
“Nevertheless, recent comprehension models have performed well on MCTest (Sachan et al., 2015; Wang et al., 2015), including a highly structured neural model (Trischler et al., 2016a).”
BackgroundMethodSelf-citation
Flagged first, then by citations
2 of the 24 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- NewsQA: A Machine Comprehension Dataset https://aclanthology.org/W17-2623/ (Trischler et al., RepL4NLP 2017)
- Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2017. NewsQA: A Machine Comprehension Dataset. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 191–200, Vancouver, Canada. Association for Computational Linguistics.
- ACL materials are Copyright © 1963–2026 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License.
Cited by
Sorted by citations
Know What You Don’t Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2124 · Cited by 2,240 · see citing works
“Machine reading comprehension has become a central task in natural language understanding, fueled by the creation of many large-scale datasets (Hermann et al., 2015; Hewlett et al., 2016; Rajpurkar et al., 2016; Nguyen et al., 2016; Trischler et al., 2017; Joshi et al., 2017).”
BackgroundInfluentialSpanBERT: Improving Pre-training by Representing and Predicting Spans
Joshi, M., Chen, D., Liu, Y., Weld, D. S., Zettlemoyer, L., & Levy, O. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00300 · Cited by 1,827 · see citing works
MS MARCO: A Human Generated MAchine Reading COmprehension DatasetPreprint
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T. G., et al. (2016) · arXiv (Cornell University) · doi:10.4230/lipics.giscience.2025.5 · Cited by 1,190 · see citing works
Self-citationCoQA: A Conversational Question Answering Challenge
Reddy, S., Chen, D., & Manning, C. D. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00266 · Cited by 975 · see citing works
Abioye, S. O., Oyedele, L. O., Akanbi, L., Ajayi, A., Davila Delgado, J. M., Bilal, M., Akinade, O. O., & Ahmed, A. (2021) · Journal of Building Engineering · doi:10.1016/j.jobe.2021.103299 · Cited by 1,015 · see citing works
RACE: Large-scale ReAding Comprehension Dataset From Examinations
Lai, G., Xie, Q., Liu, H., Yang, Y., & Hovy, E. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1082 · Cited by 1,044 · see citing works
“Articles of SQUAD, NEWSQA and MS MARCO come from Wikipedia, CNN news and the Bing search engine respectively.”
BackgroundMethodResultInfluentialQuAC: Question Answering in Context
Choi, E., He, H., Iyyer, M., Yatskar, M., Yih, W.-T., Choi, Y., Liang, P., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1241 · Cited by 714 · see citing works
“…2017) NarrativeQA (Kocisk´y et al., 2017) TriviaQA (Joshi et al., 2017) SQuAD 2.0 (Rajpurkar et al., 2018) MS Marco (Nguyen et al., 2016) NewsQA (Trischler et al., 2016) Table 1: Comparison of the QUAC dataset to other question answering datasets. tion”), which makes the dialogs more…”
BackgroundMethodInfluentialCan a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
Mihaylov, T., Clark, P., Khot, T., & Sabharwal, A. (2018) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1260 · Cited by 2,991 · see citing works
“…(Richardson et al., 2013), cloze-style (Hermann et al., 2015; Onishi et al., 2016; Hill et al., 2016), and span prediction (Rajpurkar et al., 2016; Trischler et al., 2017; Joshi et al., 2017) However, analysis (Chen et al., 2016; Sug-awara et al., 2017) of these datasets has shown that many of…”
BackgroundDocVQA: A Dataset for VQA on Document Images
Mathew, M., Karatzas, D., & Jawahar, C. V. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00225 · Cited by 570 · see citing works
“Examples of datsets for extractive QA include SQuAD 1.1 [30], NewsQA [35] and Natural Questions [25].”
MethodInfluentialUNIFIEDQA: Crossing Format Boundaries with a Single QA System
Khashabi, D., Min, S., Khot, T., Sabharwal, A., Tafjord, O., Clark, P., & Hajishirzi, H. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.171 · Cited by 514 · see citing works
A Robustly Optimized BERT Pre-training Approach with Post-training
Liu, Z., Lin, W., Shi, Y., & Zhao, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-84186-7_31 · Cited by 431 · see citing works
“As shown in Table 6, we present additional datasets for extractive question answering tasks, including RACE (Lai et al., 2017), NewsQA (Trischler et al., 2017), TrivaQA (Joshi et al., 2017), HotpotQA (Yang et al., 2018).”
MethodMLQA: Evaluating Cross-lingual Extractive Question Answering
Lewis, P., Oğuz, B., Rinott, R., Riedel, S., & Schwenk, H. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.653 · Cited by 424 · see citing works
Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences
Khashabi, D., Chaturvedi, S., Roth, M., Upadhyay, S., & Roth, D. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1023 · Cited by 372 · see citing works
“CNN/Daily Mail (Hermann et al., 2015), SQuAD (Rajpurkar et al., 2016) and NewsQA (Trischler et al., 2016) to name a few, are some of the datasets that were released recently with the goal of facilitating research in machine comprehension.”
Background(2020) · doi:10.18653/v1/2020.acl-main.450 · Cited by 356 · see citing works
Clark, J. H., Choi, E., Collins, M., Garrette, D., Kwiatkowski, T., Nikolaev, V., & Palomaki, J. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00317 · Cited by 432 · see citing works
“Question answering can also be interpreted as an exercise in verifying the knowledge of experts by finding the answer to trivia questions that are carefully crafted by someone who already knows the answer such that exactly one answer is correct such as TriviaQA and Quizbowl/Jeopoardy! questions (Ferrucci et al., 2010;…”
MethodLinkBERT: Pretraining Language Models with Document Links
Yasunaga, M., Leskovec, J., & Liang, P. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.551 · Cited by 374 · see citing works
Siriwardhana, S., Weerasekera, R., Wen, E., Kaluarachchi, T., Rana, R., & Nanayakkara, S. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00530 · Cited by 337 · see citing works
“Knowledge Base Generation: We extract 85,000 100-word passages as the knowledge base using 10,000 news articles from the NewsQA dataset (Trischler et al., 2016).”
MethodELI5: long form question answering
Fan, A., Jernite, Y., Perez, E., Grangier, D., Weston, J., & Auli, M. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1346 · Cited by 319 · see citing works
“Extractive QA Extractive question answering datasets such as TREC (Voorhees, 2003), SQuAD (Rajpurkar et al., 2016, 2018), NewsQA (Trischler et al., 2017), SearchQA (Dunn et al., 2017), and QuAC (Choi et al., 2018) constrain the answer to a word or short phrase from the input and evaluate using exact match or F1 with t…”
MethodSurvey on evaluation methods for dialogue systems
Deriu, J., Rodrigo, A., Otegi, A., Echegoyen, G., Rosset, S., Agirre, E., & Cieliebak, M. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09866-x · Cited by 333 · see citing works
BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis
Xu, H., Liu, B., Shu, L., & Yu, P. S. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1242 · Cited by 308 · see citing works
“…(Shao et al., 2018), QuAC (Choi et al., 2018), HotpotQA (Yang et al., 2018)) news and other articles (CNN/Daily Mail (Hermann et al., 2015), NewsQA (Trischler et al., 2016), RACE (Lai et al., 2017)), fic-tional stories (MCTest (Richardson et al., 2013), CBT (Hill et al., 2015), NarrativeQA…”
BackgroundGeva, M., Goldberg, Y., & Berant, J. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1107 · Cited by 284 · see citing works
“The prevalent method for creating new datasets is through crowdsourcing, where examples are generated by workers (Zaidan and Callison-Burch, 2011; Richardson et al., 2013; Bowman et al., 2015; Rajpurkar et al., 2016; Trischler et al., 2017).”
MethodCosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning
Huang, L., Bras, R. J. L., Bhagavatula, C., & Choi, Y. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1243 · Cited by 278 · see citing works
“There have been many exciting new datasets developed for reading comprehension, such as SQuAD (Rajpurkar et al., 2016), NEWSQA (Trischler et al., 2017), SearchQA (Dunn et al., 2017), Narra-tiveQA (Koˇcisk`y et al., 2018 (Hill et al., 2015), and MCScript (Ostermann et al., 2018).”
BackgroundHow Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks
Kaushik, D., & Lipton, Z. C. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1546 · Cited by 229 · see citing works
DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications
He, W. P., Liu, K., Liu, J., Lyu, Y., Zhao, S., Xiao, X., Liu, Y., Wang, Y., Wu, H. R., She, Q., et al. (2018) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/w18-2605 · Cited by 227 · see citing works
“A series of neural network models, such as Match-LSTM (Wang and Jiang, 2017), BiDAF (Seo et al., 2016), R-net (Wang et al., 2017), have been proposed, achieving promising results on a variety of MRC evaluation tasks.”
BackgroundInfluentialMRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension
Fisch, A., Talmor, A., Jia, R., Seo, M., Choi, E., & Chen, D. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5801 · Cited by 220 · see citing works
“NewsQA (Trischler et al., 2017) Two sets of crowdworkers ask and answer questions based on CNN news articles.”
BackgroundRetrospective Reader for Machine Reading Comprehension
Zhang, Z., Yang, J., & Zhao, H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i16.17705 · Cited by 200 · see citing works
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer
Vu, T., Lester, B., Constant, N., Al‐Rfou, R., & Cer, D. M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.346 · Cited by 204 · see citing works
Jiang, Z., Araki, J., Ding, H., & Neubig, G. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00407 · Cited by 177 · see citing works
“…(Hendrycks et al., 2020)) Extractive SQuAD 1.1 (wikipedia (Rajpurkar et al., 2016)), SQuAD 2 (Wikipedia (Rajpurkar et al., 2018)), NewsQA (news (Trischler et al., 2017)), Quoref (wikipedia (Dasigi et al., 2019)), ROPES (situation understanding (Lin et al., 2019)) Table 2: Datasets used in this…”
BackgroundInfluentialCoreferential Reasoning Learning for Language Representation
Ye, D., Lin, Y., Du, J., Liu, Z., Li, P., Sun, M., & Liu, Z. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.582 · Cited by 174 · see citing works
Coarse-to-Fine Question Answering for Long Documents
Choi, E., Hewlett, D., Uszkoreit, J., Polosukhin, I., Lacoste, A., & Berant, J. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1020 · Cited by 170 · see citing works
“Unlike other recently released large-scale datasets (Rajpurkar et al., 2016; Trischler et al., 2016a), WIKIREADING does not annotate answer spans, making sentence selection more challenging.”
BackgroundInfluentialMetaICL: Learning to Learn In Context
Min, S., Lewis, M., Zettlemoyer, L., & Hajishirzi, H. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.201 · Cited by 169 · see citing works
Transformer models used for text-based question answering systems
Nassiri, K., & Akhloufi, M. A. (2022) · Applied Intelligence · doi:10.1007/s10489-022-04052-8 · Cited by 177 · see citing works
Curriculum Learning for Natural Language Understanding
Xu, B., Zhang, L., Mao, Z., Wang, Q., Xie, H., & Zhang, Y. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.542 · Cited by 167 · see citing works
“NewsQA is collected from news articles of CNN with two sets of crowdworkers, the ”ques-tioners” is provided with the article’s headline only, and ”answerers” is supposed to find the answer in full article.”
MethodInfluentialBlockwise Self-Attention for Long Document Understanding
Qiu, J., Ma, H., Levy, O., Yih, W.-T., Wang, S., & Tang, J. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.232 · Cited by 179 · see citing works
Making Neural QA as Simple as Possible but not Simpler
Weissenborn, D., Wiese, G. K., & Seiffe, L. (2017) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k17-1028 · Cited by 164 · see citing works
“It shows that more than half or more than a third of all questions in SQuAD or NewsQA, respectively, are (partially) answerable by a very simple neural BoW baseline.”
BackgroundMethodInfluentialQA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension
Rogers, A., Gardner, M., & Augenstein, I. (2022) · ACM Computing Surveys · doi:10.1145/3560260 · Cited by 177 · see citing works
“Subsequently, NewsQA [287] also relied on CNN data; it is an extractive dataset with questions written by crowd workers.”
BackgroundMethodMachine Comprehension by Text-to-Text Neural Question Generation
Yuan, X., Wang, T., Gülçehre, Ç., Sordoni, A., Bachman, P., Zhang, S., Subramanian, S., & Trischler, A. (2017) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w17-2603 · Cited by 158 · see citing works
“These generally comprise (document, question, answer) triples (Hermann et al., 2015; Hill et al., 2015; Rajpurkar et al., 2016; Trischler et al., 2016; Nguyen et al., 2016), where the goal is to predict an answer, conditioned on a document and question.”
BackgroundSelf-citationNeural Machine Reading Comprehension: Methods and Trends
Liu, S., Zhang, X., Zhang, S., Wang, H., & Zhang, W. (2019) · Applied Sciences · doi:10.3390/app9183698 · Cited by 177 · see citing works
MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension
Talmor, A., & Berant, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1485 · Cited by 157 · see citing works
“(46.5 EM, 69.4 F1) (Trischler et al., 2017)), improving upon previous state-of-the-art by a large margin.”
BackgroundInfluentialOpen-Retrieval Conversational Question Answering
Qu, C., Yang, L., Chen, C., Qiu, M., Croft, W. B., & Iyyer, M. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401110 · Cited by 154 · see citing works
“As a result, many popular QA tasks and datasets either follow an answer selection setting [16, 47, 57] or a machine comprehension setting [23, 35, 36, 44].”
BackgroundNeural Arabic Question Answering
Mozannar, H., Maamary, E., Hajal, K. E., & Hajj, H. M. (2019) · Arabic Natural Language Processing Workshop (WANLP) · doi:10.18653/v1/w19-4612 · Cited by 158 · see citing works
“…understand the reasoning required to answer the questions, we manually labeled the questions according to the following reasoning categories as in (Trischler et al., 2017; Rajpurkar et al., 2016): • Word matching (synonyms) : question matches the same word pattern up to syn-onyms in the…”
MethodEfficient and Robust Question Answering from Minimal Context over Documents
Min, S., Zhong, V., Socher, R., & Xiong, C. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1160 · Cited by 147 · see citing works
“In addition, they cover different domains, including Wikipedia (Rajpurkar et al., 2016; Joshi et al., 2017), news (Hermann et al., 2015; Trischler et al., 2016), fictional stories (Richardson et al., 2013; Koˇcisk`y et al., 2017), and textbooks (Lai et al., 2017; Xie et al., 2017).”
BackgroundInfluentialRetrieving and Reading: A Comprehensive Survey on Open-domain Question AnsweringPreprint
Zhu, F., Lei, W., Wang, C., Zheng, J., Poria, S., & Chua, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.00774 · Cited by 237 · see citing works
“Online News News from CNN/Daily CNN/Daily Mail [18] News from CNN NewsQA [125]”
BackgroundUnified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks
Lu, J., Clark, C., Zellers, R., Mottaghi, R., & Kembhavi, A. (2022) · arXiv · doi:10.48550/arxiv.2206.08916 · Cited by 145 · see citing works
“…on several QA datasets including SQuAD 2.0 (Rajpurkar et al., 2016), other training datasets from the MRQA (Fisch et al., 2019) shared tasks (Trischler et al., 2017; Joshi et al., 2017; Dunn et al., 2017; Yang et al., 2018; Kwiatkowski et al., 2019), QA datasets from SuperGLUE (Wang et al.,…”
MethodInfluentialSemi-Supervised QA with Generative Domain-Adaptive Nets
Yang, Z., Hu, J., Salakhutdinov, R., & Cohen, W. W. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1096 · Cited by 142 · see citing works
“Although larger question answering datasets with hundreds of thousands of question-answer pairs have been collected, including SQuAD (Rajpurkar et al., 2016), MSMARCO (Nguyen et al., 2016), and NewsQA (Trischler et al., 2016a), the data collection process is expensive and time-consuming in practice.”
BackgroundJEC-QA: A Legal-Domain Question Answering Dataset
Zhong, H., Xiao, C., Tu, C., Zhang, T., Liu, Z., & Sun, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6519 · Cited by 152 · see citing works
A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers
Dasigi, P., Lo, K., Beltagy, I., Cohan, A., Smith, N. A., & Gardner, M. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.365 · Cited by 138 · see citing works
Improving Machine Reading Comprehension with General Reading Strategies
Sun, K., Yu, D., Yu, D., & Cardie, C. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1270 · Cited by 129 · see citing works
“…answering by randomly reordering words or shuffling sentences (Ding and Zhou, 2018; Li and Zhou, 2018) or generating questions through paraphrasing (Yang et al., 2017; Yuan et al., 2017), which require a large amount of training data or limited by the number of training instances (Yu et al., 2018).”
BackgroundMethodSelective Question Answering under Domain Shift
Kamath, A., Jia, R., & Liang, P. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.503 · Cited by 153 · see citing works
Mulla, N., & Gharpure, P. (2023) · Progress in Artificial Intelligence · doi:10.1007/s13748-023-00295-9 · Cited by 132 · see citing works
“For SQG, there are several benchmarking datasets in both open-domain and closed-domain QG. Based on the cognitive level of the question, we can choose shallow datasets like SQuAD, NewsQA, [46, 47] or deep cognitive level datasets like Learn-ingQ and NarrativeQA [106, 107].”
MethodConversational question answering: a survey
Zaib, M., Zhang, W. E., Sheng, Q. Z., Mahmood, A., & Zhang, Y. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01744-y · Cited by 134 · see citing works
“SQuAD [74], TriviaQA [34], and NewsQA [96] are some of the popular examples of extractive datasets.”
BackgroundQuestion Answering through Transfer Learning from Large Fine-grained Supervision Data
Min, S., Seo, M., & Hajishirzi, H. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-2081 · Cited by 128 · see citing works
“The recent advances in neural question answering lead to numerous datasets and successful models in these paradigms (Rajpurkar et al., 2016; Yang et al., 2015; Nguyen et al., 2016; Trischler et al., 2016).”
BackgroundRecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking Recipes
Yagcioglu, S., Erdem, A., Erdem, E., & İkizler-Cinbiş, N. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1166 · Cited by 122 · see citing works
“…e.g . cloze (fill-in-the-blank), span selection or multiple choice, and the text sources they use, such as news articles (Hermann et al., 2015; Trischler et al., 2017), fictional stories (Hill et al., 2016), Wikipedia articles (Koˇcisk´y et al., 2018; Hewlett et al., 2016; Rajpurkar et al.,…”
BackgroundMethodAutomatic question generation and answer assessment: a survey
Das, B., Majumder, M., Phadikar, S., & Sekh, A. A. (2021) · Research and Practice in Technology Enhanced Learning · doi:10.1186/s41039-021-00151-1 · Cited by 123 · see citing works
Large-scale Cloze Test Dataset Created by Teachers
Xie, Q., Lai, G., Dai, Z., & Hovy, E. H. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1257 · Cited by 119 · see citing works
“Recently proposed reading comprehension datasets are all labeled by humans to ensure a high quality (Rajpurkar et al., 2016; Joshi et al., 2017; Trischler et al., 2016; Nguyen et al., 2016).”
BackgroundFedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks
Lin, B. Y., He, C., Ze, Z., Wang, H., Hua, Y., Dupuy, C., Gupta, R., Soltanolkotabi, M., Ren, X., & Avestimehr, S. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.13 · Cited by 122 · see citing works
Transforming Question Answering Datasets Into Natural Language Inference DatasetsPreprint
Demszky, D., Guu, K., & Liang, P. (2018) · arXiv.org · doi:10.48550/arxiv.1809.02922 · Cited by 192 · see citing works
“Table 1 summarizes the properties of the five QA datasets we transform into NLI: MovieQA, NewsQA, QAMR and RACE and SQuAD.”
BackgroundSemEval-2018 Task 11: Machine Comprehension Using Commonsense Knowledge
Ostermann, S., Roth, M., Modi, A., Thater, S., & Pinkal, M. (2018) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s18-1119 · Cited by 114 · see citing works
“NewsQA (Trischler et al., 2017) contains newswire texts from CNN with crowdsourced questions and answers.”
BackgroundRethinking search: making domain experts out of dilettantes
Metzler, D., Tay, Y., Bahri, D., & Najork, M. (2021) · ACM SIGIR Forum · doi:10.1145/3476415.3476428 · Cited by 112 · see citing works
DCMN+: Dual Co-Matching Network for Multi-Choice Reading Comprehension
Zhang, S., Zhao, H. Y., Wu, Y., Zhang, Z., Zhou, X., & Zhou, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6502 · Cited by 132 · see citing works
The limitations of stylometry for detecting machine-generated fake news
Schuster, T., Schuster, R., Shah, D. J., & Barzilay, R. (2020) · Computational Linguistics · doi:10.1162/coli_a_00380 · Cited by 109 · see citing works
“We use the newsQA dataset (Trischler et al. 2017) that contains Cable News Network (CNN) articles with corresponding questions and answers (segments copied verbatim from the text).”
BackgroundMethodInfluentialTyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse LanguagesPreprint
Clark, J. H., Choi, E., Collins, M., Garrette, D., Kwiatkowski, T., Nikolaev, V., & Palomaki, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.05002 · Cited by 109 · see citing works
“Question answering can also be interpreted as an exercise in verifying the knowledge of experts by finding the answer to trivia questions that are carefully crafted by someone who already knows the answer such that exactly one answer is correct such as TriviaQA and Quizbowl/Jeopoardy! questions (Ferrucci et al., 2010;…”
MethodNeural Approaches to Conversational AI
Gao, J., Galley, M., & Li, L. (2019) · now publishers, Inc. eBooks · doi:10.1561/9781680835533 · Cited by 124 · see citing works
KLUE: Korean Language Understanding EvaluationPreprint
Park, S., Moon, J., Kim, S., Cho, W. I., Han, J., Park, J., Song, C., Kim, J., Song, Y., Oh, T., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.09680 · Cited by 153 · see citing works
“Several MRC datasets have incorporated unanswerable questions [143, 130, 96, 113, 69].”
BackgroundemrQA: A Large Corpus for Question Answering on Electronic Medical Records
Pampari, A., Raghavan, P., Liang, J., & Peng, J. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1258 · Cited by 191 · see citing works
“Recent advances in crowd-sourcing and search engines have resulted in an explosion of large-scale (100K) MC datasets for factoid QA, having ample redundant evidence in text (Rajpurkar et al., 2016; Trischler et al., 2016; Joshi et al., 2017; Dhingra et al., 2017).”
BackgroundWhat Makes Reading Comprehension Questions Easier?
Sugawara, S., Inui, K., Sekine, S., & Aizawa, A. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1453 · Cited by 105 · see citing works
“23.3 10.0 6.7 3.3 10.0 3.3 3.3 6.7 6.7 3.3 0.0 0.0 6.7 0.0 Ambiguous possible that unsolvable questions are permitted and that the system must indicate them in some datasets such as QA4MRE, NewsQA, MARCO, of these questions.”
MethodInfluentialCTRLsum: Towards Generic Controllable Text Summarization
He, J., Kryściński, W., McCann, B., Rajani, N. F., & Xiong, C. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.396 · Cited by 125 · see citing works
Learning to Generate Questions by LearningWhat not to Generate
Liu, B., Zhao, M. M., Niu, D., Lai, K., He, Y., Wei, H., & Xu, Y. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313737 · Cited by 104 · see citing works
“1 [29] and the NewsQA dataset [36].”
Trojaning Language Models for Fun and Profit
Zhang, X., Zhang, Z., Ji, S., & Wang, T. (2021) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp51992.2021.00022 · Cited by 103 · see citing works
Self-citationBeat the ai: Investigating adversarial human annotation for reading comprehension
Bartolo, M., Roberts, A., Welbl, J., Riedel, S., & Stenetorp, P. (2020) · UCL Discovery (University College London) · Cited by 102 · see citing works
Question Answering for Privacy Policies: Combining Computational and Legal Perspectives
Ravichander, A., Black, A. W., Wilson, S., Norton, T. B., & Sadeh, N. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1500 · Cited by 103 · see citing works
“Thus, setting 5 million installs as a threshold, we ensure each category includes applications with installs on both sides NarrativeQA (Koˇcisk`y et al., 2018) Fiction InsuranceQA (Feng et al., 2015) Insurance TriviaQA (Joshi et al., 2017) Wikipedia SQuAD 1.0 (Rajpurkar et al., 2016) Wikipedia SQuAD 2.0 (Rajpurkar et…”
InfluentialA Survey on Machine Reading Comprehension—Tasks, Evaluation Metrics and Benchmark Datasets
Zeng, C., Li, S., Li, Q., Hu, J., & Hu, J. (2020) · Applied Sciences · doi:10.3390/app10217640 · Cited by 109 · see citing works
Multi-Fact Correction in Abstractive Text Summarization
Dong, Y., Wang, S., Gan, Z., Cheng, Y., Cheung, J. C. K., & Liu, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.749 · Cited by 105 · see citing works
Exploring and Predicting Transferability across NLP Tasks
Vu, T., Wang, T., Munkhdalai, T., Sordoni, A., Trischler, A., Mattarella-Micke, A., Maji, S., & Iyyer, M. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.635 · Cited by 118 · see citing works
Self-citationNeural Models for Key Phrase Extraction and Question Generation
Subramanian, S., Wang, T., Yuan, X., Zhang, S., Trischler, A., & Bengio, Y. (2018) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/w18-2609 · Cited by 97 · see citing works
“In the first, lexical features (e.g., part-of-speech tags) are used to extract a key-phrase candidate list exhibiting certain types (Liu et al., 2011; Wang et al., 2016; Le et al., 2016; Yang et al., 2017).”
BackgroundMethodInfluentialSelf-citationCogLTX: Applying BERT to Long Texts
Ding, M., Zhou, C., Yang, H., & Tang, J. (2020) · neural information processing systems · Cited by 100 · see citing works
Entity-Based Knowledge Conflicts in Question Answering
Longpre, S., Perisetla, K., Chen, A., Ramesh, N., DuBois, C., & Singh, S. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.565 · Cited by 101 · see citing works
“For certain experiments we train with NewsQA (Trischler et al., 2017b), a news-oriented dataset with examples whose answers are prone to change over time (susceptible to knowledge conflicts).”
MethodMCScript: A Novel Dataset for Assessing Machine Comprehension Using Script Knowledge
Ostermann, S., Modi, A., Roth, M., Thater, S., & Pinkal, M. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2x36g84ob95p · Cited by 95 · see citing works
A Survey on Machine Reading Comprehension Systems
Baradaran, R., Ghiasi, R., & Amirkhani, H. (2022) · Natural Language Engineering · doi:10.1017/s1351324921000395 · Cited by 101 · see citing works
A Survey of Cutting-edge Multimodal Sentiment Analysis
Singh, U., Abhishek, K., & Azad, H. K. (2024) · ACM Computing Surveys · doi:10.1145/3652149 · Cited by 99 · see citing works
“…Modalities-Physiologicalsignals Self-reported DBNQADataset[89,300],SQuAD[252],CoQA[257],MSMARCO[225],HOTPOTQA[349],NEWSQ[318],QuAC[37],CliCR[306],RACE[348],Quasar[157],StoryClozeTest[305] Question-Answer Visual…”
Cosmos QA: Machine Reading Comprehension with Contextual Commonsense ReasoningPreprint
Huang, L., Bras, R. J. L., Bhagavatula, C., & Choi, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.00277 · Cited by 113 · see citing works
“There have been many exciting new datasets developed for reading comprehension, such as SQuAD (Rajpurkar et al., 2016), NEWSQA (Trischler et al., 2017), SearchQA (Dunn et al., 2017), Narra-tiveQA (Koˇcisk`y et al., 2018 (Hill et al., 2015), and MCScript (Ostermann et al., 2018).”
BackgroundQuestEval: Summarization Asks for Fact-based Evaluation
Scialom, T., Dray, P.-A., Lamprier, S., Piwowarski, B., Staiano, J., Wang, A., & Gallinari, P. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.529 · Cited by 175 · see citing works
“SQuAD-v2 provides unanswerable questions, while NewsQA is composed of news articles, corresponding to our summarization domain.”
MethodTay, Y., Wang, S., Luu, A. T., Fu, J., Phan, M. C., Yuan, X., Rao, J., Hui, S. C., & Zhang, A. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1486 · Cited by 90 · see citing works
“Teaching machines to read and comprehend is a fundamentally interesting and challenging problem in AI research (Hermann et al., 2015; Trischler et al., 2016; Rajpurkar et al., 2016).”
BackgroundSelf-citationA Review on Question Generation from Natural Language Text
Zhang, R., Guo, J., Chen, L., Fan, Y., & Cheng, X. (2021) · ACM Transactions on Information Systems · doi:10.1145/3468889 · Cited by 91 · see citing works
“NewsQA [216] is a large-scale MRC dataset including over 100K humangenerated QA pairs.”
BackgroundMethodSu, D., Xu, Y., Yu, T., Siddique, F. B., Barezi, E. J., & Fung, P. (2020) · Workshop on NLP for COVID-19 (NLP-COVID19) · doi:10.18653/v1/2020.nlpcovid19-2.14 · Cited by 91 · see citing works
“…ability to unseen data, the MRQA model is trained in a multi-task learning scheme on six datasets: SQuAD (Rajpurkar et al., 2016), NewsQA (Trischler et al., 2017), Triv-iaQA (Joshi et al., 2017), SearchQA (Dunn et al., 2017), HotpotQA (Yang et al., 2018b) and Nat-uralQuestions…”
MethodBenchmarking and Defending against Indirect Prompt Injection Attacks on Large Language Models
Yi, J., Xie, Y., Zhu, B., Kıcıman, E., Sun, G., Xie, X., & Wu, F. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3690624.3709179 · Cited by 175 · see citing works
“Since a significant portion of an LLM-integrated application’s development effort lies in designing clever prompts, prompt leakage attacks can lead to the devaluation of the application’s uniqueness and competitive advantage.”
Li, J., Liu, M., Kan, M., Zheng, Z., Wang, Z., Lei, W., Liu, T., & Qin, B. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.238 · Cited by 84 · see citing works
LearningQ: A Large-Scale Dataset for Educational Question Generation
Chen, G., Yang, J., Hauff, C., & Houben, G.-J. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.14987 · Cited by 83 · see citing works
“Several large-scale datasets have been collected to fuel the development of machine reading comprehension models, including SQuAD (Rajpurkar et al. 2016), RACE (Lai et al. 2017), NewsQA (Trischler et al. 2016), TriviaQA (Joshi et al. 2017), NarrativeQA (Koˇcisk`y et al. 2017), etc.”
Recent Advances in Neural Question GenerationPreprint
Pan, L., Lei, W., Chua, T., & Kan, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.08949 · Cited by 79 · see citing works
Supervised and Unsupervised Transfer Learning for Question Answering
Chung, Y.-A., Lee, H.-Y., & Glass, J. R. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1143 · Cited by 87 · see citing works
“A story is a sequence of sentences, and can be in the form of plain text (Trischler et al., 2017; Rajpurkar et al., 2016; Weston et al., 2016; Yang et al., 2015) or spoken content (Tseng et al.”
BackgroundImproving Factual Consistency of Abstractive Summarization via Question Answering
Nan, F., Nogueira dos Santos, C., Zhu, H., Ng, P., McKeown, K., Nallapati, R., Zhang, D., Wang, Z., Arnold, A. O., & Xiang, B. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.536 · Cited by 78 · see citing works
“Our QAGen model in Q UALS is also a BART-large model fine-tuned on the SQ UAD (Rajpurkar et al., 2016) and NewsQA (Trischler et al., 2017) datasets.”
MethodInvestigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension
Sun, K., Yu, D., Yu, D., & Cardie, C. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00305 · Cited by 83 · see citing works
“To increase task difficulty, researchers have explored a number of options including adding unanswerable (Trischler et al., 2017; Rajpurkar et al., 2018) or conversational (Choi et al., 2018; Reddy et al., 2019) questions that might require reasoning (Zhang et al., 2018a), and designing abstractive…”
BackgroundA Simple and Effective Model for Answering Multi-span Questions
Segal, E., Efrat, A., Shoham, M., Globerson, A., & Berant, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.248 · Cited by 76 · see citing works
Few-Shot Question Answering by Pretraining Span Selection
Ram, O., Kirstain, Y., Berant, J., Globerson, A., & Levy, O. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.239 · Cited by 79 · see citing works
“, 2016), NewsQA (Trischler et al., 2017), TriviaQA (Joshi et al.”
BackgroundMethodInfluentialASQA: Factoid Questions Meet Long-Form Answers
Stelmakh, I., Luan, Y., Dhingra, B., & Chang, M.-W. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.566 · Cited by 103 · see citing works
Do Large Language Models Know What They Don’t Know?
Yin, Z., Sun, Q., Guo, Q., Wu, J., Qiu, X., & Huang, X. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.551 · Cited by 75 · see citing works
The Effect of Natural Distribution Shift on Question Answering Models
Miller, J., Krauth, K., Recht, B., & Schmidt, L. (2020) · arXiv · doi:10.48550/arxiv.2004.14444 · Cited by 74 · see citing works
“1 and NewsQA (Trischler et al., 2017).”
BackgroundATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts
Asai, A., Salehi, M., Peters, M. E., & Hajishirzi, H. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.446 · Cited by 78 · see citing works
“2018), NewsQA (News; Trischler et al. 2017) and SearchQA (SQA; Dunn et al.”
BackgroundWhat to Pre-Train on? Efficient Intermediate Task Selection
Poth, C., Pfeiffer, J., Rücklé, A., & Gurevych, I. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.827 · Cited by 79 · see citing works
ScienceQA: a novel resource for question answering on scholarly articles
Saikh, T., Ghosal, T., Mittal, A., Ekbal, A., & Bhattacharyya, P. (2022) · International Journal on Digital Libraries · doi:10.1007/s00799-022-00329-y · Cited by 72 · see citing works
“There has been ongoing research on MRC on various textual documents like Wikipedia articles [37,59], news articles [7,47], etc.”
BackgroundInfluential
- Authors
- Trischler, A., Wang, T., Yuan, X., Harris, J., Sordoni, A., Bachman, P., & Suleman, K.
- Venue
- Proceedings of the 2nd Workshop on Representation Learning for NLP
- Published
- 2017
- Pages
- 191-200
- OpenAlex
- W2557764419
- Cited by
- 706
- References
- 24
- Citations by year
- 2026: 18 · 2025: 26 · 2024: 39 · 2023: 96 · 2022: 110 · 2021: 134 · 2020: 138 · 2019: 87 · 2018: 41 · 2017: 16 · 2016: 1
- FWCI
- 35.17
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Workshop on Representation Learning for NLP (RepL4NLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- machine reading comprehension, question answering, news articles, exploratory questions, reasoning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
