The DOI matches its Crossref record; 22 of the 24 references it cites were found in a registry or an index.
NewsQA: A Machine Comprehension Dataset
Open accessAbstract
We present NewsQA, a challenging machine comprehension dataset of over 100,000 human-generated question-answer pairs. Crowdworkers supply questions and answers based on a set of over 10,000 news articles from CNN, with answers consisting of spans of text in the articles. We collect this dataset through a four-stage process designed to solicit exploratory questions that require reasoning. Analysis confirms that NewsQA demands abilities beyond simple word matching and recognizing textual entailment. We measure human performance on the dataset and compare it to several strong neural models. The performance gap between humans and machines (13.3% F1) indicates that significant progress can be made on NewsQA through future research. The dataset is freely available online.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“Existing datasets vary in size, difficulty, and collection methodology; however, as pointed out by Rajpurkar et al. (2016), most suffer from one of two shortcomings: those that are designed explicitly to test comprehension (Richardson et al.”
BackgroundGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 2×
“Word embeddings are initialized using GloVe vectors (Pennington et al., 2014) pre-trained on the 840-billion Common Crawl corpus.”
BackgroundMethodBLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 1×
“BLEU is a precision-based metric popular in machine translation that uses a weighted average of variable length phrase matches ( n -grams) against the reference sentence (Papineni et al., 2002).”
MethodSQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 3×
“Existing datasets vary in size, difficulty, and collection methodology; however, as pointed out by Rajpurkar et al. (2016), most suffer from one of two shortcomings: those that are designed explicitly to test comprehension (Richardson et al., 2013) are too small for training data-intensive deep…”
BackgroundMethodInfluentialCIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · cited 2×
“ase matches (n-grams) against the reference sentence (Papineni et al., 2002). CIDEr was designed to correlate better with human judgements of sentence similarity, and uses tf-idf scores over n-grams (Vedantam et al., 2015). As given in Table 4, humans averaged 0.694 F1 on NewsQA. The human EM scores are relatively low…”
MethodInfluentialOn the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,855
Liu, C., Lowe, R., Serban, I. V., Noseworthy, M., Charlin, L., & Pineau, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1230 · Cited by 1,377
Exact solutions to the nonlinear dynamics of learning in deep linear neural networksPreprint
Saxe, A., McClelland, J. L., & Ganguli, S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.6120 · Cited by 1,010
MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
Richardson, M., Burges, C., & Renshaw, E. (2013) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1020 · Cited by 778 · cited 5×
“Like Richardson et al. (2013) we want to encourage reasoning in comprehension models.”
BackgroundInfluential
Research works printed but not linked: 3
- NewsQA: A Machine Comprehension Dataset https://aclanthology.org/W17-2623/ (Trischler et al., RepL4NLP 2017)
- Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2017. NewsQA: A Machine Comprehension Dataset. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 191–200, Vancouver, Canada. Association for Computational Linguistics.
- ACL materials are Copyright © 1963–2026 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License.
Cited by
Sorted by citations
Items 301–400 of 708
Verification mechanism to obtain an elaborate answer span in machine reading comprehension
Yu, P., Li, X., Song, J., Luo, Y., Hu, S., & Qian, W. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.08.084 · Cited by 12
IDK-MRC: Unanswerable Questions for Indonesian Machine Reading Comprehension
Putri, R. A., & Oh, A. H. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.465 · Cited by 14
Graph-free Multi-hop Reading Comprehension: A Select-to-Guide StrategyPreprint
Wu, B., Zhang, Z., & Zhao, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.11823 · Cited by 24
“N EURAL methods have achieved impressive results on single-hop machine reading comprehension (MRC) datasets [1], [2], [3], [4], [5], where questions are usually straightforward and answers are easy to find.”
MethodLabutov, I., Yang, B., Prakash, A., & Azaria, A. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1077 · Cited by 11
“, 2016), NewsQA (Trischler et al., 2016), and TriviaQA (Joshi et al.”
BackgroundMethodMulti-Granular Sequence Encoding via Dilated Compositional Units for Reading Comprehension
Tay, Y., Luu, A. T., & Hui, S. C. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1238 · Cited by 10
“New datasets have been recently released (Koˇcisk`y et al., 2017; Joshi et al., 2017; Lai et al., 2017; Welbl et al., 2017; Dhingra et al., 2017; Trischler et al., 2016), claiming to involve a greater need for going beyond simple surface-level matching.”
BackgroundAnswer Uncertainty and Unanswerability in Multiple-Choice Machine Reading Comprehension
Raina, V., & Gales, M. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.82 · Cited by 11
“In recent years, popular MRC datasets (Richardson et al., 2013; Chen et al., 2016; Lai et al., 2017; Trischler et al., 2017; Rajpurkar et al., 2018; Yang et al., 2018; Yu et al., 2020) have consistently observed increasingly competitive systems topping public leaderboards (Trischler et al.”
BackgroundQuestion Generation Using Sequence-to-Sequence Model with Semantic Role Labels
Naeiji, A., An, A., Davoudi, H., Delpisheh, M., & Alzghool, M. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.207 · Cited by 13
“Table 12 shows the results of different variations of our method on the NewsQA dataset, compared with BART and T5 baselines.”
InfluentialSocratic Question Generation: A Novel Dataset, Models, and Evaluation
Ang, B. H., Gollapalli, S. D., & Ng, S.-K. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.12 · Cited by 13
“Consider for instance, the publicly-accessible datasets available for learning question and dialog generation models (Rajpurkar et al., 2016; Ramnath et al., 2021; Talmor et al., 2017; Trischler et al., 2017; Yang et al., 2018).”
BackgroundGan, Y., & Zhu, D. (2024) · Innovations in Applied Engineering and Technology · doi:10.62836/iaet.v3i1.213 · Cited by 11
“● NewsQA Dataset NewsQA Dataset [45] is specifically designed for research in machine reading comprehension and question-answering systems.”
BackgroundEkram, S. M. S., Rahman, A. A., Altaf, M. S., Islam, M. S., Rahman, M. M., Rahman, M. M., Hossain, M. A., & Kamal, A. R. M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.186 · Cited by 13
“…al., 2018), multiple documents as context (Welbl et al., 2018), fill in the blank type questions (Šuster and Daelemans, 2018), questions in natural language (Trischler et al., 2017), single span-based replies (Rajpurkar et al., 2018), answers in natural language (Nguyen et al., 2016), and so forth.”
BackgroundInfluentialPolyglot Prompt: Multilingual Multitask Prompt Training
Fu, J., Ng, S.-K., & Liu, P. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.674 · Cited by 33
TASA: Deceiving Question Answering Models by Twin Answer Sentences Attack
Cao, Y., Li, D., Fang, M., Zhou, T., Gao, J., Zhan, Y., & Tao, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.821 · Cited by 15
Maharana, A., & Bansal, M. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.333 · Cited by 10
Farooq, H., & Kaushik, B. (2020) · International Conference on Intelligent Computing and Control Systems (ICICCS) · doi:10.1109/iciccs48265.2020.9121015 · Cited by 11
“(2017) Span prediction SQuAD and NewsQA [37] EM and F1 CNN(embedding) with lookup table for words, Bi RNN, Beam search Rules for development of a basic neural network for machine-reading comprehension was given through FastQA mainly focused on Extractive Question answering and shows that for high performance, two impo…”
MethodCopyBERT: A Unified Approach to Question Generation with Self-Attention
Varanasi, S., Amin, S., & Neumann, G. (2020) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2020.nlp4convai-1.3 · Cited by 12
“With more question answering (QA) datasets like SQuAD (Rajpurkar et al., 2016) that have been released recently (Trischler et al., 2016; Choi et al., 2018; Reddy et al., 2019; Yang et al., 2018), there has been an increased interest in QG, as these datasets can not only be used for creating QA…”
BackgroundImproving QA Generalization by Concurrent Modeling of Multiple Biases
Wu, M., Moosavi, N. S., Rücklé, A., & Gurevych, I. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.74 · Cited by 14
Project PIAF: Building a Native French Question-Answering DatasetPreprint
Keraron, R., Lancrenon, G., Bras, M., Allary, F., Moyse, G., Scialom, T., Soriano-Morales, E.-P., & Staiano, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00968 · Cited by 15
“Based on CNN news articles, NewsQA (Trischler et al., 2017) consists of 120k Q&A pairs, including unanswerable questions, collected using a 3-stage, siloed process; crowdworkers are split into 3 groups: Questioners, who see only an article’s headline and highlights to produce a question; Answerers, who see the questio…”
MethodMahbub, M., Begoli, E., Martins, S. B., Peluso, A., Tamang, S. R., & Peterson, G. D. (2023) · IEEE Access · doi:10.1109/access.2023.3235265 · Cited by 9
“Following SQuAD, researchers have developed several large-scale MRC datasets on domains such as news articles (NewsQA [13]), web search log (MS MARCO [14]), etc.”
MethodRobust Question Answering Through Sub-part Alignment
Chen, J., & Durrett, G. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.98 · Cited by 10
Learning to Generate Questions by Learning to Recover Answer-containing Sentences
Back, S., Kedia, A., Chinthakindi, S. C., Lee, H., & Choo, J. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.132 · Cited by 11
“Our approach improves question generation ability of existing methods, achieves new state-of-the-art results on MS MARCO and NewsQA, and the synthetic data increases downstream MRC accuracy across a wide range of datasets without any modification to the existing MRC models.”
MethodResultInfluentialUnified Question Generation with Continual Lifelong Learning
Yuan, W., Yin, H., He, T., Chen, T., Wang, Q., & Cui, L. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3511930 · Cited by 11
“SQuAD [58], MSMARCO [4] and newsQA [73] are three famous datasets used for answer-extraction QG, collected from Wikipedia, Bing search logs, and CNN news respectively.”
BackgroundConstructing Arabic Reading Comprehension Datasets: Arabic WikiReading and KaifLematha
Albilali, E., Al-Twairesh, N., & Hosny, M. (2022) · Language Resources and Evaluation · doi:10.1007/s10579-022-09577-5 · Cited by 14
“The NewsQA dataset (Trischler et al., 2017) reduces the lexical similarity between questions and context by decoupling the question–answer collection.”
BackgroundMethodInfluentialA Comparison of Transformer-Based Language Models on NLP Benchmarks
Greco, C. M., Tagarelli, A., & Zumpano, E. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-08473-7_45 · Cited by 10
On Continual Model Refinement in Out-of-Distribution Data Streams
Lin, B. Y., Wang, S. I., Lin, X., Jia, R., Xiao, L., Ren, X., & Yih, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.223 · Cited by 17
CS1QA: A Dataset for Assisting Code-based Question Answering in an Introductory Programming Course
Lee, C., Seonwoo, Y., & Oh, A. H. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.148 · Cited by 20
“Questions about specific domains are asked in NewsQA (Trischler et al., 2016) and TechQA (Castelli et al., 2020), and images are provided with the question in visual QA (Antol et al., 2015).”
Mulla, N., & Gharpure, P. (2023) · International Journal of Information Technology · doi:10.1007/s41870-023-01262-2 · Cited by 10
Continual Domain Adaptation for Machine Reading Comprehension
Su, L., Guo, J., Zhang, R., Fan, Y., Lan, Y., & Cheng, X. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3412047 · Cited by 10
Enhancing Dual-Encoders with Question and Answer Cross-Embeddings for Answer Retrieval
Wang, Y., Bai, J., Wang, Y., Zhang, J., Rong, W., Ji, Z., Wang, S., & Xiao, J. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.198 · Cited by 12
“, 2018) and NewsQA (Trischler et al., 2017) datasets for more experiments.”
Ji, Y., Chen, L., Dou, C., Ma, B., & Li, X. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.96 · Cited by 12
“With the help of various large-scale reading comprehension datasets (Hermann et al., 2015; Hill et al., 2015; Trischler et al., 2016; Rajpurkar et al., 2016; Lai et al., 2017; Rajpurkar et al., 2018), neural networks have achieved a great success on MRC in recent years.”
BackgroundLearning a Cost-Effective Annotation Policy for Question Answering
Kratzwald, B., Feuerriegel, S., & Sun, H. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.246 · Cited by 13
A Survey of Document Grounded Dialogue Systems (DGDS)Preprint
Ma, L., Zhang, W., Li, M., & Liu, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.13818 · Cited by 15
“In fact, if a single round QA is counted as a conversation, the DGDS could cover some popular MRC 6 tasks [34, 35, 44, 67, 79, 102].”
MethodExploring and Predicting Transferability across NLP TasksPreprint
Vu, T., Wang, T., Munkhdalai, T., Sordoni, A., Trischler, A., Mattarella-Micke, A., Maji, S., & Iyyer, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.00770 · Cited by 10
“…and Berant, 2019a) repository21, including the Stanford Question Answering datasets SQuAD-1 and SQuAD-2 (Rajpurkar et al., 2016, 2018); NewsQA (Trischler et al., 2017); HotpotQA (Yang et al., 2018) – the version where the context includes 10 paragraphs retrieved by an information retrieval…”
MethodSelf-citationMore Than Reading Comprehension: A Survey on Datasets and Metrics of Textual Question AnsweringPreprint
Bai, Y., & Wang, D. Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.12264 · Cited by 19
“NewsQA[45] is a large-scale extractive MRC dataset proposed by researchers from Microsoft.”
BackgroundMethodInfluentialForeseeing the Benefits of Incidental Supervision
He, H., Zhang, M., Ning, Q., & Roth, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.134 · Cited by 12
“For QA, we consider SQuAD (Rajpurkar et al., 2016), QAMR (Michael et al., 2017), QA-SRL Bank 2.0 (FitzGerald et al., 2018), QA-RE (Levy et al., 2017), NewsQA (Trischler et al., 2017), Triv-iaQA (Joshi et al., 2017).”
InfluentialMucciaccia, S. S., Paixão, T. M., Mutz, F. W., Souza, A. F. D., Badue, C. S., & Oliveira-Santos, T. (2025) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2025.5801 · Cited by 10
XCMRC: Evaluating Cross-Lingual Machine Reading Comprehension
Liu, P., Deng, Y., Zhu, C., & Hu, H. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32233-5_43 · Cited by 9
Event-Centric Question Answering via Contrastive Learning and Invertible Event Transformation
Lu, J., Tan, X., Pergola, G., Gui, L., & He, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.176 · Cited by 9
Interactive Machine Comprehension with Information Seeking Agents
Yuan, X., Fu, J., Côté, M.-A., Tay, Y., Pal, C., & Trischler, A. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.211 · Cited by 11
Self-citationRead and Reason with MuSeRC and RuCoS: Datasets for Machine Reading Comprehension for Russian
Fenogenova, A., Mikhailov, V., & Shevelev, D. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.570 · Cited by 9
“The task has been widely explored for the English language targeting different aspects of reading comprehension (Hermann et al., 2015; Hill et al., 2015; Rajpurkar et al., 2016; Trischler et al., 2016; Joshi et al., 2017; Rajpurkar et al., 2018).”
BackgroundAnswerQuest: A System for Generating Question-Answer Items from Multi-Paragraph Documents
Roemmele, M., Sidhpura, D., DeNeefe, S., & Tsou, L. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-demos.6 · Cited by 12
Shinoda, K., Sugawara, S., & Aizawa, A. (2021) · doi:10.18653/v1/2021.acl-srw.21 · Cited by 10
“Machine reading comprehension has gained signifi-cant attention in the NLP community, whose goal is to devise systems that can answer questions about given documents (Rajpurkar et al., 2016; Trischler et al., 2017; Joshi et al., 2017).”
BackgroundMethodTracing Origins: Coreference-aware Machine Reading Comprehension
Huang, B., Zhang, Z., & Zhao, H. Y. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.91 · Cited by 10
“…et al., 2019a; Lewis et al., 2020), and more datasets on question answering were introduced to increase the difficulty in this task, including NewsQA (Trischler et al., 2017), SearchQA (Dunn et al., 2017), QuAC (Choi et al., 2018), HotpotQA (Yang et al., 2018), NarrativeQA (Koˇciský et al., 2018),…”
BackgroundMethodInfluentialSupervised and Unsupervised Transfer Learning for Question AnsweringPreprint
Chung, Y.-A., Lee, H.-Y., & Glass, J. R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05345 · Cited by 9
“A story is a sequence of sentences, and can be in the form of plain text (Trischler et al., 2017; Rajpurkar et al., 2016; Weston et al., 2016; Yang et al., 2015) or spoken content (Tseng et al.”
BackgroundTowards Automatic Generation of Questions from Long AnswersPreprint
Mishra, S. K., Goel, P., Sharma, A., Jagannatha, A., Jacobs, D. W., & Daumé, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.05109 · Cited by 11
“…datasets with short answers (one to three sentences long on average) either as spans within longer paragraphs such as SQuAD (Rajpurkar et al., 2016) and NewsQA (Trischler et al., 2017) or human-generated short answers in MS MARCO (Nguyen et al., 2016) and Narrative QA (Kocisky et al., 2018).”
BackgroundInfluentialRethinking Search: Making Experts out of DilettantesPreprint
Metzler, D., Tay, Y., Bahri, D., & Najork, M. (2021) · arXiv (Cornell University) · Cited by 9
Hyperbolic Pre-Trained Language Model
Chen, W., Han, X., Lin, Y., He, K., Xie, R., Zhou, J., Liu, Z., & Sun, M. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3407575 · Cited by 8
“…structure brought by hyperbolic space can lead to a better performance in this task. a) Datasets: We evaluate the models on six datasets in different domains provided in MRQA benchmark [55], including SQuAD [56], NewsQA [57], TriviaQA [58], SearchQA [59], HotpotQA [60] and NaturalQA [61].”
MethodEnsemble ALBERT and RoBERTa for Span Prediction in Question Answering
Bachina, S., Balumuri, S., & Kamath, S. S. (2021) · doi:10.18653/v1/2021.dialdoc-1.9 · Cited by 8
“0 (Rajpurkar et al., 2018), Natural Questions (Kwiatkowski et al., 2019), NewsQA (Trischler et al., 2017) and HotpotQA (Yang et al., 2018).”
BackgroundEvaluation Paradigms in Question Answering
Rodríguez, P., & Boyd-Graber, J. L. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.758 · Cited by 12
Clean or Annotate: How to Spend a Limited Data Collection Budget
Chen, D., & Bowman, Z. Y. A. S. R. (2022) · Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo) · doi:10.18653/v1/2022.deeplo-1.17 · Cited by 13
“DistilRoBERTa represents a distilled version of RoBERTa-base (Liu et al., 2019).”
BackgroundLeveraging QA Datasets to Improve Generative Data Augmentation
Mekala, D., Vu, T., Schick, T., & Shang, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.660 · Cited by 10
“• NewsQA (Trischler et al., 2017)”
BackgroundDiversify Question Generation with Retrieval-Augmented Style Transfer
Gou, Q., Xia, Z., Yu, B., Yu, H., Huang, F., Li, Y., & Nguyen, C.-T. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.104 · Cited by 12
Relation Module for Non-Answerable Predictions on Reading Comprehension
Huang, K., Tang, Y., Huang, J., He, X., & Zhou, B. (2019) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k19-1070 · Cited by 11
“By taking a weighted sum of all of the relation scores that the sentence has, we generate a non-answerable score that is trained jointly with answer span scores from any MRC model to determine non-answerability.”
BackgroundInfluentialCommonsense Inference in Natural Language Processing (COIN) - Shared Task Report
Ostermann, S., Zhang, S., Roth, M., & Clark, P. E. (2019) · doi:10.18653/v1/d19-6007 · Cited by 7
“NewsQA (Trischler et al., 2017) provides news texts with crowdsourced questions and answers, which are spans of the evidence documents.”
BackgroundREM-Net: Recursive Erasure Memory Network for Commonsense Evidence Refinement
Huang, Y., Fang, M., Zhan, X., Cao, Q., & Liang, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i7.16791 · Cited by 7
Common Sense-Based Reasoning Using External Knowledge for Question Answering
Yang, Y., & Kang, S. (2020) · IEEE Access · doi:10.1109/access.2020.3045762 · Cited by 9
“There is a number of datasets for span-based QA, such as SQuAD [8], TriviaQA [9], and NewsQA [10], as well as many models such as BiDAF [11], DocQA [12], and BERT [2]) that perform well.”
MethodKeep Learning: Self-supervised Meta-learning for Learning from Inference
Kedia, A., & Chinthakindi, S. C. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.6 · Cited by 7
“We use the official implementation and pre-trained models of BERT-large-cased for SuperGLUE tasks, and BERT-base-uncased for Ubuntu Dialog Corpus, NewsQA, and for our ablation tests on SQuAD.”
BackgroundInfluentialA Survey of Machine Reading Comprehension Methods
Xu, X., Tohti, T., & Hamdulla, A. (2022) · International Conference on Asian Language Processing · doi:10.1109/ialp57159.2022.9961260 · Cited by 8
CLER: Cross-task Learning with Expert Representation to Generalize Reading and Understanding
Takahashi, T., Taniguchi, M., Taniguchi, T., & Ohkuma, T. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5824 · Cited by 9
“The shared task released six-domain datasets (Rajpurkar et al., 2016; Trischler et al., 2017; Joshi et al., 2017; Dunn et al., 2017; Yang et al., 2018; Kwiatkowski et al., 2019) to train and vali-184 date the model as in-domain settings, and unveiled six out of the twelve test datasets 1 (Dua et…”
BackgroundMethodLearning to Search in Long Documents Using Document StructurePreprint
Geva, M., & Berant, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.03529 · Cited by 11
“While early work focused on simple questions and short paragraphs (Hermann et al., 2015; Rajpurkar et al., 2016; Trischler et al., 2017; Onishi et al., 2016), current work is shifting towards more complex questions that require reasoning over long documents (Joshi et al.”
BackgroundA Vietnamese Dataset for Evaluating Machine Reading ComprehensionPreprint
Nguyen, K. V., Nguyen, D.-V., Nguyen, A. G.-T., & Nguyen, N. L.-T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.14725 · Cited by 8
“performances, which is similar to results on SQuAD and NewsQA (Trischler et al., 2016).”
BackgroundMethodResultInfluentialDiversify Question Generation with Retrieval-Augmented Style Transfer
Gou, Q., Xia, Z., Yu, B., Yu, H., Huang, F., Li, Y., & Cam-Tu, N. (2023) · arXiv · doi:10.48550/arxiv.2310.14503 · Cited by 8
“Compared to the previous methods, RAST achieve the best diversity score (the lowest Pairwise-BLEU) on SQuAD/1 and NewsQA, and the second-best on SQuAD/2.”
BackgroundMethodInfluentialHandover QG: Question Generation by Decoder Fusion and Reinforcement Learning
Chung, H.-L., Chan, Y.-H., & Fan, Y.-C. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3426292 · Cited by 8
“…are two sorts of datasets for QG training: (1) factoid-type datasets, originally created for training machine reading comprehension system (e.g., SQuAD [8] or NewsQA [9]), and (2) exam-type datasets, collected from educational reading comprehension tests (e.g., RACE [10], [11] or Learning-Q [12]).”
BackgroundMaximizing discrimination masking for faithful question answering with machine reading
Dong, L., Tang, J., Wang, P., Li, S., & Wang, T. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103915 · Cited by 7
Self-citationUnsupervised Domain Adaptation of Language Models for Reading ComprehensionPreprint
Nishida, K., Nishida, K., Saito, I., Asano, H., & Tomita, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.10768 · Cited by 19
“NewsQA is an RC dataset from CNN news (Trischler et al., 2017).”
semiPQA: A Study on Product Question Answering over Semi-structured Data
Shen, X., Barlacchi, G., Tredici, M. D., Cheng, W., & de Gispert, A. (2022) · Workshop on e-Commerce and NLP (ECNLP) · doi:10.18653/v1/2022.ecnlp-1.14 · Cited by 8
“Due to the small size of our training data, we follow the common practice of pre-finetuning the attribute ranker on four large-scale QA datasets: Natural Questions (Kwiatkowski et al., 2019), AmazonQA (McAuley and Yang, 2016), NewsQA (Trischler et al., 2017) and Squad (Rajpurkar et al., 2016).”
BackgroundMethodInfluentialSurvey of Large Language Models for Answering Questions Across Various Fields
Shailendra, P., Ghosh, R. C., Kumar, R., & Sharma, N. (2024) · International Conference on Advanced Computing and Communication Systems (ICACCS) · doi:10.1109/icaccs60874.2024.10717078 · Cited by 12
“They have used the Covid-19[25], NewsQA [26], and QAConv [27] datasets for generating knowledge bases in these domains.”
MethodCan Machines Learn to Comprehend Scientific Literature?
Park, D., Choi, Y., Kim, D., Yu, M., Kim, S., & Kang, J. (2019) · IEEE Access · doi:10.1109/access.2019.2891666 · Cited by 7
“‘‘NewsQA: A machine comprehension dataset.’’”
MethodInfluentialComprehensive Multi-Dataset Evaluation of Reading Comprehension
Dua, D., Gottumukkala, A., Talmor, A., Gardner, M., & Singh, S. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5820 · Cited by 6
“NewsQA (Trischler et al., 2017) dataset focuses on paraphrased questions with predicate-argument structure understanding.”
BackgroundImproved Machine Reading Comprehension Using Data Validation for Weakly Labeled Data
Yang, Y., Kang, S., & Seo, J. (2020) · IEEE Access · doi:10.1109/access.2019.2963569 · Cited by 7
“Recently, several approaches [14]–[19] that address the use of large scale datasets for MRC have been proposed; the datasets used in such studies include: Stanford Question Answering Dataset (SQuAD) [20], WikiQA [21], NewsQA [22], and TriviaQA [23].”
MethodAn Empirical Study of Content Understanding in Conversational Question Answering
Chiang, T.-R., Ye, H.-T., & Chen, Y.-N. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6257 · Cited by 8
Campos, J. A., Cho, K., Otegi, A., Soroa, A., Agirre, E., & Azkune, G. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.230 · Cited by 9
Semantics Altering Modifications for Evaluating Comprehension in Machine Reading
Schlegel, V., Nenadić, G., & Batista-Navarro, R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17622 · Cited by 8
Evidence Augment for Multiple-Choice Machine Reading Comprehension by Weak Supervision
Luo, D., Zhang, P., Ma, L., Zhu, X., Zhou, M., Liang, Q., Wang, B., & Wang, L. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86383-8_29 · Cited by 7
UKP-SQUARE: An Online Platform for Question Answering Research
Baumgärtner, T., Wang, K., Sachdeva, R. S., Geigle, G., Eichler, M., Poth, C., Sterz, H., Puerto, H., Ribeiro, L. F. R., Pfeiffer, J., et al. (2022) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2022.acl-demo.2 · Cited by 7
Designing and Evaluating Interfaces that Highlight News Coverage Diversity Using Discord Questions
Laban, P., Wu, C.-S., Murakhovs’ka, L., Chen, X., & Xiong, C. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581569 · Cited by 9
Bridging Speech and Textual Pre-Trained Models With Unsupervised ASR
Shi, J., Hsu, C.-J., Chung, H.-L., Gao, D., Garcia, L. P., Watanabe, S., Lee, A., & Lee, H.-Y. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096827 · Cited by 9
LiveQA: A Question Answering Dataset Over Sports Live
Liu, Q., Jiang, S., Wang, Y., & Li, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-63031-7_23 · Cited by 8
Interactive Language Learning by Question AnsweringPreprint
Yuan, X., Côté, M.-A., Fu, J., Lin, Z., Pal, C., Bengio, Y., & Trischler, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.10909 · Cited by 13
“datasets (Rajpurkar et al., 2016; Trischler et al., 2016; Nguyen et al., 2016; Reddy et al., 2018; Yang et al., 2018) and a host of neural models that address them (Wang and Jiang, 2016; Wang et al.”
BackgroundSelf-citationFAQ-Gen: An automated system to generate domain-specific FAQs to aid content comprehension
Kale, S., Khaire, G., & Patankar, J. (2024) · Journal of Computer-Assisted Linguistic Research · doi:10.4995/jclr.2024.21178 · Cited by 9
Jeong, S., Baek, J., Hwang, S. J., & Park, J. C. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.374 · Cited by 6
Exploring The Landscape of Distributional Robustness for Question Answering Models
Awadalla, A., Wortsman, M., Ilharco, G., Min, S., Magnusson, I., Hajishirzi, H., & Schmidt, L. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.441 · Cited by 20
ArQuAD: An Expert-Annotated Arabic Machine Reading Comprehension Dataset
Obeidat, R., Al-Harbi, M., Al‐Ayyoub, M., & Alawneh, L. M. (2024) · Cognitive Computation · doi:10.1007/s12559-024-10248-6 · Cited by 9
“The type of reasoning required for answering a question significantly influences computational models’ learning abilities [8].”
Answering Naturally : Factoid to Full length Answer Generation
Pal, V., Shrivastava, M., & Bhat, I. A. (2019) · Workshop on New Frontiers in Summarization (NewSum) · doi:10.18653/v1/d19-5401 · Cited by 5
“We have also performed cross-dataset evaluation on a knowledge base dataset(Freebase) and a machine comprehension dataset(NewsQA) to the generalization capability of our system.”
MethodInfluentialHandling Anomalies of Synthetic Questions in Unsupervised Question Answering
Hong, G., Kang, J., Lim, D., & Myaeng, S.-H. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.306 · Cited by 7
“The latter dataset includes not only SQuAD but also many other QA datasets such as NewsQA (Trischler et al., 2017), TriviaQA (Joshi et al., 2017), SearchQA (Dunn et al., 2017), HotpotQA (Yang et al., 2018), and NaturalQuestions (Kwiatkowski et al., 2019).”
BackgroundContextual and Semantic Fusion Network for Multiple-Choice Reading Comprehension
Duan, Q., Huang, J., & Wu, H. (2021) · IEEE Access · doi:10.1109/access.2021.3068993 · Cited by 6
“The answers of extractive MRC datasets are text spans from the given passage (e.g., CNN/Daily Mail [19], SQuAD [20], and NewsQA [21]).”
BackgroundMeta-learning for compressed language model: A multiple choice question answering study
Yan, M., & Pan, Y. F. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.01.148 · Cited by 6
AttractionDetailsQA: An Attraction Details Focused on Chinese Question Answering Dataset
Huang, W., Xu, S., Wang, Y., Jin, F., & Chang, Q. (2022) · IEEE Access · doi:10.1109/access.2022.3181188 · Cited by 7
“1 http://travel.cnr.cn/dj/20220126/t20220126_525726578.shtml that recently proposed QA datasets usually focus on news and movies domains, such as CNN/DAILYMAIL [1], NewsQA [2] and WIKIMOVIES [3].”
BackgroundTourism Information QA Datasets for Smart Tourism Chatbot
Ko, T.-S., Jwa, M.-C., & Jwa, J.-W. (2023) · International Journal of Membrane Science and Technology · doi:10.15379/ijmst.v10i1.1451 · Cited by 6
“NewsQA consists of over 100,000 question-answer pairs from over 10,000 CNN news articles (Adam Trischler, 2017).”
BackgroundThe DistilBERT Model: A Promising Approach to Improve Machine Reading Comprehension Models
P, S. V., Shahapure, N. H., Rekha, P. M., Nethravathi, B., Khandelwal, P., Anand, A., Agrawal, P., & Srivastava, V. (2023) · International Journal on Recent and Innovation Trends in Computing and Communication · doi:10.17762/ijritcc.v11i8.7957 · Cited by 9
MPrompt: Exploring Multi-level Prompt Tuning for Machine Reading Comprehension
Chen, G., Qian, Y., Wang, B., & Li, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.343 · Cited by 7
YTCommentQA: Video Question Answerability in Instructional Videos
Yang, S., Park, S., Jang, Y., & Lee, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29906 · Cited by 11
Machine Comprehension Improves Domain-Specific Japanese Predicate-Argument Structure Analysis
Takahashi, N., Shibata, T., Kawahara, D., & Kurohashi, S. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5814 · Cited by 7
“In MC studies, QA datasets consisting of triplets of a document, a question and its answer are constructed, and an MC model is trained using these datasets (e.g., Rajpurkar et al. (2016) and Trischler et al. (2017)).”
MethodCHIME: Cross-passage Hierarchical Memory Network for Generative Review Question Answering
Lu, J., Pergola, G., Gui, L., Li, B., & He, Y. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.229 · Cited by 9
Prabowo, M. W., Budi, I. D., & Santoso, H. B. (2021) · International Conference on Software and Computer Applications (ICSCA) · doi:10.1145/3457784.3457824 · Cited by 6
“Deep learning approach can be possible because of the availability for the big dataset, such as SQUAD [7], NewsQA [8], or MSMarco [9].”
MethodGenerating Answer Candidates for Quizzes and Answer-Aware Question Generators
Vachev, K., Hardalov, M., Karadzhov, G., Georgiev, G., Releva AI, S., Koychev, I. K., & Nakov, P. (2021) · Student Research Workshop .../Proceedings of the Student Research Workshop ... · doi:10.26615/issn.2603-2821.2021_028 · Cited by 7
Less Is More: Domain Adaptation with Lottery Ticket for Reading Comprehension
Zhu, H., Wang, Z., Zhang, H., Liu, M., Zhao, S., & Qin, B. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.95 · Cited by 7
“1 (Rajpurkar et al., 2016), NQ (Kwiatkowski et al., 2019) and NewsQA (Trischler et al., 2017), that differ in the sources of the context passages and question types.”
BackgroundMethodInfluentialBIOMRC: A Dataset for Biomedical Machine Reading ComprehensionPreprint
Stavropoulos, P., Pappas, D., Androutsopoulos, I., & McDonald, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.06376 · Cited by 6
“There are also several large open-domain MRC datasets annotated by humans (Kwiatkowski et al., 2019; Rajpurkar et al., 2016, 2018; Trischler et al., 2017; Nguyen et al., 2016; Lai et al., 2017).”
BackgroundActive Learning for Natural Language Generation
Perlitz, Y., Gera, A., Shmueli-Scheuer, M., Sheinwald, D., Slonim, N., & Ein‐Dor, L. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.611 · Cited by 5
Bayesian Multi-Task Transfer Learning for Soft Prompt Tuning
Lee, H., Jeong, M., Yun, S.-Y., & Kim, K.-E. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.329 · Cited by 6
Question Answering Systems Based on Pre-trained Language Models: Recent Progress
Luo, X., Luo, Y., & Yang, B. (2024) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-57808-3_13 · Cited by 6
Metadata
- Authors
- Trischler, A., Wang, T., Yuan, X., Harris, J., Sordoni, A., Bachman, P., & Suleman, K.
- Venue
- Proceedings of the 2nd Workshop on Representation Learning for NLP
- Published
- 2017
- Pages
- 191-200
- OpenAlex
- W2557764419
- Cited by
- 706
- References
- 24
- Citations by year
- 2026: 18 · 2025: 26 · 2024: 39 · 2023: 96 · 2022: 110 · 2021: 134 · 2020: 138 · 2019: 87 · 2018: 41 · 2017: 16
- FWCI
- 35.17
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Workshop on Representation Learning for NLP (RepL4NLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- machine reading comprehension, question answering, news articles, exploratory questions, reasoning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
