The DOI matches its Crossref record; 22 of the 24 references it cites were found in a registry or an index.
NewsQA: A Machine Comprehension Dataset
Open accessAbstract
We present NewsQA, a challenging machine comprehension dataset of over 100,000 human-generated question-answer pairs. Crowdworkers supply questions and answers based on a set of over 10,000 news articles from CNN, with answers consisting of spans of text in the articles. We collect this dataset through a four-stage process designed to solicit exploratory questions that require reasoning. Analysis confirms that NewsQA demands abilities beyond simple word matching and recognizing textual entailment. We measure human performance on the dataset and compare it to several strong neural models. The performance gap between humans and machines (13.3% F1) indicates that significant progress can be made on NewsQA through future research. The dataset is freely available online.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“Existing datasets vary in size, difficulty, and collection methodology; however, as pointed out by Rajpurkar et al. (2016), most suffer from one of two shortcomings: those that are designed explicitly to test comprehension (Richardson et al.”
BackgroundGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 2×
“Word embeddings are initialized using GloVe vectors (Pennington et al., 2014) pre-trained on the 840-billion Common Crawl corpus.”
BackgroundMethodBLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 1×
“BLEU is a precision-based metric popular in machine translation that uses a weighted average of variable length phrase matches ( n -grams) against the reference sentence (Papineni et al., 2002).”
MethodSQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 3×
“Existing datasets vary in size, difficulty, and collection methodology; however, as pointed out by Rajpurkar et al. (2016), most suffer from one of two shortcomings: those that are designed explicitly to test comprehension (Richardson et al., 2013) are too small for training data-intensive deep…”
BackgroundMethodInfluentialCIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · cited 2×
“ase matches (n-grams) against the reference sentence (Papineni et al., 2002). CIDEr was designed to correlate better with human judgements of sentence similarity, and uses tf-idf scores over n-grams (Vedantam et al., 2015). As given in Table 4, humans averaged 0.694 F1 on NewsQA. The human EM scores are relatively low…”
MethodInfluentialOn the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,855
Liu, C., Lowe, R., Serban, I. V., Noseworthy, M., Charlin, L., & Pineau, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1230 · Cited by 1,377
Exact solutions to the nonlinear dynamics of learning in deep linear neural networksPreprint
Saxe, A., McClelland, J. L., & Ganguli, S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.6120 · Cited by 1,010
MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
Richardson, M., Burges, C., & Renshaw, E. (2013) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1020 · Cited by 778 · cited 5×
“Like Richardson et al. (2013) we want to encourage reasoning in comprehension models.”
BackgroundInfluential
Research works printed but not linked: 3
- NewsQA: A Machine Comprehension Dataset https://aclanthology.org/W17-2623/ (Trischler et al., RepL4NLP 2017)
- Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2017. NewsQA: A Machine Comprehension Dataset. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 191–200, Vancouver, Canada. Association for Computational Linguistics.
- ACL materials are Copyright © 1963–2026 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License.
Cited by
Sorted by citations
Items 401–500 of 708
Source-Free Domain Adaptation for Question Answering with Masked Self-training
Yin, M. J., Wang, B., Dong, Y., & Ling, C. X. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00669 · Cited by 7
“As depicted in Table 1, the NewsQA seems to be the hardest one for domain adaptation.”
MethodInfluentialMaity, S., Deroy, A., & Sarkar, S. (2024) · SN Computer Science · doi:10.1007/s42979-024-03464-2 · Cited by 7
NUT-RC: Noisy User-generated Text-oriented Reading Comprehension
Huang, R., Zou, B., Hong, Y., Zhang, W., Aw, A. T., & Zhou, G. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.242 · Cited by 5
“However, most of the existing RC models are developed for formal text, such as news articles (Hermann et al., 2015; Trischler et al., 2017) and Wikipedia (Rajpurkar et al.”
BackgroundMethodA novel multi-domain machine reading comprehension model with domain interference mitigation
Zhou, C., Wang, Z., He, S., Zhang, H., & Su, J. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.05.102 · Cited by 5
Cross-Lingual QA as a Stepping Stone for Monolingual Open QA in Icelandic
Snæbjarnarson, V., & Einarsson, H. (2022) · doi:10.18653/v1/2022.mia-1.4 · Cited by 6
Vector-Quantized Input-Contextualized Soft Prompts for Natural Language Understanding
Bhardwaj, R., Saha, A., Hoi, S. C. H., & Poria, S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.455 · Cited by 8
Prompt Learning for Few-Shot Question Answering via Self-Context Data Augmentation
Qiu, J.-Q., Zhang, C.-Y., & Chen, C. L. P. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3483201 · Cited by 5
“All six datasets of split I are utilized to evaluate to our method: SQuAD [1], NewsQA [41], TriviaQA [42], SearchQA [43], HotpotQA [44], and natural questions (NQ) [45].”
Self Data Augmentation for Open Domain Question Answering
Zhang, Q., Zheng, M., Chen, S., Liu, H., & Fang, M. (2024) · ACM Transactions on Information Systems · doi:10.1145/3707449 · Cited by 7
“Trischler et al. [56] transformed news articles to reading comprehension questions.”
SCDE: Sentence Cloze Dataset with High Quality Distractors From Examinations
Kong, X., Gangal, V., & Hovy, E. H. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.502 · Cited by 7
DramaQA: Character-Centered Video Story Understanding with Hierarchical QA
Choi, S.-H., On, K.-W., Heo, Y., Jang, Y., Seo, A., Lee, S.-C., Lee, M., & Zhang, B. (2021) · 정보과학회 컴퓨팅의 실제 논문지 · doi:10.5626/ktcp.2021.27.1.7 · Cited by 6
BiRdQA: A Bilingual Dataset for Question Answering on Tricky Riddles
Zhang, Y., & Wan, X. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21430 · Cited by 7
Domain adaptive multi-task transformer for low-resource machine reading comprehension
Bai, Z., Wang, B., Wang, Z., Yuan, C., & Wang, X. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.08.057 · Cited by 6
Gain more with less: Extracting information from business documents with small data
Nguyen, M.-T., Sơn, N. H., & Le, L. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.119274 · Cited by 5
Sultan, M. A., Sil, A., & Florian, R. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.247 · Cited by 5
“, 2016) 86,588 10,507 NewsQA (Trischler et al., 2017) 74,160 4,212 NQ (Kwiatkowski et al.”
FocusQA: Open-Domain Question Answering with a Context in Focus
Barlacchi, G., Lauriola, I., Moschitti, A., Tredici, M. D., Shen, X., Vu, T., Byrne, B., & de Gispert, A. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.381 · Cited by 5
“Second, F OCUS QA contains correct answers that are extracted from multiple documents (Table 5), whereas in MRC datasets (e.g., NewsQA (Trischler et al., 2017), SQuAD (Rajpurkar et al., 2016) and N ATURAL Q UESTIONS (Kwiatkowski et al., 2019)), answers are extracted only from the documents used to…”
BackgroundMethodInfluentialShortcomings of Question Answering Based Factuality Frameworks for Error Localization
Kamoi, R., Goyal, T., & Durrett, G. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.11 · Cited by 8
Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks
Sun, K., Qi, P., Zhang, Y., Liu, L., Wang, W. Y., & Huang, Z. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.887 · Cited by 10
Rethinking the Objectives of Extractive Question Answering
Fajčík, M., Jon, J., & Smrž, P. (2021) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/2021.mrqa-1.2 · Cited by 7
Neural Skill Transfer from Supervised Language Tasks to Reading ComprehensionPreprint
Mihaylov, T., Kozareva, Z., & Frank, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.03754 · Cited by 5
ReviewQA: a relational aspect-based opinion reading datasetPreprint
Grail, Q., & Pérez, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.12196 · Cited by 10
Multi-task Learning with Sample Re-weighting for Machine Reading ComprehensionPreprint
Xu, Y., Liu, X., Shen, Y., Liu, J., & Gao, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.06963 · Cited by 5
“For example, on NewsQA (Trischler et al., 2017), our model surpassed human performance by 13.4 (46.5 vs 59.9) and 3.2 (72.6 vs 69.4) absolute points in terms of exact match and F1.”
BackgroundMethodInfluentialLarge Scale Question Answering using Tourism DataPreprint
Contractor, D., Shah, K., Partap, A., Mausam, & Singla, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.03527 · Cited by 9
“…tasks such as those based on reading comprehension require answers to be generated either based on a single passage, or after reasoning over multiple passages (or small-sized documents) (e.g. SQuAD (Ra-jpurkar, Jia, and Liang, 2018), HotpotQA (Yang et al., 2018), NewsQA (Trischler et al., 2016)).”
BackgroundInfluentialMulti-Fact Correction in Abstractive Text SummarizationPreprint
Dong, Y., Wang, S., Gan, Z., Cheng, Y., Cheung, J. C. K., & Liu, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.02443 · Cited by 5
Vietnamese Legal Question Answering: An Experimental Study
Ha, N. T., Nguyen, T.-P., Trung, K. T., Le, H.-L., Hương, L. T., Nguyen, C. T., & Nguyen, M.-T. (2024) · International Conference on Knowledge and Systems Engineering/Proceedings · doi:10.1109/kse63888.2024.11063637 · Cited by 5
“After SQuAD, several open-domain English MRC datasets were introduced, such as MS MARCO [4], NewsQA [5], and NarrativeQA [7].”
Assessing Distractors in Multiple-Choice Tests
Raina, V., Liusie, A., & Gales, M. (2023) · Workshop on Evaluation and Comparison of NLP Systems (Eval4NLP) · doi:10.18653/v1/2023.eval4nlp-1.2 · Cited by 9
Language Models for Multi-Lingual Tasks - A Survey
Jafari, A. R., Heidary, B., Farahbakhsh, R., Salehi, M., & Crespi, N. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.01506146 · Cited by 6
Automatic Task Requirements Writing Evaluation via Machine Reading Comprehension
Xu, S., Xu, G., Jia, P., Ding, W., Wu, Z., & Liu, Z. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-78292-4_36 · Cited by 5
“Another line of research on MRC is how to construct high-quality datasets and lots of works have been done [18,7,27,10,23].”
BackgroundSyntax-Aware Multi-Spans Generation for Reading Comprehension
Zhang, Z., Zhang, Y., & Zhao, H. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3138679 · Cited by 10
Ask to Understand: Question Generation for Multi-hop Question Answering
Li, J., Ren, M., Gao, Y., & Yang, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-6207-5_2 · Cited by 4
Do Question Answering Modeling Improvements Hold Across Benchmarks?
Liu, N. F., Lee, T., Jia, R., & Liang, P. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.736 · Cited by 23
“We use the MRQA NewsQA, MRQA DROP, and MRQA HotpotQA benchmarks exactly as released by the MRQA 2019 shared task (Fisch et al., 2019).”
BackgroundMethodInfluentialEnhancing text comprehension for Question Answering with Contrastive Learning
Lee, S., & Lee, M. (2023) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2023.repl4nlp-1.7 · Cited by 4
“For other two datasets, SQuAD and NewsQA, the learning rates are set to 5 e − 5 and 1 e − 4 , respectively, and the model is trained for three epochs and five epochs, respectively.”
BackgroundMethodInfluentialComparing Neural Question Generation Architectures for Reading Comprehension
Perkoff, E. M., Bhattacharyya, A., Cai, J., & Cao, J. (2023) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/2023.bea-1.47 · Cited by 4
“The CVAE approach demonstrated that incorporating question type did improve diversity of responses both on SQuAD and NewsQA (Trischler et al., 2017).”
MethodPost-Abstention: Towards Reliably Re-Attempting the Abstained Instances in QA
Varshney, N., & Baral, C. R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.55 · Cited by 15
PIE-QG: Paraphrased Information Extraction for Unsupervised Question Generation from Small Corpora
Nagumothu, D., Ofoghi, B., Huang, G., & Eklund, P. (2022) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2022.conll-1.24 · Cited by 5
Towards Data-Effective Educational Question Generation with Prompt-Based Learning
Wu, Y., Nouri, J., Megyesi, B. B., Henriksson, A., Duneld, M., & Li, X. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-37717-4_11 · Cited by 6
Levy, M., Ravfogel, S., & Goldberg, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.569 · Cited by 5
Extractive Question Answering with Contrastive Puzzles and Reweighted Clues
Liu, C., Yang, J., & Li, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70552-6_6 · Cited by 4
Improving Low resource Reading Comprehension via Cross lingual Transposition Rethinking
Wu, G., Xu, B., Qin, Y., Wang, W., & Wang, G. (2021) · International Joint Conference on Knowledge Graphs (IJCKG) · doi:10.1145/3502223.3502234 · Cited by 6
“1 (Rajpurkar et al., 2016), TriviaQA (Joshi et al., 2017), NewsQA (Trischler et al., 2016), SQuAD 2.0 (Rajpurkar et al., 2018) and NQ (Kwiatkowski et al., 2019), along with powerful pretraining language models including BERT (Devlin et al., 2018), XLNet (Yang et al., 2019), GPT (Radford et al., 2018), RoBERTa (Liu et…”
InfluentialInteractive Gated Decoder for Machine Reading Comprehension
Cui, Y., Che, W., Yang, Z., Liu, T., Qin, B., Wang, S., & Hu, G.-P. (2022) · ACM Trans. Asian Low Resour. Lang. Inf. Process. · doi:10.1145/3501399 · Cited by 5
“We evaluate our approach on two popular span-extraction machine reading comprehension benchmarks: SQuAD 2.0 [23] and NewsQA [29].”
MethodInfluentialQasar: Self-Supervised Learning Framework for Extractive Question Answering
Assem, H., Sarkar, R., & Dutta, S. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671570 · Cited by 5
“• NewsQA Dataset [34]: Released by Microsoft, the dataset is formed of 100 , 000 human-generated question-answer pairs, where crowd-workers create questions and answers based on a set of over 10,000 news articles from CNN.”
BackgroundMethodSurvey of Machine Reading Comprehension Models and its Evaluation Metrics
Therasa, M., & Mathivanan, G. (2022) · International Conference on Computing Methodologies and Communication (ICCMC) · doi:10.1109/iccmc53470.2022.9754070 · Cited by 4
Task Transfer and Domain Adaptation for Zero-Shot Question Answering
Pan, X., Sheng, A., Shimshoni, D., Singhal, A., & Sil, S. R. A. A. (2022) · Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo) · doi:10.18653/v1/2022.deeplo-1.12 · Cited by 4
Le, H. T.-T., Ho, V.-D., Nguyen, D.-V., & Nguyen, N. L.-T. (2022) · NAFOSTED Conference on Information and Computer Science (NICS) · doi:10.1109/nics56915.2022.10013390 · Cited by 4
“Consequently, the researchers added unanswerable questions to the MRC datasets later on [5], [25], [26].”
BackgroundContext-Aware Transformer Pre-Training for Answer Sentence Selection
Liello, L. D., Garg, S., & Moschitti, A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.40 · Cited by 4
Few-shot Adaptation Works with UnpredicTable Data
Chan, J. S., Pieler, M., Jao, J., Scheurer, J., & Perez, E. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.102 · Cited by 7
Zeng, H., Wei, B., Liu, J., & Fu, W. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.500 · Cited by 5
“NewsQA (Trischler et al., 2017), and SQuAD (Rajpurkar et al., 2016) and three conversational datasets as the target datasets: CoQA (Reddy et al., 2019), QuAC (Choi et al., 2018), and DoQA (Cam-pos et al., 2020).”
MethodRobust Question Answering against Distribution Shifts with Test-Time Adaption: An Empirical Study
Ye, H., Ding, Y., Li, J., & Ng, H. T. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.460 · Cited by 4
SQuAD-SRC: A Dataset for Multi-Accent Spoken Reading Comprehension
Tang, Y., & Tung, A. K. H. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/578 · Cited by 4
“Trischler et al. , 2017; Yang et al. , 2018] and visual QA datasets [Goyal et al. , 2019; Hudson and Manning, 2019] have provoked the development of high-performance QA models.”
BackgroundXu, F., Liu, J., Lin, Q., Zhao, T., Zhang, J., & Zhang, L. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3317254 · Cited by 6
“Also, some domain-specific datasets like NewsQA [37], TextbookQA [38], Pub-MedQA [39], aim at the area of journalism, education and biology, respectively.”
Robust Machine Reading Comprehension by Learning Soft labels
Zhao, Z., Wu, S., Yang, M., Chen, K., & Zhao, T. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.248 · Cited by 4
“SQuAD 2.0 and NewsQA both contain unanswerable questions, while QUOREF doesn’t.”
MethodInfluentialComQA: Compositional Question Answering via Hierarchical Graph Neural Networks
Wang, B., Yao, T., Chen, W., Xu, J., & Wang, X. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449993 · Cited by 5
Posthoc Verification and the Fallibility of the Ground Truth
Ding, Y., Botzer, N., & Weninger, T. (2022) · doi:10.18653/v1/2022.dadc-1.3 · Cited by 6
Yiğit, G., & Amasyalı, M. F. (2021) · Bilişim Teknolojileri Dergisi · doi:10.17671/gazibtd.810362 · Cited by 4
Intelligence Is Asking The Right Question: A Study On Japanese Question Generation
Nio, L., & Murakami, K. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639526 · Cited by 4
“The recent growing interest on reading comprehension tasks has led some researchers to build some base-line datasets such as Stanford Question Answering Dataset (SQuAD) [6], MS MARCO [7], and NewsQA [24].”
BackgroundTT-Net: Topic Transfer-Based Neural Network for Conversational Reading Comprehension
Gu, Y., Gui, X., & Li, D. (2019) · IEEE Access · doi:10.1109/access.2019.2936011 · Cited by 4
Representing Words in Vector Space and Beyond
Wang, B., Buccio, E. D., & Melucci, M. (2019) · STEAM-H: Science, Technology, Engineering, Agriculture, Mathematics & Health · doi:10.1007/978-3-030-25913-6_5 · Cited by 4
“…Question answering for insurance IBM Watson ASRU 2015 Wiki QA [123] Open-domain question answering MS Narrative QA [53] Reading Comprehension DeepMind TACL 2018 SQuAD 1.0 [85] Questions for machine comprehension Standford MS Marco [76] Human-generated machine reading MS. NIPS 2016 NewsQA [107, 108]”
BackgroundInfluentialSemantics-Aware Inferential Network for Natural Language Understanding
Zhang, S., Zhao, H., Zhou, J., Zhou, X., & Zhou, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i16.17697 · Cited by 4
Using Natural Language Relations between Answer Choices for Machine ComprehensionPreprint
Pujari, R., & Goldwasser, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.15837 · Cited by 4
“Trischler et al. (2016b), Kumar et al. (2016), Liu and Perez (2017), Min et al. (2018) and Xiong et al. (2016) proposed successful models on those datasets.”
BackgroundAmazonQA: A Review-Based Question Answering TaskPreprint
Gupta, M., Kulkarni, N., Chanda, R., Rayasam, A., & Lipton, Z. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.04364 · Cited by 31
“NewsQA [Trischler et al., 2017] Crowd-Sourced CNN Span of words 100K 10K SearchQA [Dunn et al.”
Modelling Latent Skills for Multitask Language GenerationPreprint
Cao, K., & Yogatama, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09543 · Cited by 3
“• NewsQA news article QA (Trischler et al., 2017).”
ReCO: A Large Scale Chinese Reading Comprehension Dataset on OpinionPreprint
BingningWang, Yao, T., Zhang, Q., Xu, J., & Wang, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12146 · Cited by 4
“NewsQA (Trischler et al. 2017) is based on CNN/Daily Mail, the answers and questions are generated by different people to solicit exploratory questions that require reasoning.”
MethodImproving Factual Consistency of Abstractive Summarization via Question AnsweringPreprint
Nan, F., dos Santos, C. N., Zhu, H., Ng, P., McKeown, K. R., Nallapati, R. M., Zhang, D., Wang, Z., Arnold, A. O., & Xiang, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.04623 · Cited by 4
“Our QAGen model in Q UALS is also a BART-large model fine-tuned on the SQ UAD (Rajpurkar et al., 2016) and NewsQA (Trischler et al., 2017) datasets.”
MethodWhy Machine Reading Comprehension Models Learn Shortcuts?Preprint
Lai, Y., Zhang, C., Feng, Y., Huang, Q., & Zhao, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01024 · Cited by 4
“SpM-Para QWM-Para In this work, we focus on paraphrasing ( Para ) as the complex reasoning challenge, since it widely exists in many recent MRC datasets (Trischler et al., 2017; Reddy et al., 2019; Clark et al., 2019).”
Bian, N., Han, X., Chen, B., Lin, H., He, B., & Sun, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.08582 · Cited by 5
COVIDRead: A Large-scale Question Answering Dataset on COVID-19Preprint
Saikh, T., Sahoo, S. K., Ekbal, A., & Bhattacharyya, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.09321 · Cited by 6
“The studies like [10, 11] have suggested that noun phrases of a particular document are the plausible answer of that particular document.”
Improving Controllability of Educational Question Generation by Keyword ProvisionPreprint
Chan, Y.-H., Chung, H.-L., & Fan, Y.-C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.01012 · Cited by 4
“2016) or NewsQA (Trischler et al. 2016)), and therefore generate factoid questions, which are too simple for advanced reading practice assessment.”
BackgroundMethodLite Unified Modeling for Discriminative Reading Comprehension
Zhao, Y., Zhao, H., Shen, L., & Zhao, Y. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.594 · Cited by 4
Liu, L., Wu, Y., Yin, L., Ren, J., Song, R., & Xu, G. (2022) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc55868.2022.00039 · Cited by 3
“And this method achieves SOTA level on NewsQA [13], HotpotQA [14] and 20NewsGroups [15].”
MethodHsu, C.-J., Chung, H.-L., Lee, H.-Y., & Tsao, Y. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096189 · Cited by 4
“The test set is derived from SQuAD, NEWSQA [25] and QuAC [26], and the speech is read by human.”
MethodEnd-to-End Quiz-Style Question Generation for Educational Purposes
Mohammed, M. A., Mohamed, M. L., Bahai, A., Borhamy, M. A., Abd-Elhassib, M. H., Saad, S., Awad, M. A., & Fath-Allah, A. S. (2023) · International Conference on Intelligent Computing and Information Systems (ICICIS) · doi:10.1109/icicis58388.2023.10391138 · Cited by 3
“• BoolQ [13] is a dataset designed for answering yes/no questions and comprises 15,942 examples.”
BackgroundWeight Averaging and re-adjustment ensemble for QRCD
Aftab, E., & Malik, M. K. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.102037 · Cited by 3
TibetanQA2.0: Dataset with Unanswerable Questions for Tibetan Machine Reading Comprehension
Dan, Z., & Sun, Y. (2024) · Data Intelligence · doi:10.3724/2096-7004.di.2024.0029 · Cited by 3
Investigating legal question generation using large language models
Deroy, A., Ghosh, K., & Ghosh, S. (2025) · Artificial Intelligence and Law · doi:10.1007/s10506-025-09452-y · Cited by 5
A Near-Real-Time Answer Discovery for Open-Domain With Unanswerable Questions From the Web
Kim, M., Lee, S., Oh, Y., Choi, H., & Kim, W. (2020) · IEEE Access · doi:10.1109/access.2020.3020245 · Cited by 3
“However, previous studies primarily target cases in which the correct answer always exists in the document, such as SQuAD, NewsQA [16], and MCTest [17].”
BackgroundSummary-Oriented Question Generation for Informational Queries
Yin, X., Zhou, L., Small, K., & May, J. (2021) · doi:10.18653/v1/2021.dialdoc-1.11 · Cited by 4
“…recent empirical improvements due to several factors including novel neural architectures (Chen and Yih, 2020), access to pre-trained contextualized embeddings (Devlin et al., 2019), and the development of large QA training corpora (Rajpurkar et al., 2016; Trischler et al., 2017; Yu et al., 2020).”
BackgroundMethodPlug-and-Play Module for Commonsense Reasoning in Machine Reading Comprehension
Dai, D., Zheng, H., Sui, Z., & Chang, B. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17189-5_3 · Cited by 3
Evaluating NewsQA Dataset With ALBERT
Maqsood, I. (2022) · International Conference on Emerging Technologies · doi:10.1109/icet56601.2022.10004665 · Cited by 3
“[11] Implemented two models on NewsQA dataset.”
MethodInfluentialMuHeQA: Zero-shot question answering over multiple and heterogeneous knowledge bases
Badenes-Olmedo, C., & Corcho, Ó. (2023) · Semantic Web · doi:10.3233/sw-233379 · Cited by 3
“The News dataset contains 5,126 human annotated QA pairs based on 10,000 news articles selected from the NewsQA dataset [41].”
MethodDua, D., Strubell, E., Singh, S., & Verga, P. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.807 · Cited by 6
Efficiently Tuned Parameters Are Task Embeddings
Zhou, W., Xu, C., & McAuley, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.334 · Cited by 8
Cascading Biases: Investigating the Effect of Heuristic Annotation Strategies on Data and Models
Malaviya, C., Bhatia, S., & Yatskar, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.438 · Cited by 3
“World Knowledge” in Multiple Choice Reading Comprehension
Liusie, A., Raina, V., & Gales, M. (2023) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/2023.fever-1.5 · Cited by 3
Question Generation for English Reading Comprehension Exercises using Transformers
Maas, A., Yamada, K. D., Nagahama, T., Kawada, T., & Horita, T. (2024) · IIAI Letters on Informatics and Interdisciplinary Research · doi:10.52731/liir.v005.183 · Cited by 4
Regularization of Distinct Strategies for Unsupervised Question Generation
Kang, J., Hong, G., Roman, H. P. S., & Myaeng, S.-H. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.293 · Cited by 3
“Evaluation The in-domain MRQA shared task (Fisch et al., 2019) 5 including modified SQuAD 1.1 (Rajpurkar et al., 2016), NewsQA (Trischler et al., 2017), TriviaQA (Joshi et al., 2017), SearchQA (Dunn et al., 2017), HotpotQA (Yang et al., 2018), and Natural Questions (Kwiatkowski et al., 2019) is used…”
MethodInfluentialQuesBELM: A BERT based Ensemble Language Model for Natural Questions
Pranesh, R. R., Shekhar, A., & Pallavi, S. (2020) · International Conference on Computing, Communication and Security (ICCCS) · doi:10.1109/icccs49678.2020.9277176 · Cited by 3
MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension
Luo, H., Shi, Y., Gong, M., Shou, L., & Li, T. (2020) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2020.aacl-main.69 · Cited by 3
Khosla, S., Lovelace, J., Dutt, R., & Pratapa, A. (2021) · doi:10.18653/v1/2021.dialdoc-1.13 · Cited by 3
“These DESC type questions are more prevelant in SQuAD, NewsQA, and NQ, which might explain their efficacy.”
InfluentialLong-Tailed Question Answering in an Open World
Dai, Y. Q., Lang, H., Zheng, Y., Huang, F., & Li, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.351 · Cited by 6
Sliding-Bert: Striding Towards Conversational Machine Comprehension in Long Contex
Zhang, L., Wang, W., Yu, K., Huang, J., Lyu, Q., Xue, H., & Hetang, C. (2023) · Advances in Artificial Intelligence and Machine Learning · doi:10.54364/aaiml.2023.1178 · Cited by 4
“Earlier QA datasets, such as NewsQA [11], HotpotQA [12], and SQuAD [1], are largely comprised of a corpus of contextual information (passages) with single question-answer pairs, where reference answers are usually a span of contiguous text within the provided context.”
MethodWho framed Roger Rabbit? Multiple choice questions answering about movie plot
Dzendzik, D., Vogel, C. M., & Liu, Q. (2017) · Arrow@dit (Dublin Institute of Technology) · Cited by 3
Momentum Contrastive Pre-training for Question Answering
Hu, M., Li, M., Wang, Y., & King, I. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.291 · Cited by 3
“Specifically, TriviaQA (Joshi et al., 2017) and NewsQA (Trischler et al., 2017) are out-of-domain datasets from MRQA 2019 Shared Task (Fisch et al., 2019).”
BackgroundMethodInfluentialQuestion Answering for Privacy Policies: Combining Computational and Legal PerspectivesPreprint
Ravichander, A., Black, A. W., Wilson, S., Norton, T. B., & Sadeh, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00841 · Cited by 51
“Thus, setting 5 million installs as a threshold, we ensure each category includes applications with installs on both sides NarrativeQA (Koˇcisk`y et al., 2018) Fiction InsuranceQA (Feng et al., 2015) Insurance TriviaQA (Joshi et al., 2017) Wikipedia SQuAD 1.0 (Rajpurkar et al., 2016) Wikipedia SQuAD 2.0 (Rajpurkar et…”
InfluentialForeshadowing the Benefits of Incidental SupervisionPreprint
He, H., Zhang, M., Qiang, N., & Roth, D. (2020) · arXiv (Cornell University) · Cited by 3
Shinoda, K., Sugawara, S., & Aizawa, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.03238 · Cited by 3
“Machine reading comprehension has gained signifi-cant attention in the NLP community, whose goal is to devise systems that can answer questions about given documents (Rajpurkar et al., 2016; Trischler et al., 2017; Joshi et al., 2017).”
BackgroundMethodHow Optimal is Greedy Decoding for Extractive Question Answering?Preprint
Castel, O., Ram, O., Efrat, A., & Levy, O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.05857 · Cited by 5
“…(Ram et al., 2021), created by subsampling 8 datasets from the MRQA 2019 shared task (Fisch et al., 2019): SQuAD (Rajpurkar et al., 2016), NewsQA (Trischler et al., 2017), TriviaQA (Joshi et al., 2017), SearchQA (Dunn et al., 2017), HotpotQA (Yang et al., 2018), Natural Questions (Kwiatkowski et…”
MethodInfluentialProceedings of the First Workshop on Commonsense Inference in Natural Language Processing
(2019) · doi:10.18653/v1/d19-60 · Cited by 5
“NewsQA (Trischler et al., 2017) provides news texts with crowdsourced questions and answers, which are spans of the evidence documents.”
Multilingual approaches to extractive question answering in political texts
Alsarra, S., Alrashoud, M. R., Osorio, J., D'Orazio, V. J., Khan, L. R., & Brandt, P. T. (2025) · Computational and Mathematical Organization Theory · doi:10.1007/s10588-025-09408-2 · Cited by 3
“The study uses two QA datasets for English: NewsQA (Trischler et al. 2017) and SQuAD v1.”
Generating Instructive Questions from Multiple Articles to Guide Reading in E-Bibliotherapy
Xin, Y., Cao, L., Wang, X., He, X., & Feng, L. (2021) · Sensors · doi:10.3390/s21093223 · Cited by 2
Metadata
- Authors
- Trischler, A., Wang, T., Yuan, X., Harris, J., Sordoni, A., Bachman, P., & Suleman, K.
- Venue
- Proceedings of the 2nd Workshop on Representation Learning for NLP
- Published
- 2017
- Pages
- 191-200
- OpenAlex
- W2557764419
- Cited by
- 706
- References
- 24
- Citations by year
- 2026: 18 · 2025: 26 · 2024: 39 · 2023: 96 · 2022: 110 · 2021: 134 · 2020: 138 · 2019: 87 · 2018: 41 · 2017: 16
- FWCI
- 35.17
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Workshop on Representation Learning for NLP (RepL4NLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- machine reading comprehension, question answering, news articles, exploratory questions, reasoning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
