The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
The NarrativeQA Reading Comprehension Challenge
Open accessAbstract
Reading comprehension (RC)—in contrast to information retrieval—requires integrating information and reasoning about events, entities, and their relations across a full document. Question answering is conventionally used to assess RC ability, in both artificial agents and children learning to read. However, existing RC datasets and tasks are dominated by questions that can be solved by selecting answers using superficial information (e.g., local context similarity or global term frequency); they thus fail to test for the essential integrative aspect of RC. To encourage progress on deeper comprehension of language, we present a new dataset and set of tasks in which the reader must answer questions about stories by reading entire books or movie scripts. These tasks are designed so that successfully answering their questions requires understanding the underlying narrative rather than relying on shallow pattern matching or salience. We show that although humans solve the tasks easily, standard RC models struggle on the tasks presented here. We provide an analysis of the dataset and the challenges it presents.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“NarrativeQA tests deep comprehension and synthesis of full-length literary texts and movie scripts.[2]”
From When Context Overwhelms: A Study of Long-Context vs. Retrieval-Based QA Under Noise (2026) · cites this paper as [2]
In the citing paper
“because narrative comprehension requires preserving global document structure , character arcs, temporal sequences, thematic callbacks , that chunk-level retrieval systematically destroys.[2]”
From When Context Overwhelms: A Study of Long-Context vs. Retrieval-Based QA Under Noise (2026) · cites this paper as [2]
In the citing paper
“The type of questions to be generated range from the typical short answer span questions to multiple spans for question generation from the text [42, 47, 106, 107, 112].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [107]
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 1×
“8 We consider three similarity metrics for extracting spans: Bleu-1, Rouge-L, and the cosine similarity between bag-of-words embedding of the query and the candidate span using pre-trained GloVe word embeddings (Pennington et al., 2014).”
Method(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 2×
“neural models out of the box. We further discuss the challenges and possible approaches in the following sections. We require the use of metrics for generated text. We evaluate using Bleu-1, Bleu-4 (Papineni et al., 2002), Meteor (Denkowski and Lavie, 2011), and Rouge-L (Lin, 2004), using two references for each quest…”
MethodSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 2×
“ndidate span using pre-trained GloVe word embeddings (Pennington et al., 2014). 4.2 Neural Benchmarks As a first benchmark we consider a simple bidirectional LSTM sequence to sequence (Seq2Seq) model (Sutskever et al., 2014) predicting the answer directly from the query. Importantly, we provide no context information f…”
MethodInfluential(2004) · Cited by 9,718
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Scripts, Plans, Goals, and Understanding: An Inquiry into Human Knowledge Structures
Hunt, E. B., Schank, R. C., & Abelson, R. P. (1979) · The American Journal of Psychology · doi:10.2307/1421499 · Cited by 2,938
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,855 · cited 9×
“The tasks associated with these datasets are all selecting an answer from a set of options, which is explicitly provided for CBT and BookTest, and is implicit for CNN/Daily Mail, as the answers are always entities from the document.”
BackgroundMethodInfluentialSelf-citationMS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Bidirectional Attention Flow for Machine ComprehensionPreprint
Seo, M. J., Kembhavi, A., Farhadi, A., & Hajishirzi, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01603 · Cited by 1,698 · cited 5×
“Simple models scoring and/or extracting candidate spans conditioned on the question and superficial signal from the rest of the document do well (Seo et al., 2016, e.g.).”
BackgroundMethodInfluential
Flagged first, then by citations
Research works printed but not linked: 1
- Blei David M., 2003, Journal of Machine Learning Research, 3, 993
Cited by
Sorted by citations
Items 301–400 of 615
Causal Narrative Comprehension: A New Perspective for Emotion Cause Extraction
Cao, W., Zhang, K., Ruan, S., Tao, H., Zhao, S., Wang, H., Liu, Q., & Chen, E. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3206960 · Cited by 6
“[65], [66], reading comprehension [67], [68], [69], and so on.”
BackgroundTopic-aware multi-hop machine reading comprehension using weighted graphs
Mohammadi, A., Ramezani, R., & Baraani-Dastjerdi, A. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.119873 · Cited by 8
The DistilBERT Model: A Promising Approach to Improve Machine Reading Comprehension Models
P, S. V., Shahapure, N. H., Rekha, P. M., Nethravathi, B., Khandelwal, P., Anand, A., Agrawal, P., & Srivastava, V. (2023) · International Journal on Recent and Innovation Trends in Computing and Communication · doi:10.17762/ijritcc.v11i8.7957 · Cited by 9
MPrompt: Exploring Multi-level Prompt Tuning for Machine Reading Comprehension
Chen, G., Qian, Y., Wang, B., & Li, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.343 · Cited by 7
Arabic Narrative Question Answering (QA) Using Transformer Models
Ateeq, M. A., Tiun, S. B., Abdelhaq, H., & Rahhal, N. (2023) · IEEE Access · doi:10.1109/access.2023.3348410 · Cited by 8
“This involves employing transfer learning techniques and fine-tuning these pre-trained models on the NarrativeQA dataset [21].”
BackgroundMethodInfluentialKotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0005 · Cited by 6
Leveraging Structured Information from a Passage to Generate Questions
Xu, J., Sun, Y., Gan, J., Zhou, M., & Wu, D. (2022) · Tsinghua Science & Technology · doi:10.26599/tst.2022.9010034 · Cited by 6
Asking the Right Question: Inferring Advice-Seeking Intentions from Personal Narratives
Fu, L., Chang, J. P., & Danescu-Niculescu-Mizil, C. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1052 · Cited by 9
“This type of task can be set up in various forms, as the system can be asked to summarize the story (summarization, see Nenkova (2011); Allah-yari et al. (2017) for surveys), generate a question that is answerable from the text (question generation (Du et al., 2017)), or answer a question for which the information can…”
BackgroundMulti-span Style Extraction for Generative Reading ComprehensionPreprint
Yang, J., Zhang, Z., & Zhao, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.07382 · Cited by 11
“NarrativeQA (Koisk et al., 2018) is the first large-scale question-answering dataset on full-length books and movie scripts, requiring understanding the underlying narrative rather than relying on shallow pattern matching or salience.”
BackgroundGrid Search Hyperparameter Benchmarking of BERT, ALBERT, and LongFormer on DuoRCPreprint
Quijano, A. J., Nguyen, S., & Ordoñez, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.06326 · Cited by 10
“Other similar datasets to DuoRC are the MovieQA [17], NarrativeQA [8], and NewsQA [18].”
MethodComplex QA and language models hybrid architectures, SurveyPreprint
Daull, X., Bellot, P., Bruno, E., Martin, V., & Murisasco, É. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.09051 · Cited by 32
A Simple and Effective $L_2$ Norm-Based Strategy for KV Cache Compression
Devoto, A., Zhao, Y., Scardapane, S., & Minervini, P. (2024) · arXiv · doi:10.48550/arxiv.2406.11430 · Cited by 6
“Lower L 2 norm appears to correspond to embeddings that drive disproportionately high attention values due to their alignment with a common "sink" direction.”
Background(2025) · doi:10.1145/3833415 · Cited by 18
“…QA HotpotQA [261] 2WikiMultiHopQA [89] StrategyQA [75] MuSiQue [231] MultiHop-RAG [227] Long-form QA ELI5 [62] WebCPM [192] NarrativeQA (NQA) [118] Information Extraction Entity Linking ZESHEL [150] CoNLL [90] Relation Extraction T-REx [59] ZsRE [125] Text Understanding Text Classification…”
Adversarial Entity Graph Convolutional Networks for multi-hop inference question answering
Du, Y., Yan, R., Hou, Y., Pei, Y., & Han, H. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125098 · Cited by 6
m2LLM: A Multi-Dimensional Optimization Framework for LLM Inference on Mobile Devices
Liu, K., Zhou, X., & Li, L. (2025) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2025.3587445 · Cited by 7
“On the NarrativeQA [63] dataset, we evaluate the differences between the compressed and original prompts using BLEU [52], METEOR [45], ROUGE [44], and BERTScore [53].”
User-defined trade-offs in LLM benchmarking: balancing accuracy, scale, and sustainability
Gjorgjevikj, A., Nikolikj, A., Seljak, B. K., & Eftimov, T. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114405 · Cited by 5
Examining the Feasibility of AI-Generated Questions in Educational Settings
Zeghouani, O., Ali, Z., van Dijkhuizen, W. S., Hong, J., & Clos, J. (2024) · International Symposium on Trustworthy Autonomous Systems (TAS) · doi:10.1145/3686038.3686652 · Cited by 5
“To solve this, we created our own training dataset composed of SQuAD, AdverserialQA, NarrativeQA and SciQ [1, 12, 33, 37].”
InfluentialLearning and Evaluating Character Representations in Novels
Inoue, N., Pethe, C., Kim, A., & Skiena, S. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.81 · Cited by 9
“We extract character-related questions (Angelidis et al., 2019) from NarrativeQA (Koˇciský et al., 2017).”
MethodTowards Diverse and Effective Question-Answer Pair Generation from Children Storybooks
Eo, S., Moon, H., Kim, J., Hur, Y., Kim, J., Lee, S., Chun, C., Park, S. S., & Lim, H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.380 · Cited by 6
Cai, S., Ma, Q., Hou, Y., & Zeng, G. (2024) · Electronics · doi:10.3390/electronics13081436 · Cited by 6
Prompt Learning for Few-Shot Question Answering via Self-Context Data Augmentation
Qiu, J.-Q., Zhang, C.-Y., & Chen, C. L. P. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3483201 · Cited by 5
“QA is one of the most challenging tasks in NLP. Based on the formats of the questions and answers, QA tasks can be categorized into four types [22]: cloze style [23], multiple choice [24], span prediction [1], and free-form answer [25].”
Bend but Don’t Break? Multi-Challenge Stress Test for QA Models
Pugaliya, H., Route, J., Ma, K., Geng, Y., & Nyberg, E. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5818 · Cited by 5
“This model has achieved promising performance on the NarrativeQA (Kocisky et al., 2018) and WikiHop (Welbl”
MethodA Densely Connected Encoder Stack Approach for Multi-type Legal Machine Reading Comprehension
Nai, P., Li, L., & Tao, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-62008-0_12 · Cited by 5
DramaQA: Character-Centered Video Story Understanding with Hierarchical QA
Choi, S.-H., On, K.-W., Heo, Y., Jang, Y., Seo, A., Lee, S.-C., Lee, M., & Zhang, B. (2021) · 정보과학회 컴퓨팅의 실제 논문지 · doi:10.5626/ktcp.2021.27.1.7 · Cited by 6
Yao, B., Sen, P., Popa, L. C., Hendler, J., & Wang, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.821 · Cited by 7
LittleBird: Efficient Faster & Longer Transformer for Question Answering
Lee, M., Han, K., & Shin, M. C. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.352 · Cited by 7
Chen, T., Li, H., Kasamatsu, M., Utsuro, T., & Kawada, Y. (2020) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/2020.fever-1.4 · Cited by 5
“Another type of non-factoid QA dataset is NarrativeQA (Kočiský et al., 2018) dataset (in En-”
BackgroundIs Killed More Significant than Fled? A Contextual Model for Salient Event Detection
Jindal, D., Deutsch, D., & Roth, D. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.10 · Cited by 7
Shao, Z., Shang, L., Liu, Q., & Huang, M. (2021) · doi:10.18653/v1/2021.acl-long.318 · Cited by 5
Incremental BERT with commonsense representations for multi-choice reading comprehension
Li, R., Wang, L., Jiang, Z., Liu, D., Zhao, M., & Lu, X. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11197-0 · Cited by 7
Asking the Right Question: Inferring Advice-Seeking Intentions from Personal NarrativesPreprint
Fu, L., Chang, J. P., & Danescu-Niculescu-Mizil, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01587 · Cited by 5
“This type of task can be set up in various forms, as the system can be asked to summarize the story (summarization, see Nenkova (2011); Allah-yari et al. (2017) for surveys), generate a question that is answerable from the text (question generation (Du et al., 2017)), or answer a question for which the information can…”
BackgroundThe Shmoop Corpus: A Dataset of Stories with Loosely Aligned SummariesPreprint
Chaudhury, A., Tapaswi, M., Kim, S. W., & Fidler, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.13082 · Cited by 13
“Our Shmoop corpus strikes a balance between short-form large-scale datasets such as the Children’s Book Test and long-form smallscale corpora like NarrativeQA.”
MethodInfluentialWizard of Search Engine: Access to Information Through Conversations with Search EnginesPreprint
Ren, P., Liu, Z., Song, X., Tian, H., Chen, Z., Ren, Z., & de Rijke, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.08301 · Cited by 9
Yao, B., Wang, D., Wu, T., Hoang, T. M., Sun, B., Li, T. J.-J., Yu, M., & Xu, Y. (2021) · arXiv (Cornell University) · Cited by 5
ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference
Liu, X., Tang, Z., Dong, P., Li, Z., Liu, Y., Li, B., Hu, X., & Chu, X. (2025) · arXiv · doi:10.48550/arxiv.2502.00299 · Cited by 4
“This benchmark was created for multi-task evaluation of long-context inputs and includes 17 datasets covering tasks such as single-document QA [87, 88], multi-document QA [89–92], summarization [93–96], few-shot learning [97–99], synthetic tasks and code generation [100, 101].”
Vietnamese Legal Question Answering: An Experimental Study
Ha, N. T., Nguyen, T.-P., Trung, K. T., Le, H.-L., Hương, L. T., Nguyen, C. T., & Nguyen, M.-T. (2024) · International Conference on Knowledge and Systems Engineering/Proceedings · doi:10.1109/kse63888.2024.11063637 · Cited by 5
“After SQuAD, several open-domain English MRC datasets were introduced, such as MS MARCO [4], NewsQA [5], and NarrativeQA [7].”
Auto-Grading Comprehension on Reference-Student Answer Pairs using the Siamese-based Transformer
Sayeed, M. A., Gupta, D., & Kanjirangat, V. (2024) · IEEE International Conference for Convergence in Technology (I2CT) · doi:10.1109/i2ct61223.2024.10543346 · Cited by 4
Wang, H., Hu, K., & Gao, L. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887668 · Cited by 4
“While QA systems have greatly diversified in terms of content input, ranging from short-text snippets [4] [5] [6] to long documents [7], and from purely textual data to mixed media such as images (VQA [8], ChartQA [9], TextVQA [10]), document images (DocVQA [11]) and videos (VideoQA [1]), current…”
Hierarchical Encoders for Modeling and Interpreting Screenplays
Bhat, G., Saluja, A., Dye, M., & Florjanczyk, J. (2021) · doi:10.18653/v1/2021.nuse-1.1 · Cited by 4
Syntax-Aware Multi-Spans Generation for Reading Comprehension
Zhang, Z., Zhang, Y., & Zhao, H. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3138679 · Cited by 10
Comparing Neural Question Generation Architectures for Reading Comprehension
Perkoff, E. M., Bhattacharyya, A., Cai, J., & Cao, J. (2023) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/2023.bea-1.47 · Cited by 4
“They discovered that fine-tuning on the FairytaleQA dataset outperforms the BARTQA model fine-tuned on the NarrativeQA and FairytaleQA (Kočiský et al., 2018) corpora.”
BackgroundMethodThe Computational Evolution of Cognitive Architectures
Kotseruba, I., & Tsotsos, J. K. (2025) · doi:10.1093/oso/9780192844835.001.0001 · Cited by 4
Arabic Reading Comprehension Benchmarks Created Semiautomatically
Biltawi, M. M., Awajan, A., & Tedmori, S. (2020) · International Arab Conference on Information Technology (ACIT) · doi:10.1109/acit50332.2020.9300111 · Cited by 4
“Examples on free-form datasets created semi-automatically include; TriviaQA [7], SearchQA [8], and NarrativeQA [9].”
MethodSurvey of Machine Reading Comprehension Models and its Evaluation Metrics
Therasa, M., & Mathivanan, G. (2022) · International Conference on Computing Methodologies and Communication (ICCMC) · doi:10.1109/iccmc53470.2022.9754070 · Cited by 4
Few-shot Adaptation Works with UnpredicTable Data
Chan, J. S., Pieler, M., Jao, J., Scheurer, J., & Perez, E. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.102 · Cited by 7
Araujo, V., Soto, Á., & Moens, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.830 · Cited by 4
Investigating better context representations for generative question answering
Francis, S., & Moens, M. (2023) · Information Retrieval · doi:10.1007/s10791-023-09420-7 · Cited by 5
“Table 8 shows the NarrativeQA test set results in terms of ROUGE-L and BLEU-1.”
BackgroundInfluentialIn-Context Demonstration Selection with Cross Entropy Difference
Iter, D., Pryzant, R., Xu, R., Wang, S., Liu, Y., Yi‐chong, X., & Zhu, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.81 · Cited by 13
Context-Aware Question-Answer for Interactive Media Experiences
Jorgensen, K., Zhao, Z., Wang, H., Wang, M., & He, Z. (2021) · ACM International Conference on Interactive Media Experiences · doi:10.1145/3452918.3458795 · Cited by 5
“The MovieQa dataset [30] was introduced to evaluate automatic story comprehension (a multiple choice test), NarrativeQA datasets [12] were released to investigate abstract reasoning across an entire document for answering questions, and the BoolQ Dataset [4] to test the ability of a model to answer yes or no questions.”
MethodTT-Net: Topic Transfer-Based Neural Network for Conversational Reading Comprehension
Gu, Y., Gui, X., & Li, D. (2019) · IEEE Access · doi:10.1109/access.2019.2936011 · Cited by 4
An Efficient Machine Reading Comprehension Method Based on Attention Mechanism
Jin, W., Yang, G., & Zhu, H. C. (2019) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa-bdcloud-sustaincom-socialcom48970.2019.00186 · Cited by 4
“Deepmind released the NarrativeQA [6] in 2017.”
Weakly-Supervised Question Answering with Effective Rank and Weighted Loss over Candidates
Qin, H., Zhu, J., & Shen, B. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449919 · Cited by 4
“Experiments on three datasets, WikiSQL [37] for semantic parsing, NarrativeQA [15] and NaturalQuestions [16] for machine reading comprehension, verify the effectiveness of our method.”
BackgroundMethodInfluentialRepresenting Words in Vector Space and Beyond
Wang, B., Buccio, E. D., & Melucci, M. (2019) · STEAM-H: Science, Technology, Engineering, Agriculture, Mathematics & Health · doi:10.1007/978-3-030-25913-6_5 · Cited by 4
“QA [32] Question answering for insurance IBM Watson ASRU 2015 Wiki QA [123] Open-domain question answering MS Narrative QA [53] Reading Comprehension DeepMind TACL 2018 SQuAD 1.0 [85] Questions for machine comprehension Standford MS Marco [76] Human-generated machine reading MS. NIPS 2016 NewsQA…”
BackgroundInfluentialSemantics-Aware Inferential Network for Natural Language Understanding
Zhang, S., Zhao, H., Zhou, J., Zhou, X., & Zhou, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i16.17697 · Cited by 4
FastFusionNet: New State-of-the-Art for DAWNBench SQuADPreprint
Wu, F., Li, B., Wang, L., Lao, N., Blitzer, J. C., & Weinberger, K. Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.11291 · Cited by 5
“An array of reading comprehension datasets have been released in the past years including WikiQA [37], SQuAD [25], SQuAD v2 [26], TriviaQA [13], SearchQA [8], NarrativeQA [17], CoQA [27], QuAC [4], and Natural Questions [18].”
A Knowledge Hunting Framework for Common Sense ReasoningPreprint
Emami, A., Cruz, N. D. L., Trischler, A., Suleman, K., & Cheung, J. C. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.01375 · Cited by 4
“, 2016), the Narrative cloze task (Taylor, 1953), or its more difficult counterpart, the NarrativeQA Reading Comprehension Challenge (Kočiskỳ et al., 2017).”
BackgroundAmazonQA: A Review-Based Question Answering TaskPreprint
Gupta, M., Kulkarni, N., Chanda, R., Rayasam, A., & Lipton, Z. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.04364 · Cited by 31
The TechQA DatasetPreprint
Castelli, V., Chakravarti, R., Dana, S., Ferritto, A., Florian, R., Franz, M., Garg, D., Khandelwal, D., McCarley, S., McCawley, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.02984 · Cited by 26
ReCO: A Large Scale Chinese Reading Comprehension Dataset on OpinionPreprint
BingningWang, Yao, T., Zhang, Q., Xu, J., & Wang, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12146 · Cited by 4
“In NarrativeQA (Kociský et al. 2018) or NaturalQuestions (Kwiatkowski et al.”
BackgroundInfluentialLite Unified Modeling for Discriminative Reading Comprehension
Zhao, Y., Zhao, H., Shen, L., & Zhao, Y. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.594 · Cited by 4
MuseumQA: A Fine-Grained Question Answering Dataset for Museums and Artifacts
Jin, F., Chang, Q., & Xu, Z. (2023) · International Conference on Machine Learning and Natural Language Processing (MLNLP) · doi:10.1145/3639479.3639525 · Cited by 4
Towards a benchmark dataset for large language models in the context of process automation
Tizaoui, T., & Tan, R. (2024) · Digital Chemical Engineering · doi:10.1016/j.dche.2024.100186 · Cited by 4
CriSPO: Multi-Aspect Critique-Suggestion-guided Automatic Prompt Optimization for Text Generation
He, H., Liu, Q., Xu, L., Shivade, C., Zhang, Y., Srinivasan, S., & Kirchhoff, K. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i22.34575 · Cited by 8
An LSTM model for cloze-style machine comprehension
Wang, S., & Jiang, J. (2018) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 3
Contrasting Human Opinion of Non-factoid Question Answering with Automatic Evaluation
Ji, T., Graham, Y., & Jones, G. J. F. (2020) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3343413.3377996 · Cited by 3
“We employ NarrativeQA dataset as test data which was built to encourage deeper comprehension of language [15].”
MethodANALYSIS OF READING COMPREHENSION FINAL TEST AT ENGLISH DEPARTMENT OF MUHAMMADIYAH ACEH UNIVERSITY
Nisa, R., & Helmanda, C. M. (2020) · Getsempena English Education Journal · doi:10.46244/geej.v7i1.987 · Cited by 3
Aksu, T., Liu, Z., Kan, M., & Chen, N. F. (2021) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2021.sigdial-1.14 · Cited by 3
“…comprehension are largely limited to written text, e.g. , SQuAD (Rajpurkar et al., 2018b), MARCO (Nguyen et al., 2016), RACE (Lai et al., 2017), TriviaQA (Joshi et al., 2017) and many others (Hermann et al., 2015; Hill et al., 2016; Richardson et al., 2013; Kocisk´y et al., 2017; He et al., 2018).”
BackgroundLi, J., Yu, M., Meng, F., Lin, Z., Fu, P., Wang, W., & Zhou, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.306 · Cited by 3
Narrative Why-Question Answering: A Review of Challenges and Datasets
Kalbaliyev, E., & Sirts, K. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.48 · Cited by 3
“Typically, in order to reason about long narratives, the parts relevant to reasoning are retrieved first (Koˇciský et al., 2018; Tay et al., 2019; Frermann, 2019; Mou et al., 2020, 2021).”
BackgroundShahriar, M., Chowdhury, A. A. F., Ehsan, M. A., & Kamal, A. R. M. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-main.29 · Cited by 3
Answer Relevance Score (ARS): Novel Evaluation Metric for Question Answering System
Gupta, T., & Kumar, E. (2023) · International Conference on Advances in Computation, Communication and Information Technology (ICAICCIT) · doi:10.1109/icaiccit60255.2023.10466080 · Cited by 3
“Furthermore, a small comparative study by Anthony et al. [3] on NarrativeQA, SemEval, and ROPES datasets [12, 13, 14] concluded that METEOR [15] performs better than ROUGE, BERTScore, and variants of BLEU metrics.”
MethodHow Natural Language Processing Enables AIGC Recognition? --Latest Trends and Future Prospects
Wang, W., & Qiao, H. (2024) · International Conference on Software Engineering and Information Management (ICSIM) · doi:10.1145/3647722.3647738 · Cited by 3
Zhu, E., Yu, Z., Huang, Y., Gao, S., & Xian, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128639 · Cited by 3
An Effective Retrieval Method to Improve RAG Performance
Mengmeng, S., Zhibin, L., Qingwei, W., Man, H., & Feiyang, X. (2024) · International Conference on Data Science and Information Technology (DSIT) · doi:10.1109/dsit61374.2024.10881380 · Cited by 7
Descriptor: Open-Domain Long-Form Context-Aware Question-Answering Dataset (DragonVerseQA)
Lahiri, A. K., & Hu, Q. (2025) · IEEE data descriptions. · doi:10.1109/ieeedata.2025.3562173 · Cited by 3
Harry Potter and the Action Prediction Challenge from Natural Language
Vilares, D., & Gómez‐Rodríguez, C. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1218 · Cited by 7
“Kocisky et al. (2018) introduce NarrativeQA, where answering the questions requires to process entire stories.”
BackgroundZero-Shot Clinical Questionnaire Filling From Human-Machine Interactions
Toudeshki, F. G., Jolivet, P., Durand-Salmon, A., & Liednikova, A. (2021) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/2021.mrqa-1.5 · Cited by 3
Four-way Bidirectional Attention for Multiple-choice Reading Comprehension
Hu, L., Zou, D., Guo, X., Qi, L., Tang, Y. G., Song, H., & Yuan, J. (2021) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc52423.2021.9658632 · Cited by 3
“Different from early datasets such as SQuAD [2] and Narrative QA [3], where most of the correct answers are explicitly mentioned in the original context, answers in CosmosQA cannot be found in the context paragraph as a text span but need commonsense reasoning.”
MethodLong-Tailed Question Answering in an Open World
Dai, Y. Q., Lang, H., Zheng, Y., Huang, F., & Li, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.351 · Cited by 6
Personality Understanding of Fictional Characters during Book Reading
Yu, M., Li, J., Yao, S., Pang, W., Zhou, X., Xiao, Z., Meng, F., & Zhou, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.826 · Cited by 13
Multi-VQG: Generating Engaging Questions for Multiple Images
Yeh, M.-H., Chen, V., Huang, T.-H., & Ku, L. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.19 · Cited by 4
Learning Enhancement Using Question-Answer Generation for e-Book Using Contrastive Fine-Tuned T5
Kumar, S., Chauhan, A., & Kumar, C. P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-24094-2_5 · Cited by 3
Question Answering for Privacy Policies: Combining Computational and Legal PerspectivesPreprint
Ravichander, A., Black, A. W., Wilson, S., Norton, T. B., & Sadeh, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00841 · Cited by 51
Easy, Reproducible and Quality-Controlled Data Collection with CrowdaqPreprint
Ning, Q., Wu, H., Dasigi, P., Dua, D., Gardner, M., Logan, R. L., Marasović, A., & Nie, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06694 · Cited by 3
ReadTwice: Reading Very Large Documents with MemoriesPreprint
Zemlyanskiy, Y., Ainslie, J., de Jong, M., Pham, P., Eckstein, I., & Sha, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.04241 · Cited by 3
“NQA asks questions about entire books, requiring a successful QA system to model very long-range dependencies.”
InfluentialPyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling
Cai, Z., Zhang, Y., Gao, B., Liu, Y., Li, Y., Liu, T., Lu, K., Xiong, W., Dong, Y., Hu, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.02069 · Cited by 2
“It includes 17 datasets covering tasks such as single-document QA [20, 6], multi-document QA [33, 16], summarization [17, 35, 10], few-shot learning [21, 13, 19], synthetic, and code generation [14, 24].”
BackgroundMore with Less: ZeroQA and Relevant Subset Selection for AI2 Reasoning Challenge
Pătras, C. C.-B., Pîrtoacă, G.-S., Rebedea, T. E., & Ruşeţi, Ş. (2022) · Procedia Computer Science · doi:10.1016/j.procs.2022.09.334 · Cited by 2
MedicalQA: A dataset of medical domain for machine reading comprehension
Ma, N., Wenrong, L. V., & Zecheng, G. U. O. (2024) · China Scientific Data · doi:10.11922/11-6035.csd.2022.0030.zh · Cited by 2
None of the above: comparing scenarios for answerability detection in question answering systems
Reyes-Montesinos, J., Rodrigo, Á., & Peñas, A. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06765-y · Cited by 3
“One solution is to build benchmarks based on naturally occurring questions such as MS MARCO [ 12], NarrativeQA [14] and Natural Questions [15 ].”
YMCQ: Reasoning-Enhanced MCQ Generation
Dutulescu, A., Ruşeţi, Ş., Iorga, D., Dascălu, M., & McNamara, D. S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98465-5_39 · Cited by 2
Kyoung, M., Lim, J.-H., & Kim, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3617449 · Cited by 2
A collaborative reasoning framework for large language models in long-context Q&A
Yao, J., He, G., & Xu, X. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129960 · Cited by 3
Diverse and high-quality text generation assisted by large language models
Nguyen, T. H., & Luong, N. H. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114954 · Cited by 3
Retrieving on a topic graph for long document question answering
Chen, S., Liang, B., Yu, Y., Wong, K., Wang, H., & Xu, R. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.132447 · Cited by 2
A COMPARATIVE STUDY OF CHUNKING STRATEGIES FOR RETRIEVAL-AUGMENTED GENERATION
Yuvzhenko, D., & Stirenko, S. (2026) · Kibernetyka ta Systemnyi Analiz · doi:10.34229/kca2522-9664.26.3.4 · Cited by 1
Open-Domain Long-Form Question–Answering Using Transformer-Based Pipeline
Dash, A., Awachar, M., Patel, A. M., & Rudra, B. (2023) · SN Computer Science · doi:10.1007/s42979-023-02039-x · Cited by 3
Metadata
- Authors
- Kočiský, T., Schwarz, J., Blunsom, P., Dyer, C., Hermann, K. M., Melis, G., & Grefenstette, E.
- Affiliations
- University of Oxford (GB)
- Google DeepMind (United Kingdom) (GB)
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2018-12
- Volume
- 6
- Pages
- 317-328
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2779681640
- Cited by
- 596
- References
- 31
- Citations by year
- 2026: 51 · 2025: 65 · 2024: 40 · 2023: 77 · 2022: 71 · 2021: 87 · 2020: 80 · 2019: 91 · 2018: 32
- FWCI
- 40.86
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- reading comprehension, narrative comprehension, question answering, screenplays, deep comprehension
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
