Cited by 615Topic ModelingMultimodal Machine Learning ApplicationsNatural Language Processing Techniques

The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.

The NarrativeQA Reading Comprehension Challenge

Open access

Kočiský, Tomáš, Schwarz, Jonathan, Blunsom, Phil, Dyer, Chris, Hermann, Karl Moritz, Melis, Gábor, Grefenstette, Edward·2018·Transactions of the Association for Computational Linguistics

DOI 10.1162/tacl_a_00023

Open this paper in CoChat — free account Open PDF

Abstract

Reading comprehension (RC)—in contrast to information retrieval—requires integrating information and reasoning about events, entities, and their relations across a full document. Question answering is conventionally used to assess RC ability, in both artificial agents and children learning to read. However, existing RC datasets and tasks are dominated by questions that can be solved by selecting answers using superficial information (e.g., local context similarity or global term frequency); they thus fail to test for the essential integrative aspect of RC. To encourage progress on deeper comprehension of language, we present a new dataset and set of tasks in which the reader must answer questions about stories by reading entire books or movie scripts. These tasks are designed so that successfully answering their questions requires understanding the underlying narrative rather than relying on shallow pattern matching or salience. We show that although humans solve the tasks easily, standard RC models struggle on the tasks presented here. We provide an analysis of the dataset and the challenges it presents.

Integrity signals

Registry

Crossref · 10.1162/tacl_a_00023

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 1×
“8 We consider three similarity metrics for extracting spans: Bleu-1, Rouge-L, and the cosine similarity between bag-of-words embedding of the query and the candidate span using pre-trained GloVe word embeddings (Pennington et al., 2014).”
Method
Latent dirichlet allocation
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 2×
“neural models out of the box. We further discuss the challenges and possible approaches in the following sections. We require the use of metrics for generated text. We evaluate using Bleu-1, Bleu-4 (Papineni et al., 2002), Meteor (Denkowski and Lavie, 2011), and Rouge-L (Lin, 2004), using two references for each quest…”
Method
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 2×
“ndidate span using pre-trained GloVe word embeddings (Pennington et al., 2014). 4.2 Neural Benchmarks As a ﬁrst benchmark we consider a simple bidirectional LSTM sequence to sequence (Seq2Seq) model (Sutskever et al., 2014) predicting the answer directly from the query. Importantly, we provide no context information f…”
MethodInfluential
Untitled work
(2004) · Cited by 9,718
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Scripts, Plans, Goals, and Understanding: An Inquiry into Human Knowledge Structures
Hunt, E. B., Schank, R. C., & Abelson, R. P. (1979) · The American Journal of Psychology · doi:10.2307/1421499 · Cited by 2,938
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,855 · cited 9×
“The tasks associated with these datasets are all selecting an answer from a set of options, which is explicitly provided for CBT and BookTest, and is implicit for CNN/Daily Mail, as the answers are always entities from the document.”
BackgroundMethodInfluentialSelf-citation
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Bidirectional Attention Flow for Machine ComprehensionPreprint
Seo, M. J., Kembhavi, A., Farhadi, A., & Hajishirzi, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01603 · Cited by 1,698 · cited 5×
“Simple models scoring and/or extracting candidate spans conditioned on the question and superﬁcial signal from the rest of the document do well (Seo et al., 2016, e.g.).”
BackgroundMethodInfluential

Flagged first, then by citations

Show all 31 references

Research works printed but not linked: 1

Blei David M., 2003, Journal of Machine Learning Research, 3, 993

Sorted by citations

Items 501–600 of 615

Multiway Bidirectional Attention and External Knowledge for Multiple-choice Reading Comprehension
Zou, D., Zhang, X., Song, X., Yu, Y., Yang, Y., & Xi, K. (2022) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53654.2022.9945587
“Different from SquAD [2]and Narrative QA [3], answers in CosmosQA cannot be found directly in the context paragraph, but require commonsense reasoning.”
Method
Developing Amharic Question Answering Model Over Unstructured Data Source Using Deep Learning Approach
Elema, A. M. (2022) · International Conference on Information and Communication Technology for Development for Africa (ICT4DA) · doi:10.1109/ict4da56482.2022.9971413
Entity-Aware Social Media Reading Comprehension
Liu, H., Hong, Y., & Zhu, Q. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20865-2_15
A Framework for Automated Worker Evaluation Based on Free-Text Responses with No Ground TruthPreprint
Geva, T., Goldstein, A., & Yahav, I. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4288950
Answer Annotation for Short-Answer Questions Based on Dynamic Network
洪, 旭. (2022) · Computer Science and Application · doi:10.12677/csa.2022.129225
Improving Student Reading Comprehension Using Narrative Text Question and Answer Method
Kusuma, H., & Aswir, A. (2022) · Jurnal Studi Guru dan Pembelajaran · doi:10.30605/jsgp.5.2.2022.1476
Revisión bibliográfica de la comprensión lectora en personas adultas estudiantes: insumos para la educación distancia en Costa Rica
Poltronieri-Méndez, S. (2022) · Revista Ensayos Pedagógicos · doi:10.15359/rep.17-2.16
ЗАГАЛЬНОДОСТУПНІ НАБОРИ ДАНИХ ТА МЕТРИКИ ДЛЯ СПРИЯННЯ ДОСЛІДЖЕННЯМ СИСТЕМ НАДАННЯ ВІДПОВІДЕЙ
Vyshniak, M. Y., & Pyrozhenko, M. Y. (2023) · Таврійський науковий вісник Серія Технічні науки · doi:10.32851/tnv-tech.2023.1.2
Pupils’ Reading Motivation and Reading Literacy Level
Ilago, M. R. T., & Tantiado, R. C. (2023) · International Journal of Research Publications · doi:10.47119/ijrp1001281720235163
Proceedings of the The 5th Workshop on Narrative Understanding
(2023) · doi:10.18653/v1/2023.wnu-1
Story Settings: A Dataset
Rittichier, K. J. (2023) · Workshop on Narrative Understanding (WNU) · doi:10.18653/v1/2023.wnu-1.12
“The first Narrative Q & A dataset (Koˇciský et al., 2018) was offered in 2018 to evaluate reading comprehension.”
Background
GameQA: Gamified Mobile App Platform for Building Multiple-Domain Question-Answering Datasets
Skarphedinsson, N., Gudmundsson, B., Smari, S., Larusdottir, M. K., Einarsson, H., Khan, A., Nyberg, E., & Loftsson, H. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-demo.18
“CoQa (Reddy et al., 2019), NarrativeQA (Koˇciský et al., 2018), and NewsQA (Trischler et al., 2016) are other examples of datasets falling into this category.”
Background
Multi-hop question answering using sparse graphs
Hemmati, N., & Ghassem-Sani, G. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107128
Few-shot Unified Question Answering: Tuning Models or Prompts?
Bansal, S., Yavuz, S., Pang, B., Bhat, M., & Zhou, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.550 · Cited by 3
PARROT: Zero-Shot Narrative Reading Comprehension via Parallel Reading
Zhao, C., Vijjini, A. R., & Chaturvedi, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.895
NarrativeXL: a Large-scale Dataset for Long-Term Memory Models
Moskvichev, A., & Mai, K.-V. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1005 · Cited by 2
Impacto del plan lector en estudiantes de educación primaria
Moreno-Guerrero, A.-J., Marín, J. A. M., Carmona-Serrano, N., & López-Belmonte, J. (2023) · Innoeduca International Journal of Technology and Educational Innovation · doi:10.24310/innoeduca.2023.v9i2.15175
Multi-Task Learning of Japanese How-to Tip Machine Reading Comprehension by a Generative Model
Wang, X., Li, T., TAMURA, T., NISHIDA, S., & Utsuro, T. (2023) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2023edp7113
The Correlation Among Reading Habits, Reading Motivation, and Reading Comprehension of The Eleventh Grade Students of SMA Negeri 10 Palembang
Aditya, A., Dafa, F., Budiyanto, D., & Ridho, M. R. (2023) · Journal of English Education · doi:10.61994/jee.v1i1.59
“According to Klingner et al. (2007) and Kočiský et al. (2018), reading comprehension is a multi-component, highly complex process that includes numerous interactions between the reader and what they contribute to the text, as well as text-related variables.”
Background
Facilitating Students’ Higher Order Thinking Skill (Hots) Through Three-Phase Reading Activity at Eis Class
Muzammil, S., Widiati, A., & Alfarisi, M. A. (2023) · At-Turats · doi:10.24260/at-turats.v17i2.2843
“…comprehension, the lecturer proposed a creative response question which constitutes higher-level questions that were intended to familiarize students with higher-order thinking skills (Agarwal, 2019; Kočiský, Schwarz, Blunsom, Dyer, Hermann, Melis & Grefenstette, 2018; Putra & Abdullah, 2019).”
Background
SpanBERT-based Multilayer Fusion Model for Extractive Reading Comprehension
Zhang, P., He, L., & Xi, D. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.0150149 · Cited by 1
“NarrativeQA [19] is a dataset that represents generative reading comprehension, and UniLMv2 [20] is a model that represents this type of task.”
Background
When RAG and LLM Conflict: Who Will AI Listen to?
Fang, Q. (2024) · doi:10.59350/7gfzv-88m36
Automated Multi-Choice Question Answering System using Natural Language Processing
Madoo, R., & Hole, V. (2024) · International Conference for Innovation in Technology (INOCON) · doi:10.1109/inocon60754.2024.10511814
“Not surprisingly, one of the trickiest jobs in the zone of natural language processing is question answering (QA), as it requires high-level analysis [1, 5] in addition to common sense knowledge [2, 3, 4] and general text comprehension.”
Background
Pengaruh Strategi DRTA dan SQ3R Terhadap Kemampuan Membaca Pemahaman Pada Siswa Sekolah Dasar
Sulastri, S., Amprasto, A., & Suhandoko, A. D. J. (2024) · Syntax Literate Jurnal Ilmiah Indonesia · doi:10.36418/syntax-literate.v9i5.15455
Tasks for LLMs and Their Evaluation
Ostapuk, N., & Audiffren, J. (2024) · Springer eBooks · doi:10.1007/978-3-031-54827-7_6 · Cited by 2
BERT-Based Question-Answering for Narrative Text on Document Corpus
Wadawadagi, R. S., Tiwari, S., & Hanji, S. V. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-75605-4_26
SimLayerKV: A Simple Framework for Layer-Level KV Cache ReductionPreprint
Zhang, X., Du, C., Du, C., Pang, T., Gao, W., & Lin, M. C. (2024) · Qeios · doi:10.32388/epai4t · Cited by 2
DLUE: Benchmarking Document Language Understanding
Xu, R. F., Lin, H., Guan, X., Sun, Y., & Sun, L. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-8367-0_23
Yabancı Dilde Okunan Metnin Anlama Durumunun Fonksiyonel Yakın Kızılötesi Spektroskopisi Sinyalleri ve Ardışıl İleri İçsel Mod Fonksiyonu Seçme Yaklaşımı ile Otomatik Belirlenmesi
Akıncıoğlu, U., Aydemir, Ö., Çıl, A., & Baydere, M. (2024) · Karadeniz Fen Bilimleri Dergisi · doi:10.31466/kfbd.1517668
Towards Efficient Dataset Development: A Case Study of M2Q2+ in Movie QA Systems
Aggoune, A., & Mihoubi, Z. (2024) · International Conference on Advanced Aspects of Software Engineering · doi:10.1109/icaase64542.2024.10850962 · Cited by 2
CritiPrefill: A Segment-wise Criticality-based Approach for Prefilling Acceleration in LLMs
Lv, J., Feng, Y., Xie, X., Jia, X., Peng, Q., & Xie, G. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887916
“A. Settings 1) Dataset: We conduct thorough evaluations across four datasets in both Single-hop [30] and Multi-hop QA [2] scenarios.”
Elevating Textual Question Answering with On-Demand Visual Augmentation
Ehsani, S., & Liu, J. (2025) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3729231 · Cited by 1
Streamrag: a Lock-Aware and Traffic-Aware Query Coordinator in Stream-Based Rag Systems
Jeong, Y., Park, K., & Park, S. (2025) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid64434.2025.00065
Foreword
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.002.0005
Figure
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.002.0007
Learning
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0007
What is this book about?
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0001
Taxonomies of Cognitive Architectures
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0004
Practical Applications of Cognitive Architectures
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0010
Cognitive Architectures, AI, and Cognitive Science
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0003
Copyright Page
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.002.0004
Reasoning and Decision-Making
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0008
Challenges of the Past and Opportunities Ahead
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0013
Memory
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0006
Cognitive Architectures in the Deep Learning Era
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0012
Putting It All Together
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0009
What Are Cognitive Architectures?
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0002
Evaluating Cognitive Architectures
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0011
UQuAD+: Benchmark Dataset for Urdu Machine Reading Comprehension
Kazi, S., & Khoja, S. A. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3759455
“SQuAD [52], NewsQA[60], TriviaQA [26](partially), NarrativeQA [35], XQuAD [7](partially) Automatic Creation Questions, answers, or evidence passages are generated automatically using algorithms or rule-based systems.”
Efficient Benchmarking via Bias-Bounded Subset Selection
Zhuang, Y., Yu, J., Liu, Q., Sun, Y., Li, J., Huang, Z., & Chen, E. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3598031
“When M = 1 , it represents a binary evaluation score ( y ∈ { 0 , 1 } ), such as in multiple-choice questions in various QA benchmarks [43], [44].”
Corm: Cache Optimization with Recent Message for Large Language Model InferencePreprint
Huang, Z., Jiang, H., & Dai, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5388897
DataSculpt: A Holistic Data Management Framework for Long-Context LLMs Training
Lu, K., Nie, X., Liang, Z., Pan, D., Zhang, S., Zhao, K., Chen, W., Zhou, Z., Dong, G., Cui, B., et al. (2025) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde65448.2025.00316 · Cited by 8
Knowledge Retrieval Enhancement for Power System Based on Adaptive Dynamic Text Segmentation and Hierarchical Multi-Query
Zhao, S., Feng, J. H., Shen, X., Yan, H., Chen, Z., Du, J., Luo, P., & Wang, Z. (2025) · Power System and Green Energy Conference (PSGEC) · doi:10.1109/psgec66102.2025.11151081
Deceiving question-answering models: A hybrid word-level adversarial approach
Li, J., Ni, M.-Z., Gong, Y.-S., & Liu, W. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108105
M2Q2: A Text-to-MQL Dataset for Movie QA Systems
Aggoune, A., & Mihoubi, Z. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-90893-4_29
KV-RAPTOR: Scalable Tree-Structured Retrieval with KV Cache Compression for Question-Answering Systems
da Silva, J. G. J., de Oliveira, S. S. T., Alves, L. A., Eiris, N., & Filho, A. R. G. (2025) · Simpósio Brasileiro de Bancos de Dados (SBBD) · doi:10.5753/sbbd.2025.247245
“NarrativeQA dataset [Koˇciský et al. 2018] consists of narrative-style documents in English, comprising 1,572 unique document summaries after filtering.”
ROADBLOCKS TO PROFICIENCY: TOEFL READING COMPREHENSION CHALLENGES FOR UNIVERSITY STUDENTS
Wicaksono, A., Sulistyani, S., Suhartono, S., Khoiriyah, K., & Aji, M. P. P. (2025) · JEELL (Journal of English Education Linguistics and Literature) English Departement of STKIP PGRI Jombang · doi:10.32682/jeell.v12i3.90
“…is a very challenging because it assesses not only linguistic proficiency but also cognitive skills like critical analysis, inference, and effective information processing in reading comprehension is especially difficult (Elleman & Oslund, 2019; Kocisky et al., 2018; Enright et al., 2000).”
Use of Graph-Based Knowledge Organization to Improve the Results of Retrieval Augmented Generation for Narrative Texts
Podstawski, M., & Кamola, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-03711-4_12
Asking Questions with Thoughts: An Efficient Difficulty-Controllable Question Generation Method with Posterior Knowledge Distillation
Wu, S., Chen, J., Zhou, Y., Yang, Z., & Zhou, W. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761043
“Finally, in Table 2, we have constructed two datasets based on two well-known QA datasets FairytaleQA [28] and NarrativeQA [13].”
Towards Complex Question Answering in Polish Language
Wojtasik, K., Domagała, A., Oleksy, M., & Piasecki, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-09318-9_18 · Cited by 1
ReSSFormer: A Recursive Sparse Structured Transformer for Scalable and Long-Context Reasoning
You, H., & Liu, B. (2025) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3743093.3770926
“We evaluate ReSSFormer on three task categories: (1) Long-context reasoning with NarrativeQA [15], Hot-potQA [38], and GSM8K [5]; (2) Scalable modeling efficiency via Wikitext-103 [29] and PG-19 [7]; and (3) Structure generalization us-ing TabFact [3], OGB-Arxiv [10], and a synthetic…”
STARC: Selective Token Access with Remapping and Clustering for Efficient LLM Decoding on PIM Systems
Fan, Z., Liu, Y., Gagnon, G., Liu, L., Hou, Y., Benmeziane, H., Maghraoui, K. E., & Liu, L. (2026) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3779212.3790226 · Cited by 6
Unlocking Potential: How the RADEC Model and Motivation Transform Expository Narrative Essay Writing
Marcelina, L., Sukma, E., Fetri Yeni, J., Tiarina, Y., & Fauziah, D. N. (2025) · International Journal of Elementary Education · doi:10.23887/ijee.v9i3.96677
Dispenser: Hierarchical KV Cache Management for Efficient LLM Generative Inference
Cao, B., Bian, K., Luo, G., & Kim, J. (2025) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads67057.2025.11323033
“…from LongBench [26], covering tasks such as multi-document QA: HotpotQA [21], 2WikiMQA [22], Musique [23], single-document QA: Narra-tiveQA [24], Qasper [25], MultiFieldQA [26], Summarization: GovReport [27], MultiNews [28], Qmsum [29], Few shot: Trec [30], TriviaQA [31], Samsum [32], as…”
Enhancing Multi-Hop Reading Comprehension through Multi-Hop Attention Diffusion with Hierarchical Graph Extensions
Zhang, Y., & Cheng, B. H. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11343127
“To support Multi-hop RC research, several high-quality datasets have been developed, including NarrativeQA [8], HotpotQA [4], R 4 C [9], 2WikiMultiHopQA [10], and MuSiQue [11].”
CCFBQGD: Chinese Cross-Domain Fill-in-the-Blank Question Generation Dataset and Its Benchmarking Methodology
Liu, H. T., Zhang, Y., Meng, Q., & Zhang, X. (2025) · International Conference on Big Data Analytics and Artificial Intelligence Application · doi:10.1145/3788108.3788109
A Spanish Language Proficiency Dataset for AI Evaluation
Peñas, A., Rodrigo, Á., Giménez, J. F., Pastor, I. S., Moreno‐Álvarez, S., Pérez, A., & Reyes-Montesinos, J. (2026) · Information · doi:10.3390/info17020159
Question Answering on Lengthy Arabic Narrative Books
Ateeq, M. A., Tiun, S. B., Abdelhaq, H., & Kaur, W. (2026) · IEEE Access · doi:10.1109/access.2026.3661853
ParaMind: Parallelized Adaptive Memory Integration for Long-Context Large Language Models
Basrur, S. (2025) · International Conference on Intelligent Systems and Embedded Design (ISED) · doi:10.1109/ised67359.2025.11405146
Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive-k
Taguchi, C., Maekawa, S., & Bhutani, N. (2026) · Journal of Natural Language Processing · doi:10.5715/jnlp.33.382
Context-Driven Question-Answering on Narrative Text Document Corpus
Wadawadagi, R. S., Tiwari, S., & Naithani, K. (2026) · SN Computer Science · doi:10.1007/s42979-026-04854-4
TAMTRL: Teacher-aligned reward reshaping for multi-turn reinforcement learning in long-context compression
Wang, L., Wang, Y., Yu, X., Zhang, K., Peng, T., & Wu, W. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109492 · Cited by 1
GRACE-RAG: Graph Retrieval with Adaptive Chunk Extraction for Long-Context Question Answering
Huang, T., Lei, S., Hamdulla, A., Gao, C., & Zhang, H. (2026) · International Conference on Communication Networks and Machine Learning (CNML) · doi:10.1109/cnml68938.2026.11452294
Sequential Context Injection-Based Chunking for Mitigating Context Fragmentation
Lee, E.-J., Han, D.-S., Lee, S.-M., & Park, K.-S. (2026) · Journal of Digital Contents Society · doi:10.9728/dcs.2026.27.3.849
Self-Consistency Falls Short! The Adverse Effects of Positional Bias on Long-Context Problems
Byerly, A., & Khashabi, D. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.625 · Cited by 8
Human-Machine Dialogue in the Medical Field. Using Dialog to Collect Important Patient Information
Liednikova, A. (2022) · theses.fr (ABES) · doi:10.70675/b572f8cbz5876z4683z828bzfb09085ddd78
Vers la mitigation des biais en traitement neuronal des langues
Berre, G. L. (2023) · theses.fr (ABES) · doi:10.70675/f37c3802z821az4a6dzb877zf631994cacd5
Meta-Evaluation Methodology and Benchmark for Automatic Story Generation
Chhun, C. (2024) · theses.fr (ABES) · doi:10.70675/ec296eb8z87a1z4fdbzbbbdz12bb27449b8f
On the Importance of a Multi-Scale Calibration for Quantization
Son, S., Seong, I., Kim, J., Jang, H., & Jeon, Y. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463220
TCC: Using Topic Chains to Compress Prompts for Long Document Question Answering
Chen, S. F., Liang, B., Yan, J., Liu, L., Wang, H., Yu, Y., Wong, K., & Xu, R. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463253
FictionRAG: A Stateful Metacognitive Framework for High-Fidelity Long-Narrative Role-Playing
Deng, Y., Zhang, Y., Yang, J., & Fang, M. (2026) · Algorithms · doi:10.3390/a19050383
SES-RAG: An Adaptive Framework with ModernBERT Segmentation and Generative-Alignment Expansion
Gao, C. S., He, H., & Zhang, Q. (2026) · Applied Sciences · doi:10.3390/app16104849
SparKV: Overhead-Aware KV Cache Loading for Efficient On-Device LLM Inference
Liu, H., Zhai, L., Wang, J., Fang, Z., Chen, J., & Huang, J. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3695763
Unified AI-Driven Multimodel Framework for Real-Time Lecture Understanding and Study Support
Kiruthiga, P. J., Harshith, K. S. V., Sahith, K., Kavitha, M. L., & Deekshitha, K. L. (2026) · International Conference on Augmented Intelligence and Sustainable Systems (ICAISS) · doi:10.1109/icaiss68683.2026.11526631
“Kocisky et al. [7] looked into transformer-based methods to produce educational flashcards from textbooks.”
HITL-DEE: A Human-in-the-Loop Dynamic Early-Exit Framework via Agentic Retrieval-Augmented GenerationPreprint
Zhu, B., Feng, L., Dong, Z., Chen, R., Gong, C., Liu, S., Zhang, C., Zhang, X.-L., & Li, X. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6876299
Estimating difficulty for the open-ended reading comprehension questions via Large Language Models
Chen, J., Wu, S., He, W., Yang, Z., & Zhou, W. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.115268
Ebbinghaus Forgetting Curve and Large Language Model Memory Management: A Cognitive Science-Driven Long-Context Enhancement Framework Design and Prospects
Jiang, L., & Cheng, Y. (2026) · International Conference on Information and Computer Technologies (ICICT) · doi:10.1145/3803291.3803294
“• Question Answering Tasks (6): narrativeqa [25], qasper [26], multifieldqa_en, hotpotqa [27], 2wikimqa, musique • Document Length Range: The test data is divided into 4K-8K and 8K+ intervals according to length to systematically check the performance of the framework at different context lengths…”
HeteroRAGCache: Software-Hardware Co-Design for Efficient RAG Caching using Emerging Memories
Park, J., Suh, K., Ponzina, F., & Rosing, T. (2026) · ACM Great Lakes Symposium on VLSI (GLSVLSI) · doi:10.1145/3787109.3815268
“QAsper [6] reuses about 20% of vectors per document, while NarrativeQA [18] shows only 1% reuse.”
Beyond factual events: Evaluating LLMs’ capability of cognitive processes understanding in narratives
Liu, F., Zhong, Z., Yang, A., Liu, Z., Jia, Q., Wu, Y., Yang, F., & Jing, N. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.105025
Long-document Representation for Retrieval and Retrieval-augmented Generation: A Representation-First, Distillation-aware SurveyPreprint
Martínez-Cruz, R., López-López, Á. J., Portela, J., & Zayat, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7247158
AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruption
Wang, Y., Geng, R., Chen, Y., & Jia, J. (2026) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp63933.2026.00198 · Cited by 13
SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving
Zhou, Q., Yin, P., Zuo, P., Wang, C., & Cheng, J. S.-C. (2026) · International Conference on Supercomputing · doi:10.1145/3797905.3800528 · Cited by 12
“…experiments using multiple datasets from LongBench [5], which covers various task types including Question Answering (Qasper [12], NarrativeQA [22], MultifieldQA [19], Dureader [18]), Document Summarization (GovReport [20], QMSum [52], MultiNews [14], VCSum [43]), and Code Generation (LCC…”
Fast and Faithful: Real-Time Verification for Long-Document Retrieval-Augmented Generation Systems
Liu, X., He, B., Liu, X., Zhang, H., & Chen, H. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808493
VNFinEval: A Benchmark for Evaluating Large Language Models in Vietnamese Finance
Nguyen, H., Hoang, N., Pham, G., Xuan, H. N., & Quoc, T. N. (2026) · International Conference on Computational Intelligence and Communication Networks · doi:10.1109/cicn70047.2026.11594251
Synergistic multi-stage processing for enhanced context relevance in RAG architecture
Tien, L. N., & Tieu, B. H. (2026) · Discover Computing · doi:10.1007/s10791-026-10319-2
Incentivizing Retrieval-Augmented Generation via Inner Adaptive Context Selection
Cui, C., Shen, L., Fan, H., Zhu, S., Dai, F., & Li, B. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809671
Beyond accuracy: completeness and relevance metrics for evaluating the quality of long answers
Cortes, E. G., Vieira, R., & Barone, D. A. C. (2026) · Language Resources and Evaluation · doi:10.1007/s10579-026-09936-6
“Moreover, widely used benchmarks such as MS MARCO Bajaj et al. (2018), Nat - ural Questions Kwiatkowski et al. (2019), and NarrativeQA Kočiský et al. (2018) pri - marily utilize metrics like ROUGE and BLEU for evaluation, which primarily focus on textual fidelity Lin (2004); Sulem et al. (2018); Papineni et al. (2002).”
A Review of Deep Learning-Based Question Generation: Challenges, Methods, and Future Directions
Yehia, M. F., Helmi, Y. M., & Bahloul, M. M. (2026) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-026-01497-4
EC-RAG: Towards Efficient Edge-Cloud Retrieval-Augmented Generation Systems
Wang, L., Wang, K., Jia, R., Lu, K., Wan, J., Huo, H., Zhai, Y., Liang, Z., & Wang, D. (2026) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde65706.2026.00082
“We select four datasets from Long-Bench [29] for evaluation: HotpotQA [21], 2WikiMultiHopQA (2WikiMQA) [43], NarrativeQA [44], and Qasper [45].”
Context-Aware Educational Question Generation for Children’s Storybooks via a Modular Prompt-Optimized Pipeline
Hassan, A. N. A. A., Zhao, Z., Farea, I. K. Q., & Hassan, Q. N. A. (2026) · International Conference on Emerging Smart Technologies and Applications (eSmarTA) · doi:10.1109/esmarta70636.2026.11652148
“On the dataset side, NarrativeQA [14] and TellMeWhy [15] established benchmarks for whole-story understanding and causal inference respectively, but neither was designed with a pedagogical taxonomy or age-appropriate question design in mind.”

Authors: Kočiský, T., Schwarz, J., Blunsom, P., Dyer, C., Hermann, K. M., Melis, G., & Grefenstette, E.
Affiliations: University of Oxford (GB)
Google DeepMind (United Kingdom) (GB)
ORCID: Schwarz, Jonathan
Melis, Gábor
Grefenstette, Edward
Venue: Transactions of the Association for Computational Linguistics
Publisher: MIT Press
Published: 2018-12
Volume: 6
Pages: 317-328
ISSN: 2307-387X (ISSN-L)
DOI: 10.1162/tacl_a_00023
OpenAlex: W2779681640
Cited by: 596
References: 31
Citations by year: 2026: 51 · 2025: 65 · 2024: 40 · 2023: 77 · 2022: 71 · 2021: 87 · 2020: 80 · 2019: 91 · 2018: 32
FWCI: 40.86
Type: journal article
Language: en
Open access: diamond
doi.org
Licence: cc-by
Locations: Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
arXiv (Cornell University) (submittedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
arXiv (Cornell University)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords: reading comprehension, narrative comprehension, question answering, screenplays, deep comprehension

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

The NarrativeQA Reading Comprehension Challenge

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks