QuAC: Question Answering in Context
Open accessAbstract
We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total).The dialogs involve two crowd workers: (1) a student who poses a sequence of freeform questions to learn as much as possible about a hidden Wikipedia text, and (2) a teacher who answers the questions by providing short excerpts from the text.QuAC introduces challenges not found in existing machine comprehension datasets: its questions are often more open-ended, unanswerable, or only meaningful within the dialog context, as we show in a detailed qualitative evaluation.We also report results for a number of reference models, including a recently state-ofthe-art reading comprehension architecture extended to model dialog context.Our best model underperforms humans by 20 F1, suggesting that there is significant room for future work on this data.Dataset, baseline, and leaderboard available at http://quac.ai.How was perversion handled?How long was he there?How popular did she become?How did Mark Felt contact Woodword?How did the meeting go?How did it do on the charts?When was she born?When was it founded?When was the breakup?
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most currently available datasets for training question generation systems are crowd-sourced, which largely affects the quality of generated questions.
“Most datasets that are currently available for training question generation systems are crowd-sourced [42, 47, 84, 92, 114, 115, 119, 120, 122, 123], which largely impacts the quality of generated questions.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [114]
For QA, on SQuADv2, QuAC, and many other datasets, fine-tuned models have superior performance, while on CoQA, LLMs perform as well as fine-tuned models.
“For question answering (QA), on SQuADv2 [97], QuAC [24], and many other datasets, fine-tuned models have superior performance, while on CoQA [98], LLMs perform as well as fine-tuned models [25].”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 2× · see citing works
“BiDAF++ We use a re-implementation of a top-performing SQuAD model (Peters et al., 2018) that augments bidirectional attention flow (Seo et al., 2016, BiDAF) with self-attention (Clark and Gardner, 2018) and contextualized embeddings.”
MethodSelf-citationSQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 2× · see citing works
“Reading Comprehension Our work builds on span based reading comprehension (Rajpurkar et al., 2016; Joshi et al., 2017; Trischler et al., 2016), while also incorporating innovations such as curating questions independently of supporting text to reduce trivial lexical overlap (Joshi et al., 2017;…”
BackgroundSelf-citationSuchanek, F. M., Kasneci, G., & Weikum, G. (2007) · International World Wide Web Conference (WWW) · doi:10.1145/1242572.1242667 · Cited by 3,886 · see citing works
Bidirectional Attention Flow for Machine ComprehensionPreprint
Seo, M., Kembhavi, A., Farhadi, A., & Hajishirzi, H. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1611.01603 · Cited by 2,322 · cited 1× · see citing works
“BiDAF++ We use a re-implementation of a top-performing SQuAD model (Peters et al., 2018) that augments bidirectional attention flow (Seo et al., 2016, BiDAF) with self-attention (Clark and Gardner, 2018) and contextualized embeddings.”
MethodKnow What You Don’t Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2124 · Cited by 2,240 · see citing works
Self-citationSupervised Learning of Universal Sentence Representations from Natural Language Inference Data
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1070 · Cited by 2,049 · cited 1× · see citing works
“Pretrained InferSent To test the importance of lexical matching in our dataset, we output the sentence in s whose pretrained InferSent representation (Conneau et al., 2017) has the highest cosine similarity to that of the question.”
MethodTriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1147 · Cited by 1,690 · cited 8× · see citing works
“adopted in other reading comprehension datasets such as SQuAD, TriviaQA (Joshi et al., 2017), and NewsQA (Trischler et al.”
BackgroundMethodInfluentialSelf-citationAllenNLP: A Deep Semantic Natural Language Processing Platform
Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N. F., Peters, M. E., Schmitz, M., & Zettlemoyer, L. (2018) · Workshop for NLP Open Source Software (NLP-OSS) · doi:10.18653/v1/w18-2501 · Cited by 1,273 · cited 1× · see citing works
“The AllenNLP (Gardner et al., 2017) implementation we use reaches 82.”
MethodSelf-citationDeep Reinforcement Learning for Dialogue Generation
Li, J., Monroe, W., Ritter, A., Jurafsky, D., Galley, M., & Gao, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1127 · Cited by 1,221 · cited 1× · see citing works
“19 Dialog fits into an increasing interest in open domain dialog, mostly studied in the context of social chit-chat (Li et al., 2016; Ritter et al., 2011; Fang et al., 2017; Ghazvininejad et al., 2018).”
BackgroundMS MARCO: A Human Generated MAchine Reading COmprehension DatasetPreprint
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T. G., et al. (2016) · arXiv (Cornell University) · doi:10.4230/lipics.giscience.2025.5 · Cited by 1,190 · see citing works
CoQA: A Conversational Question Answering Challenge
Reddy, S., Chen, D., & Manning, C. D. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00266 · Cited by 975 · cited 6× · see citing works
“CoQA (Reddy et al., 2018) CSQA (Saha et al., 2018) CQA (Talmor and Berant, 2018) SQA (Iyyer et al., 2017) NarrativeQA (Kocisk´y et al., 2017) TriviaQA (Joshi et al., 2017) SQuAD 2.0 (Rajpurkar et al., 2018) MS Marco (Nguyen et al., 2016) NewsQA (Trischler et al., 2016) Table 1: Comparison of the QUAC dataset to other…”
BackgroundMethodInfluentialNewsQA: A Machine Comprehension Dataset
Trischler, A., Wang, T., Yuan, X., Harris, J., Sordoni, A., Bachman, P., & Suleman, K. (2017) · Proceedings of the 2nd Workshop on Representation Learning for NLP · doi:10.18653/v1/w17-2623 · Cited by 706 · cited 6× · see citing works
“…2017) NarrativeQA (Kocisk´y et al., 2017) TriviaQA (Joshi et al., 2017) SQuAD 2.0 (Rajpurkar et al., 2018) MS Marco (Nguyen et al., 2016) NewsQA (Trischler et al., 2016) Table 1: Comparison of the QUAC dataset to other question answering datasets. tion”), which makes the dialogs more…”
BackgroundMethodInfluentialThe NarrativeQA Reading Comprehension Challenge
Kočiský, T., Schwarz, J., Blunsom, P., Dyer, C., Hermann, K. M., Melis, G., & Grefenstette, E. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00023 · Cited by 596 · cited 5× · see citing works
“CoQA (Reddy et al., 2018) CSQA (Saha et al., 2018) CQA (Talmor and Berant, 2018) SQA (Iyyer et al., 2017) NarrativeQA (Kocisk´y et al., 2017) TriviaQA (Joshi et al., 2017) SQuAD 2.0 (Rajpurkar et al., 2018) MS Marco (Nguyen et al., 2016) NewsQA (Trischler et al., 2016) Table 1: Comparison of the QUAC dataset to other…”
BackgroundInfluentialData-Driven Response Generation in Social Media
Ritter, A., Cherry, C. A., & Dolan, W. B. (2011) · Cited by 566 · see citing works
The Web as a Knowledge-Base for Answering Complex Questions
Talmor, A., & Berant, J. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1059 · Cited by 520 · cited 4× · see citing works
“CoQA (Reddy et al., 2018) CSQA (Saha et al., 2018) CQA (Talmor and Berant, 2018) SQA (Iyyer et al., 2017) NarrativeQA (Kocisk´y et al., 2017) TriviaQA (Joshi et al., 2017) SQuAD 2.0 (Rajpurkar et al., 2018) MS Marco (Nguyen et al., 2016) NewsQA (Trischler et al., 2016) Table 1: Comparison of the QUAC dataset to other…”
BackgroundInfluentialZero-Shot Relation Extraction via Reading Comprehension
Levy, O., Seo, M., Choi, E., & Zettlemoyer, L. (2017) · Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017) · doi:10.18653/v1/k17-1034 · Cited by 482 · see citing works
Self-citationZero-Shot Relation Extraction via Reading Comprehension
Levy, O., Seo, M., Choi, E., & Zettlemoyer, L. (2017) · arXiv · doi:10.48550/arxiv.1706.04115 · Cited by 482
Self-citationA Knowledge-Grounded Neural Conversation Model
Ghazvininejad, M., Brockett, C., Chang, M.-W., Dolan, B., Gao, J., Yih, W.-T., & Galley, M. (2018) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v32i1.11977 · Cited by 457 · see citing works
Self-citationDas, A., Kottur, S., Gupta, K., Singh, A., Jain, D., Moura, J. M. F., Parikh, D., & Batra, D. (2017) · Cited by 432 · see citing works
Simple and Effective Multi-Paragraph Reading Comprehension
Clark, C., & Gardner, M. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1078 · Cited by 402 · cited 2× · see citing works
“, 2016, BiDAF) with self-attention (Clark and Gardner, 2018) and contextualized embeddings.”
BackgroundDeal or No Deal? End-to-End Learning of Negotiation Dialogues
Lewis, M., Yarats, D., Dauphin, Y., Parikh, D., & Batra, D. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1259 · Cited by 347 · cited 2× · see citing works
“More explicit goal driven scenarios, such as bargaining (Lewis et al., 2017)”
BackgroundExpanding the scope of the ATIS task
Dahl, D. A., Bates, M., Brown, M., Fisher, W. M., Hunicke-Smith, K., Pallett, D. S., Pao, C., Rudnicky, A. I., & Shriberg, E. E. (1994) · DARPA Workshop on Speech and Natural Language (HLT) · doi:10.3115/1075812.1075823 · Cited by 306 · cited 2× · see citing works
“Such open information seeking was studied in semantic parsing on knowledge bases (Dahl et al., 1994) and more recently with modern approaches (Saha et al.”
BackgroundSearch-based Neural Structured Learning for Sequential Question Answering
Iyyer, M., Yih, W.-T., & Chang, M. (2017) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/p17-1167 · Cited by 219 · cited 4× · see citing works
“CoQA (Reddy et al., 2018) CSQA (Saha et al., 2018) CQA (Talmor and Berant, 2018) SQA (Iyyer et al., 2017) NarrativeQA (Kocisk´y et al., 2017) TriviaQA (Joshi et al., 2017) SQuAD 2.0 (Rajpurkar et al., 2018) MS Marco (Nguyen et al., 2016) NewsQA (Trischler et al., 2016) Table 1: Comparison of the QUAC dataset to other…”
BackgroundMethodInfluentialSelf-citationLearning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddings
He, H., Balakrishnan, A., Eric, M., & Liang, P. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1162 · Cited by 180 · cited 2× · see citing works
“and item guessing (He et al., 2017) have also been explored, but the language is more constrained than in .”
BackgroundSelf-citationSaha, A., Pahuja, V., Khapra, M. M., Sankaranarayanan, K., & Chandar, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11332 · Cited by 179 · see citing works
Interpretation of Natural Language Rules in Conversational Machine Reading
Saeidi, M., Bartolo, M., Lewis, P., Singh, S., Rocktäschel, T., Sheldon, M., Bouchard, G., & Riedel, S. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1233 · Cited by 141 · cited 1× · see citing works
“Concurrent to our work, Saeidi et al. (2018) proposed a task of generating and answering yes/no questions for rule focused text (such as traffic laws) by interacting with a user through dialog.”
BackgroundInformation-Seeking Chat: Dialogues Driven by Topic-Structure
Stede, M., & Schlangen, D. (2004) · PUB – Publications at Bielefeld University (Bielefeld University) · Cited by 13 · see citing works
Analysis of Wikipedia-based Corpora for Question AnsweringPreprint
Jurczyk, T., Deshmane, A. A., & Choi, J. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.02073 · Cited by 6 · cited 2× · see citing works
“SQuAD, which is unsurprising as most SQuAD answers are either entities or numerics (Jurczyk et al., 2018) while questions can be more open-ended.”
Background
Flagged first, then by citations
Research works printed but not linked: 4
- Hao Fang, Hao Cheng, Elizabeth Clark, Ariel Holtz- man, Maarten Sap, Mari Ostendorf, Yejin Choi, and Noah A Smith. 2017. Sounding board-university of washingtons alexa prize submission. Alexa Prize Proceedings.
- Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. Conference of the North American Chapter of the Association for Computational Lin- guistics.
- Marzieh Saeidi, Max Bartolo, Patrick Lewis, Sameer Signh, Tim Rocktschel, Mike Sheldon, Guillaume Bouchard, and Sebastian Riedel. 2018. Interpreta- tion of natural language rules in conversational ma- chine reading. In Proceedings of Empirical Methods in Natural Language Processing.
- Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: a core of semantic knowl- edge. In Proceedings of the World Wide Web Con- ference.
Cited by
Sorted by citations
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
“Academic researchers have adopted our proposal and released datasets with accompanying datasheets [e.g., 7, 10, 23, 26].”
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · see citing works
“Academic researchers have adopted our proposal and released datasets with accompanying datasheets [e.g., 7, 10, 23, 26].”
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · see citing works
Pre-Training With Whole Word Masking for Chinese BERT
Cui, Y., Che, W., Liu, T., Qin, B., & Yang, Z. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3124365 · Cited by 1,722 · see citing works
“As we traverse several popular machine reading comprehension benchmarks, such as SQuAD [3], CoQA [4], QuAC [5], NaturalQuestions [6], RACE [7], we can see that most of the top-performing models are based on BERT and its variants [8], [9], [10], demonstrating that the pre-trained language models…”
MethodA Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257 · see citing works
“StoryCloze [308], LAMBADA [309], LCSTS [310], AdGen [311], E2E [312], CHID [313], CHID-FC [286] Physical Knowledge and World Understanding PIQA [314], TriviaQA [315], ARC [316], ARC-Easy [316], ARC-Challenge [316], PROST [317], Open-BookQA [318], WebNLG [319], DogWhistle Insider & Outsider [320] Contextual Language Un…”
InfluentialFinetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101 · see citing works
“Conversational question-answering: QuAC (Choi et al., 2018) and CoQA (Reddy et al., 2019) 2.”
BackgroundCoQA: A Conversational Question Answering Challenge
Reddy, S., Chen, D., & Manning, C. D. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00266 · Cited by 975 · see citing works
An introduction to Deep Learning in Natural Language Processing: Models, techniques, and tools
Lauriola, I., Lavelli, A., & Aiolli, F. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.05.103 · Cited by 721 · see citing works
Harnessing the power of llms in practice: a survey on chatGPT and beyond
Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Zhong, S., Yin, B., & Hu, X. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3649506 · Cited by 644 · see citing works
“For question answering (QA) , on SQuADv2 [97], QuAC [24], and many other datasets, fine-tuned models have superior performance, while on CoQA [98], LLMs perform as well as fine-tuned models [25].”
BackgroundWizard of Wikipedia: Knowledge-Powered Conversational agentsPreprint
Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., & Weston, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01241 · Cited by 610 · see citing works
“d performing retrieval over the entirety of Wikipedia (Chen et al., 2017). Recently, the QuAC dataset investigates similar themes, but as a sequence of questions and answers in dialogue form instead (Choi et al., 2018). In this work we do not address question answering, but focus on natural human dialogues which conta…”
BackgroundMultitask Prompted Training Enables Zero-Shot Task Generalization
Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.08207 · Cited by 612 · see citing works
BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment AnalysisPreprint
Xu, H., Liu, B., Shu, L., & Yu, P. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02232 · Cited by 504 · see citing works
“Many datasets have been created for MRC from formally written and objective texts, e.g., Wikipedia (WikiReading (Hewlett et al., 2016), SQuAD (Rajpurkar et al., 2016, 2018), Wiki-Hop (Welbl et al., 2018), DRCD (Shao et al., 2018), QuAC (Choi et al., 2018), HotpotQA (Yang et al., 2018)) news and other articles (CNN/Dai…”
BackgroundDROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
Dua, D., Wang, Y., Dasigi, P., Stanovsky, G., Singh, S., & Gardner, M. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1246 · Cited by 393 · see citing works
“All of these datasets throw in additional complexities to the reading comprehension challenge, around tracking conversational state (Reddy et al., 2019; Choi et al., 2018), requiring passage retrieval (Joshi et al.”
BackgroundDon’t Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases
Clark, C., Yatskar, M., & Zettlemoyer, L. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1418 · Cited by 381 · see citing works
“For example, both CoQA (Reddy et al., 2019) and QuAC (Choi et al., 2018) take steps to prevent annotators from us-ing words that occur in the context passage, VQA 2.0 (Goyal et al., 2018) selects examples to limit the effectiveness of question-only models, and others have filtered examples solvable…”
BackgroundInfluentialSelf-citationBoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., & Toutanova, K. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1300 · Cited by 391 · see citing works
“Yes/No questions do appear as a subset of some existing datasets (Reddy et al., 2018; Choi et al., 2018; Yang et al., 2018).”
BackgroundInfluentialClark, J. H., Choi, E., Collins, M., Garrette, D., Kwiatkowski, T., Nikolaev, V., & Palomaki, J. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00317 · Cited by 432 · see citing works
“Recognizing the need to align research with the impact it will have on real users, the community has responded with datasets of information-seeking questions such as WikiQA (Yang et al., 2015), MS MARCO (Nguyen et al., 2016), QuAC Pronounced tie dye Q. A. —like the colorful t-shirt.”
BackgroundInfluentialSelf-citationELI5: long form question answering
Fan, A., Jernite, Y., Perez, E., Grangier, D., Weston, J., & Auli, M. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1346 · Cited by 319 · see citing works
“Extractive QA Extractive question answering datasets such as TREC (Voorhees, 2003), SQuAD (Rajpurkar et al., 2016, 2018), NewsQA (Trischler et al., 2017), SearchQA (Dunn et al., 2017), and QuAC (Choi et al., 2018) constrain the answer to a word or short phrase from the input and evaluate using exact match or F1 with t…”
BackgroundMethodAsking Clarifying Questions in Open-Domain Information-Seeking Conversations
Aliannejadi, M., Zamani, H., Crestani, F., & Croft, W. B. (2019) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3331184.3331265 · Cited by 317 · see citing works
“o the task of question answering. This makes it challenging and more expensive to pair two workers as participants of the same conversation. There are some examples of this approach in the literature [11, 36]; however, they address the task of reading comprehension, a task that is considerably simpler than identifying…”
BackgroundBERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis
Xu, H., Liu, B., Shu, L., & Yu, P. S. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1242 · Cited by 308 · see citing works
“Many datasets have been created for MRC from formally written and objective texts, e.g., Wikipedia (WikiReading (Hewlett et al., 2016), SQuAD (Rajpurkar et al., 2016, 2018), Wiki-Hop (Welbl et al., 2018), DRCD (Shao et al., 2018), QuAC (Choi et al., 2018), HotpotQA (Yang et al., 2018)) news and other articles (CNN/Dai…”
BackgroundRecent advances in deep learning based dialogue systems: a systematic survey
Ni, J., Young, T., Pandelea, V., Xue, F., & Cambria, E. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10248-8 · Cited by 279 · see citing works
“Question Answering Rajpurkar et al. (2018) LIF A dataset constructed based onQuAC.”
BackgroundMethodInfluentialDREAM: A Challenge Data Set and Models for Dialogue-Based Reading Comprehension
Sun, K., Yu, D., Chen, J., Yu, D., Choi, Y., & Cardie, C. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00264 · Cited by 245 · see citing works
“data set construction (Hermann et al., 2015; Hill et al., 2016; Onishi et al., 2016; Rajpurkar et al., 2016; Bajgar et al., 2016; Nguyen et al., 2016; Trischler et al., 2017; Joshi et al., 2017; Choi et al., 2018).”
BackgroundSelf-citationBERT with History Answer Embedding for Conversational Question Answering
Qu, C., Yang, L., Qiu, M., Croft, W. B., Zhang, Y., & Iyyer, M. (2019) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3331184.3331341 · Cited by 220 · see citing works
Self-citationMRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension
Fisch, A., Talmor, A., Jia, R., Seo, M., Choi, E., & Chen, D. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5801 · Cited by 220 · see citing works
“Interest in MRQA also stems from their use in industry applications, such as search engines (Kwiatkowski et al., 2019) and dialogue systems (Reddy et al., 2019; Choi et al., 2018).”
BackgroundSelf-citationFoundation metrics for evaluating effectiveness of healthcare conversations powered by generative AI
Abbasian, M., Khatibi, E., Azimi, I., Oniani, D., Shakeri Hossein Abad, Z., Thieme, A., Sriram, R., Yang, Z., Wang, Y., Lin, B., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01074-z · Cited by 213 · see citing works
Gao, J., Galley, M., & Li, L. C. (2019) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000074 · Cited by 44 · see citing works
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No QuestionsPreprint
Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., & Toutanova, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10044 · Cited by 425 · see citing works
“Yes/No questions do appear as a subset of some existing datasets (Reddy et al., 2018; Choi et al., 2018; Yang et al., 2018).”
BackgroundInfluentialReCoRD: Bridging the Gap between Human and Machine Commonsense Reading ComprehensionPreprint
Zhang, S., Liu, X., Liu, J., Gao, J., Duh, K., & Durme, B. V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.12885 · Cited by 201 · see citing works
“There are other large-scale datasets (Nguyen et al., 2016; Joshi et al., 2017; Lai et al., 2017; Dunn et al., 2017; Kocisky et al., 2018; Reddy et al., 2018; Choi et al., 2018; Yang et al., 2018) targeting different aspects of reading comprehension.”
BackgroundRetrospective Reader for Machine Reading Comprehension
Zhang, Z., Yang, J., & Zhao, H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i16.17705 · Cited by 200 · see citing works
SParC: Cross-Domain Semantic Parsing in Context
Yu, T., Zhang, R., Yasunaga, M., Tan, Y. C., Lin, X., Li, S., Er, H., Li, I., Pang, B., Chen, T., et al. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1443 · Cited by 176 · see citing works
“Recently, some conversational question answering datasets have been introduced, such as QuAC (Choi et al., 2018) and CoQA (Reddy et al., 2018).”
BackgroundRecent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and ApproachesPreprint
Storks, S., Gao, Q., & Chai, J. Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01172 · Cited by 234 · see citing works
“While the conversational element of CoQA is not new, e.g., QuAC by Choi, He, Iyyer, Yatskar, Yih, Choi, Liang, and Zettlemoyer (2018), CoQA becomes unique by including questions pertaining directly to commonsense reasoning, following Ostermann et al. (2018) in addressing a growing need for reading comprehension datase…”
BackgroundMethodInfluentialTransformer models used for text-based question answering systems
Nassiri, K., & Akhloufi, M. A. (2022) · Applied Intelligence · doi:10.1007/s10489-022-04052-8 · Cited by 177 · see citing works
QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension
Rogers, A., Gardner, M., & Augenstein, I. (2022) · ACM Computing Surveys · doi:10.1145/3560260 · Cited by 177 · see citing works
“CANARD [89] is a dataset of rewritten QuAC questions (50% of the original QuAC), and SaAC [9] is similarly based on a collection of TREC resources.”
BackgroundInfluentialCan You Unpack That? Learning to Rewrite Questions-in-Context
Elgohary, A., Peskov, D., & Boyd-Graber, J. L. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1605 · Cited by 159 · see citing works
“C ontext A bstraction: N ecessary A dditional R ewritten D iscourse—a new dataset that rewrites Q U AC (Choi et al., 2018) questions.”
BackgroundMethodInfluentialNeural Machine Reading Comprehension: Methods and Trends
Liu, S., Zhang, X., Zhang, S., Wang, H., & Zhang, W. (2019) · Applied Sciences · doi:10.3390/app9183698 · Cited by 177 · see citing works
MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension
Talmor, A., & Berant, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1485 · Cited by 157 · see citing works
“Thus, we do not evaluate abstractive (Nguyen et al., 2016) or conversational datasets (Choi et al., 2018; Reddy et al., 2018).”
BackgroundOpen-Retrieval Conversational Question Answering
Qu, C., Yang, L., Chen, C., Qiu, M., Croft, W. B., & Iyyer, M. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401110 · Cited by 154 · see citing works
Self-citationGoing Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer
Powalski, R., Borchmann, Ł., Jurkiewicz, D., Dwojak, T., Pietruszka, M., & Pałka, G. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86331-9_47 · Cited by 163 · see citing works
Retrieving and Reading: A Comprehensive Survey on Open-domain Question AnsweringPreprint
Zhu, F., Lei, W., Wang, C., Zheng, J., Poria, S., & Chua, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.00774 · Cited by 237 · see citing works
“The recently released conversational MRC datasets like CoQA [130] and QuAC [131] are aimed at enabling a Reader to answer the latest question by comprehending not only the given context passage but also the conversation history so far.”
BackgroundInfluentialLearning and Evaluating General Linguistic IntelligencePreprint
Yogatama, D., d’Autume, C. D. M., Connor, J. T., Kočiský, T., Chrzanowski, M., Kong, L., Lazaridou, A., Wang, L., Yu, L., Dyer, C., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.11373 · Cited by 171 · see citing works
“We use two additional reading comprehension datasets: TriviaQA (Joshi et al., 2017) and QuAC 4 (Choi et al., 2018) Semantic Role Labeling.”
MethodInterpretation of Natural Language Rules in Conversational Machine Reading
Saeidi, M., Bartolo, M., Lewis, P., Singh, S., Rocktäschel, T., Sheldon, M., Bouchard, G., & Riedel, S. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1233 · Cited by 141 · see citing works
“In parallel to our work, both Choi et al. (2018) and Reddy et al. (2018) have to began to investigate QA dialogs with background text.”
BackgroundA Dataset of Information-Seeking Questions and Answers Anchored in Research Papers
Dasigi, P., Lo, K., Beltagy, I., Cohan, A., Smith, N. A., & Gardner, M. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.365 · Cited by 138 · see citing works
Improving Machine Reading Comprehension with General Reading Strategies
Sun, K., Yu, D., Yu, D., & Cardie, C. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1270 · Cited by 129 · see citing works
“, 2018) are free-form human generated texts based on given documents (Nguyen et al., 2016; Reddy et al., 2018; Choi et al., 2018).”
BackgroundMethodInfluentialMulla, N., & Gharpure, P. (2023) · Progress in Artificial Intelligence · doi:10.1007/s13748-023-00295-9 · Cited by 132 · see citing works
Conversational question answering: a survey
Zaib, M., Zhang, W. E., Sheng, Q. Z., Mahmood, A., & Zhang, Y. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01744-y · Cited by 134 · see citing works
“The answer Ai can either be a free-form text with evidence [76] or a text span [13].”
BackgroundMethodInfluentialDatasets for large language models: a comprehensive survey
Liu, Y., Cao, J., Liu, C., Ding, K., & Jin, L. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11403-7 · Cited by 159 · see citing works
Query Resolution for Conversational Search with Limited Supervision
Voskarides, N., Li, D., Ren, P., Kanoulas, E., & de Rijke, M. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401130 · Cited by 113 · see citing works
“omplex information seeking conversations [34], researchers have proposed information seeking tasks such as conversational question answering (QA) over simple contexts, such as a single-paragraph text [7, 37]. In contrast to conversational QA over simple contexts, in conversational search, a user aims to interactively…”
InfluentialMulti-step Reasoning via Recurrent Dual Attention for Visual Dialog
Gan, Z., Cheng, Y., Kholy, A. E., Li, L., Liu, J., & Gao, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1648 · Cited by 108 · see citing works
“, 2018), and text-only conversational question answering (Reddy et al., 2018; Choi et al., 2018).”
BackgroundNeural Approaches to Conversational AI
Gao, J., Galley, M., & Li, L. (2019) · now publishers, Inc. eBooks · doi:10.1561/9781680835533 · Cited by 124 · see citing works
Attentive History Selection for Conversational Question Answering
Qu, C., Yang, L., Qiu, M., Zhang, Y., Chen, C., Croft, W. B., & Iyyer, M. (2019) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3357384.3357905 · Cited by 109 · see citing works
Self-citationBeat the ai: Investigating adversarial human annotation for reading comprehension
Bartolo, M., Roberts, A., Welbl, J., Riedel, S., & Stenetorp, P. (2020) · UCL Discovery (University College London) · Cited by 102 · see citing works
Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models
Chen, Z., Xu, L., Zheng, H., Chen, L., Tolba, A. K. R., Zhao, L., Yu, K., & Feng, H. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.052618 · Cited by 112 · see citing works
Question Answering for Privacy Policies: Combining Computational and Legal Perspectives
Ravichander, A., Black, A. W., Wilson, S., Norton, T. B., & Sadeh, N. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1500 · Cited by 103 · see citing works
“Our evaluation metric for answer-sentence selection is sentence-level F1, implemented similar to (Choi et al., 2018; Rajpurkar et al., 2016).”
A Survey on Machine Reading Comprehension—Tasks, Evaluation Metrics and Benchmark Datasets
Zeng, C., Li, S., Li, Q., Hu, J., & Hu, J. (2020) · Applied Sciences · doi:10.3390/app10217640 · Cited by 109 · see citing works
Pretrained Transformers for Text Ranking: BERT and BeyondPreprint
Lin, J., Nogueira, R. F., & Yates, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06467 · Cited by 101 · see citing works
Anik, A. I., & Bunt, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445736 · Cited by 103 · see citing works
“Many machine learning researchers have begun adopting this procedure when releasing their datasets [21,93,100] and this approach is starting to gain traction in some organizations (e.”
BackgroundA review on big data based on deep neural network approaches
Rithani, M., Kumar, R. P., & Doss, S. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10512-5 · Cited by 104 · see citing works
“QuAC Dataset was released in 2018 (Choi et al. 2018), in a conversational style, it comprises of 14 000 questions and the situations that go with them.”
MethodA Survey on Machine Reading Comprehension Systems
Baradaran, R., Ghiasi, R., & Amirkhani, H. (2022) · Natural Language Engineering · doi:10.1017/s1351324921000395 · Cited by 101 · see citing works
COBERT: COVID-19 Question Answering System Using BERT
Alzubi, J. A. A., Jain, R., Singh, A., Parwekar, P., & Gupta, M. (2021) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-021-05810-5 · Cited by 100 · see citing works
“The COBERT is model inspired by the DrQA Open Domain Question Answer model, developed by Chen et al. [25].”
BackgroundMethodA Survey of Cutting-edge Multimodal Sentiment Analysis
Singh, U., Abhishek, K., & Azad, H. K. (2024) · ACM Computing Surveys · doi:10.1145/3652149 · Cited by 99 · see citing works
“…Modalities-Physiologicalsignals Self-reported DBNQADataset[89,300],SQuAD[252],CoQA[257],MSMARCO[225],HOTPOTQA[349],NEWSQ[318],QuAC[37],CliCR[306],RACE[348],Quasar[157],StoryClozeTest[305] Question-Answer Visual…”
Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading
Qin, L., Galley, M., Brockett, C., Liu, X., Gao, X., Dolan, B., Choi, Y., & Gao, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1539 · Cited by 97 · see citing works
“Questions are treated as entirely independent in these “single-turn” MRC models, so recent work (e.g., CoQA (Reddy et al., 2019) and QuAC (Choi et al., 2018)) focuses on multi-turn MRC, modeling sequences of questions and answers in a conversation.”
BackgroundSelf-citationA Review on Question Generation from Natural Language Text
Zhang, R., Guo, J., Chen, L., Fan, Y., & Cheng, X. (2021) · ACM Transactions on Information Systems · doi:10.1145/3468889 · Cited by 91 · see citing works
“Qu estion A nswering in C ontext (QuAC) [40] is a large-scale dataset of information-seeking dialogues over sections from Wikipedia articles, which contains 14K crowdsourced QA dialogues (100K total QA pairs).”
BackgroundConversational Information Seeking
Zamani, H., Trippas, J. R., Dalton, J., & Radlinski, F. (2023) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000081 · Cited by 124 · see citing works
Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints
Baheti, A., Ritter, A., Li, J., & Dolan, B. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1431 · Cited by 87 · see citing works
Li, J., Liu, M., Kan, M., Zheng, Z., Wang, Z., Lei, W., Liu, T., & Qin, B. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.238 · Cited by 84 · see citing works
doc2dial: A Goal-Oriented Document-Grounded Dialogue Dataset
Feng, S., Wan, H., Gunasekara, C., Patel, S., Joshi, S., & Lastras, L. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.652 · Cited by 89 · see citing works
Documenting Computer Vision Datasets
Miceli, M., Yang, T., Naudts, L., Schuessler, M., Serbanescu, D. A., & Hanna, A. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445880 · Cited by 94 · see citing works
“preprocessing, uses, distribution [22, 23, 47], and maintenance [11, 22, 27, 47].”
Keyvan, K., & Huang, J. X. (2022) · ACM Computing Surveys · doi:10.1145/3534965 · Cited by 80 · see citing works
(2024) · ACM eBooks · doi:10.1145/3674127 · Cited by 82 · see citing works
Episodic Memory in Lifelong Language Learning
d'Autume, C. D. M., Ruder, S., Kong, L., & Yogatama, D. (2019) · arXiv · doi:10.48550/arxiv.1906.01076 · Cited by 95 · see citing works
Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension
Sun, K., Yu, D., Yu, D., & Cardie, C. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00305 · Cited by 142 · see citing works
“…researchers have explored a number of options including adding unanswerable (Trischler et al., 2017; Rajpurkar et al., 2018) or conversational (Choi et al., 2018; Reddy et al., 2019) questions that might require reasoning (Zhang et al., 2018a), and designing abstractive answers (Nguyen et…”
BackgroundTopiOCQA: Open-domain Conversational Question Answering with Topic Switching
Adlakha, V., Dhuliawala, S., Suleman, K., de Vries, H., & Reddy, S. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00471 · Cited by 83 · see citing works
“, 2019) and QuAC (Choi et al., 2018) have facilitated much of the research in this area.”
BackgroundMethodASQA: Factoid Questions Meet Long-Form Answers
Stelmakh, I., Luan, Y., Dhingra, B., & Chang, M.-W. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.566 · Cited by 103 · see citing works
ExT5: Towards Extreme Multi-Task Scaling for Transfer LearningPreprint
Aribandi, V., Tay, Y., Schuster, T., Rao, J., Zheng, H., Mehta, S. V., Zhuang, H., Tran, V. Q., Bahri, D., Ni, J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.10952 · Cited by 94 · see citing works
Towards goal-oriented semantic signal processing: Applications and future challenges
Kalfa, M., Gok, M., Atalık, A., Tegin, B., Duman, T. M., & Arıkan, O. (2021) · Digital Signal Processing · doi:10.1016/j.dsp.2021.103134 · Cited by 74 · see citing works
“Examples include question answering [38–40], captioning of images and videos [41–44], and discourse parsing [45–47].”
BackgroundConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering
Chen, Z., Li, S., Smiley, C., Ma, Z., Shah, S., & Wang, W. Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.421 · Cited by 100 · see citing works
The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents
Shuster, K., Da, J. U., Roller, S., Dinan, E., Boureau, Y.-L., & Weston, J. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.222 · Cited by 72 · see citing works
A Short Survey of Pre-trained Language Models for Conversational AI-A New Age in NLP
Zaib, M., Sheng, Q. Z., & Zhang, W. E. (2020) · Australasian Computer Science Week Multiconference (ACSW) · doi:10.1145/3373017.3373028 · Cited by 72 · see citing works
“High-quality conversational datasets such as QuAC [4] and CoQA [16] have provided the researchers a great source to work deeply in the field of CMC.”
BackgroundLet the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactions
Abbasiantaeb, Z., Yuan, Y., Kanoulas, E., & Aliannejadi, M. (2024) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3616855.3635856 · Cited by 72 · see citing works
“) dataset [11], which serves as a widely recognized benchmark for evaluating the effectiveness of CQA models.”
BackgroundMethodInfluentialGraph Neural Networks for Natural Language Processing: A SurveyPreprint
Wu, L., Chen, Y., Shen, K., Guo, X., Gao, H., Li, S., Pei, J., & Long, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.06090 · Cited by 70 · see citing works
SberQuAD – Russian Reading Comprehension Dataset: Description and Analysis
Efimov, P., Chertok, A., Boytsov, L., & Braslavski, P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58219-7_1 · Cited by 70 · see citing works
Conversational Information Seeking
Dalton, J., Fischer, S., Owoicho, P., Radlinski, F., Rossetto, F., Trippas, J. R., & Zamani, H. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3532678 · Cited by 66 · see citing works
Getting Closer to AI Complete Question Answering: A Set of Prerequisite Real Tasks
Rogers, A., Kovaleva, O., Downey, M., & Rumshisky, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6398 · Cited by 68 · see citing works
“The most interesting alternative was suggested in QuAC: one Turker tries to learn the information from a text that only the other Turker can see (Choi et al. 2018).”
BackgroundTWEETQA: A Social Media Focused Question Answering Dataset
Xiong, W., Wu, J., Wang, H., Kulkarni, V., Yu, M., Chang, S., Guo, X., & Wang, W. Y. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1496 · Cited by 67 · see citing works
“More recently, researchers propose even challenging datasets that require QA within dialogue or conversational context (Reddy et al., 2018; Choi et al., 2018).”
BackgroundFQuAD: French Question Answering Dataset
d’Hoffschmidt, M., Belblidia, W., Heinrich, Q., Brendlé, T., & Vidal, M. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.107 · Cited by 82 · see citing works
Lin, S.-C., Yang, J.-H., Nogueira, R. F., Tsai, M.-F., Wang, C., & Lin, J. (2021) · ACM Transactions on Information Systems · doi:10.1145/3446426 · Cited by 70 · see citing works
“Fortunately, open-domain QA research has produced QuAC [13], a diverse, large-scale dataset that contains conversational natural language questions of exploratory information needs, as well as CANARD [25], a derived conversational question-in-context rewriting dataset with human-generated questions…”
BackgroundChen, Y., Wu, L., & Zaki, M. J. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/171 · Cited by 65 · see citing works
“, 2018) or add previous answer locations to the passage (Choi et al., 2018; Yatskar, 2018), and treat the task as a single-turn MRC while ignoring the important information from the conversation flow.”
BackgroundMethodInfluentialLin, S.-C., Yang, J.-H., Nogueira, R., Tsai, M.-F., Wang, C.-J., & Lin, J. (2020) · arXiv · doi:10.48550/arxiv.2005.02230 · Cited by 62 · see citing works
“Most QA research [26, 40, 50, 56, 59], including conversational QA studies [10, 41], focuses on a restricted version of the open-domain QA problem posed in [8, 18, 20]: returning answers from a finite set of relevant documents—a relevant article [40, 56] or multi-hop hyperlinked documents [58].”
BackgroundMeasuring Attribution in Natural Language Generation Models
Rashkin, H., Nikolaev, V., Lamm, M., Aroyo, L., Collins, M., Das, D., Petrov, S., Tomar, G. S., Turc, I., & Reitter, D. (2023) · Computational Linguistics · doi:10.1162/coli_a_00486 · Cited by 71 · see citing works
“2019), QUAC (Choi et al. 2018), and CAST-19 (Dalton et al.”
MethodMacaw: An Extensible Conversational Information Seeking Platform
Zamani, H., & Craswell, N. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401415 · Cited by 61 · see citing works
SituatedQA: Incorporating Extra-Linguistic Contexts into QA
Zhang, M. J. Q., & Choi, E. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.586 · Cited by 72 · see citing works
Self-citationCareful Selection of Knowledge to Solve Open Book Question Answering
Banerjee, P., Pal, K. K., Mitra, A., & Baral, C. R. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1615 · Cited by 53 · see citing works
“…answering (Rajpurkar et al., 2016; Joshi et al., 2017; Khashabi et al., 2018; Richardson et al., 2013; Lai et al., 2017; Reddy et al., 2018; Choi et al., 2018; Tafjord et al., 2018; Mitra et al., 2019) and many attempts have been made to solve these challenges (Devlin et al., 2018; Vaswani…”
A Qualitative Comparison of CoQA, SQuAD 2.0 and QuAC
Yatskar, M. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1241 · Cited by 56 · see citing works
“In this work, we analyze three such new proposed datasets, SQuAD 2.0 (Rajpurkar et al., 2018), QuAC (Choi et al., 2018), and CoQA (Reddy et al., 2018).”
MethodInfluentialSelf-citationBeyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access
Kim, S., Eric, M., Gopalakrishnan, K., Hedayatnia, B., Liu, Y., & Hakkani-Tür, D. Z. (2020) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2020.sigdial-1.35 · Cited by 57 · see citing works
The role of large language models in agriculture: harvesting the future with LLM intelligence
Shaikh, T. A., Rasool, T., Veningston, K., & Yaseen, S. M. (2024) · Progress in Artificial Intelligence · doi:10.1007/s13748-024-00359-4 · Cited by 61 · see citing works
XPersona: Evaluating Multilingual Personalized Chatbot
Lin, Z., Liu, Z., Winata, G. I., Cahyawijaya, S., Madotto, A., Bang, Y., Ishii, E., & Fung, P. (2021) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2021.nlp4convai-1.10 · Cited by 57 · see citing works
Interactive Question Answering Systems: Literature Review
Biancofiore, G. M., Deldjoo, Y., Noia, T. D., Sciascio, E. D., & Narducci, F. (2024) · ACM Computing Surveys · doi:10.1145/3657631 · Cited by 57 · see citing works
“Initiative system-driven [47, 167], user-driven [31, 110], mixed [26].”
Arabic machine reading comprehension on the Holy Qur’an using CL-AraBERT
Malhas, R., & Elsayed, T. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103068 · Cited by 59 · see citing works
The 10 Research Topics in the Internet of Things
Zhang, W. E., Sheng, Q. Z., Mahmood, A., Tran, D. H., Zaib, M., Hamad, S. A., Aljubairy, A., Alhazmi, A., Sagar, S., & Ma, C. (2020) · IEEE International Conference on Collaboration and Internet Computing (CIC) · doi:10.1109/cic50333.2020.00015 · Cited by 51 · see citing works
“Most of the IoT conversational agents are single-turn agents where they do not keep track of the previous conversation, and thus provide inaccurate answers [80], [81].”
BackgroundIntroducing MANtIS: a novel Multi-Domain Information Seeking Dialogues DatasetPreprint
Penha, G., Bălan, A. O., & Hauff, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.04639 · Cited by 60 · see citing works
“MultiWOZ [4] EMNLP NLP 8,438 3 3 3 3 QuAC [8] EMNLP NLP 13,594 3 3 a 3 3 WoW [11] ICLR ML 22,311 3 3 3 3”
- Authors
- Choi, E., He, H., Iyyer, M., Yatskar, M., Yih, W.-T., Choi, Y., Liang, P., & Zettlemoyer, L.
- Affiliations
- University of Washington (US)
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 2174-2184
- OpenAlex
- W2888302696
- Cited by
- 714
- References
- 28
- Citations by year
- 2026: 20 · 2025: 41 · 2024: 84 · 2023: 105 · 2022: 101 · 2021: 150 · 2020: 115 · 2019: 88 · 2018: 10
- FWCI
- 36.11
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Funders
- Allen Institute for Artificial Intelligence
- Defense Advanced Research Projects Agency
- Army Research Office (W911NF-15-1-, W911NF)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- information-seeking questions, open-ended questions, unanswerable questions, reading comprehension
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
