The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
QuAC: Question Answering in Context
Open accessAbstract
We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total).The dialogs involve two crowd workers: (1) a student who poses a sequence of freeform questions to learn as much as possible about a hidden Wikipedia text, and (2) a teacher who answers the questions by providing short excerpts from the text.QuAC introduces challenges not found in existing machine comprehension datasets: its questions are often more open-ended, unanswerable, or only meaningful within the dialog context, as we show in a detailed qualitative evaluation.We also report results for a number of reference models, including a recently state-ofthe-art reading comprehension architecture extended to model dialog context.Our best model underperforms humans by 20 F1, suggesting that there is significant room for future work on this data.Dataset, baseline, and leaderboard available at http://quac.ai.How was perversion handled?How long was he there?How popular did she become?How did Mark Felt contact Woodword?How did the meeting go?How did it do on the charts?When was she born?When was it founded?When was the breakup?
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Most datasets that are currently available for training question generation systems are crowd-sourced [42, 47, 84, 92, 114, 115, 119, 120, 122, 123], which largely impacts the quality of generated questions.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [114]
In the citing paper
“For question answering (QA), on SQuADv2 [97], QuAC [24], and many other datasets, fine-tuned models have superior performance, while on CoQA [98], LLMs perform as well as fine-tuned models [25].”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 2×
“BiDAF++ We use a re-implementation of a top-performing SQuAD model (Peters et al., 2018) that augments bidirectional attention flow (Seo et al., 2016, BiDAF) with self-attention (Clark and Gardner, 2018) and contextualized embeddings.”
MethodSelf-citationSQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 2×
“Reading Comprehension Our work builds on span based reading comprehension (Rajpurkar et al., 2016; Joshi et al., 2017; Trischler et al., 2016), while also incorporating innovations such as curating questions independently of supporting text to reduce trivial lexical overlap (Joshi et al., 2017;…”
BackgroundSelf-citationYago: a core of semantic knowledge
Suchanek, F. M., Kasneci, G., & Weikum, G. (2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242667 · Cited by 2,512
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Know What You Don’t Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2124 · Cited by 2,240
Self-citationSupervised Learning of Universal Sentence Representations from Natural Language Inference Data
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1070 · Cited by 2,049 · cited 1×
“Pretrained InferSent To test the importance of lexical matching in our dataset, we output the sentence in s whose pretrained InferSent representation (Conneau et al., 2017) has the highest cosine similarity to that of the question.”
MethodBidirectional Attention Flow for Machine ComprehensionPreprint
Seo, M. J., Kembhavi, A., Farhadi, A., & Hajishirzi, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01603 · Cited by 1,698 · cited 1×
“BiDAF++ We use a re-implementation of a top-performing SQuAD model (Peters et al., 2018) that augments bidirectional attention flow (Seo et al., 2016, BiDAF) with self-attention (Clark and Gardner, 2018) and contextualized embeddings.”
MethodTriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1147 · Cited by 1,690 · cited 8×
“adopted in other reading comprehension datasets such as SQuAD, TriviaQA (Joshi et al., 2017), and NewsQA (Trischler et al.”
BackgroundMethodInfluentialSelf-citationDeep Reinforcement Learning for Dialogue Generation
Li, J., Monroe, W., Ritter, A., Jurafsky, D., Galley, M., & Gao, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1127 · Cited by 1,393 · cited 1×
“19 Dialog fits into an increasing interest in open domain dialog, mostly studied in the context of social chit-chat (Li et al., 2016; Ritter et al., 2011; Fang et al., 2017; Ghazvininejad et al., 2018).”
BackgroundAllenNLP: A Deep Semantic Natural Language Processing Platform
Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N. F., Peters, M. E., Schmitz, M., & Zettlemoyer, L. (2018) · Workshop for NLP Open Source Software (NLP-OSS) · doi:10.18653/v1/w18-2501 · Cited by 1,376 · cited 1×
“The AllenNLP (Gardner et al., 2017) implementation we use reaches 82.”
MethodSelf-citation
Flagged first, then by citations
Research works printed but not linked: 3
- Hao Fang, Hao Cheng, Elizabeth Clark, Ariel Holtz- man, Maarten Sap, Mari Ostendorf, Yejin Choi, and Noah A Smith. 2017. Sounding board-university of washingtons alexa prize submission. Alexa Prize Proceedings.
- Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. Conference of the North American Chapter of the Association for Computational Lin- guistics.
- Marzieh Saeidi, Max Bartolo, Patrick Lewis, Sameer Signh, Tim Rocktschel, Mike Sheldon, Guillaume Bouchard, and Sebastian Riedel. 2018. Interpreta- tion of natural language rules in conversational ma- chine reading. In Proceedings of Empirical Methods in Natural Language Processing.
Cited by
Sorted by citations
Items 201–300 of 704
Qi, P., Zhang, Y., & Manning, C. D. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.3 · Cited by 23
Towards Answer-unaware Conversational Question Generation
Nakanishi, M., Kobayashi, T., & Hayashi, Y. (2019) · Proceedings of the 2nd Workshop on Machine Reading for Question Answering · doi:10.18653/v1/d19-5809 · Cited by 21
“Datasets such as CoQA (Reddy et al., 2019) and QuAC (Choi et al., 2018) have been developed to facilitate the relevant research efforts (Yatskar, 2019).”
BackgroundA T5-based interpretable reading comprehension model with more accurate evidence training
Guan, B., Zhu, X., & Yuan, S. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103584 · Cited by 29
Chai, Z., & Wan, X. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.21 · Cited by 25
“In recent years, the appearance of large-scale conversational QA datasets like CoQA (Reddy et al., 2019) and QuAC (Choi et al., 2018) makes it possible to train data-driven SQG models, and the CoQA dataset was widely adopted by prior works.”
BackgroundTransformer-Based Extractive Social Media Question Answering on TweetQA
Butt, S., Ashraf, N., Fahim, H., Sidorov, G., & Gelbukh, A. F. (2021) · Computación y Sistemas · doi:10.13053/cys-25-1-3897 · Cited by 25
“Extractive question-answer datasets such as SQuAD [15], NewsQA [19], TRECQA [20], CoQA [16], SearchQA [6], and QuAC [2] focus on extracting the span of answers from the context or document and evaluate using F1 or ROUGE with the ground truth span.”
BackgroundMethodInfluentialNangia, N., Sugawara, S., Trivedi, H. K., Warstadt, A., Vania, C., & Bowman, S. R. (2021) · doi:10.18653/v1/2021.acl-long.98 · Cited by 29
“QuAC (Choi et al., 2018) is crowdsourced by pairing crowd-”
BackgroundPeng, W., Hu, Y., Yu, J., Xing, L., & Xie, Y. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107364 · Cited by 28
Wu, Z., Lu, B.-R., Hajishirzi, H., & Mari Ostendorf, E. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.140 · Cited by 26
“With the assumption of a long grounding document, our task differs from prior work in conversational question answering (Choi et al., 2018; Reddy et al., 2019), which focuses on answering a sequence of factoid questions about a short text snippet.”
BackgroundMethodBias in Conversational Search: The Double-Edged Sword of the Personalized Knowledge Graph
Gerritse, E. J., Hasibi, F., & de Vries, A. P. (2020) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3409256.3409834 · Cited by 28
“rch engines [13]. Measuring bias in conversational search raises even more challenges, due to the difficulties of evaluating the systems themselves. While test collections like TREC CAST [7] and QuAC [6] exist, they are often restricted to the question-answering part of conversational systems. Evaluating multiple aspe…”
BackgroundE3: Entailment-driven Extracting and Editing for Conversational Machine Reading
Zhong, V., & Zettlemoyer, L. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1223 · Cited by 22
“Recently, there has been growing interest in question answering (QA) in a dialogue setting (Choi et al., 2018; Reddy et al., 2019).”
BackgroundSelf-citationBias in Conversational Search: The Double-Edged Sword of the Personalized Knowledge Graph
Gerritse, E. J., Hasibi, F., de Vries, A. P., & Balog, K. (2020) · Radboud Repository (Radboud University) · Cited by 23
A Comparative Study of Transformer-Based Language Models on Extractive Question AnsweringPreprint
Pearce, K., Zhan, T., Komanduri, A., & Zhan, J. Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03142 · Cited by 53
“[6] 4) CovidQA : CovidQA is a Question Answering dataset consisting of 2,019 different question/answer pairs.”
BackgroundMethodProceedings of the 2nd Workshop on Natural Language Processing for Conversational AI
(2020) · doi:10.18653/v1/2020.nlp4convai-1 · Cited by 23
“It first uses a shared pretrained BERT (Devlin et al., 2018) to encode a context embedding for each user turn, a slot embedding for each slot, and a slot value embedding for each slot value in the candidate list of the categorical slots.”
InfluentialTWEETSUMM - A Dialog Summarization Dataset for Customer Service
Feigenblat, G., Gunasekara, C., Sznajder, B., Joshi, S., Konopnicki, D., & Aharonov, R. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.24 · Cited by 25
NewsPod: Automatic and Interactive News Podcasts
Laban, P., Ye, E., Korlakunta, S., Canny, J., & Hearst, M. A. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511147 · Cited by 26
“In conversational question answering projects such as CoQA [52] and QuAC [7], a model must answer a user’s question as part of a longer, contextualized sequence of questions.”
BackgroundChristmann, P., Roy, R. S., & Weikum, G. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591682 · Cited by 25
MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents
Feng, S., Patel, S. S., Wan, H., & Joshi, S. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.498 · Cited by 44
“In particular, our work is closely related to the recently proposed open-retrieval conversational question answering (OR-CQA) setting for QuAC dataset, i.e., OR-QuAC (Qu et al., 2020b).”
BackgroundMethodInfluentialContinual Adaptation for Efficient Machine Communication
Hawkins, R., Kwon, M., Sadigh, D., & Goodman, N. D. (2020) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2020.conll-1.33 · Cited by 21
RoR: Read-over-Read for Long Document Machine Reading Comprehension
Zhao, J., Bao, J., Wang, Y., Zhou, Y., Wu, Y., He, X., & Zhou, B. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.160 · Cited by 22
LangTest: A comprehensive evaluation library for custom LLM and NLP models
Nazir, A., Chakravarthy, T. K., Cecchini, D., Khajuria, R., Sharma, P., Mirik, A. T., Kocaman, V., & Talby, D. (2024) · Software Impacts · doi:10.1016/j.simpa.2024.100619 · Cited by 22
Liu, Z., Lim, H., Suhaimi, N. F. A., Tong, S. C., Ong, S., Ng, A., Lee, S., Macdonald, M. R., Ramasamy, S., Krishnaswamy, P., et al. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-2004 · Cited by 26
“CoQA (Reddy et al., 2018) and QuAC (Choi et al., 2018) are designed to mimic multi-turn information-seeking discussions of the given material.”
BackgroundFluent Response Generation for Conversational Question Answering
Baheti, A., Ritter, A., & Small, K. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.19 · Cited by 26
Unstructured Text Enhanced Open-Domain Dialogue System: A Systematic Survey
Ma, L., Li, M., Zhang, W., Li, J., & Liu, T. (2021) · ACM Transactions on Information Systems · doi:10.1145/3464377 · Cited by 24
“The differences between DGC/BBC and Conversational QA (such as CoQA [149], QuAC [22] and MANTIS [137]) are the dialogue of DGC/BBC is more diversified (including chit-chat or recommendation) and not limited to QA.”
FQuAD: French Question Answering DatasetPreprint
d’Hoffschmidt, M., Belblidia, W., Brendlé, T., Heinrich, Q., & Vidal, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06071 · Cited by 21
“0 [5] En 150k NewsQA [9] En 100k+ CoQA [6] En 127k+ QuAC [10] En 98k+ HotpotQA [11] En 113k+ SQuAD-es Es 87595 FQuAD Fr 25k+ Table 1: Benchmark of existing Reading Compre-hension datasets, including FQuAD.”
BackgroundInfluentialKnowledge Fusion and Semantic Knowledge Ranking for Open Domain Question AnsweringPreprint
Banerjee, P., & Baral, C. R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.03101 · Cited by 22
“…in recent years for natural language question answering such as (Rajpurkar et al., 2016; Joshi et al., 2017; Khashabi et al., 2018a; Richardson et al., 2013; Lai et al., 2017; Reddy et al., 2018; Choi et al., 2018; Tafjord et al., 2018; Mitra et al., 2019b; Sap et al., 2019; Bisk et al., 2019).”
BackgroundZPR2: Joint Zero Pronoun Recovery and Resolution using Multi-Task Learning and BERT
Song, L., Xu, K., Zhang, Y., Chen, J., & Yu, D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.482 · Cited by 22
“…ambiguity to applications such as machine translation (Nakaiwa and Shirai, 1996; Wang et al., 2016; Takeno et al., 2016), question answering (Choi et al., 2018; Reddy et al., 2019; Sun et al., 2019; [ 警 方 ] 怀 疑 这 是 一 起 黑 枪 案 件 ， φ 1 将 枪 械 和 皮 包 交 送 市 里 φ 2 以 清 理 案 情 。 [ The police ]…”
A Brief Survey of Question Answering Systems
Caballero, M. (2021) · International Journal of Artificial Intelligence & Applications · doi:10.5121/ijaia.2021.12501 · Cited by 21
Luo, X., Liu, D., Kong, H., Huai, S., Chen, H., Xiong, G., & Liu, W. (2024) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3701728 · Cited by 27
MIMOQA: Multimodal Input Multimodal Output Question Answering
Singh, H., Nasery, A., Mehta, D., Agarwal, A., Lamba, J., & Srinivasan, B. V. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.418 · Cited by 23
“While earlier datasets were small, beginning with SQuAD (Rajpurkar et al., 2016) several large datasets (Ra-jpurkar et al., 2018; Yang et al., 2018; Choi et al., 2018; Reddy et al., 2019; Kwiatkowski et al., 2019) have been proposed.”
MethodA New Multi-choice Reading Comprehension Dataset for Curriculum Learning.
Liang, Y., Li, J., & Yin, J. P. (2019) · Asian Conference on Machine Learning · Cited by 18
PersianQuAD: The Native Question Answering Dataset for the Persian Language
Kazemi, A., Mozafari, J., & Nematbakhsh, M. A. (2022) · IEEE Access · doi:10.1109/access.2022.3157289 · Cited by 25
“…Native 100K+ Natural questions: a benchmark for question answering research [10] 2019 English Native 300K+ Quac: Question answering in context [11] 2018 English Native 100K+ Coqa: A conversational question answering challenge [12] 2019 English Native 127K+ Newsqa: A machine comprehension…”
BackgroundMajumder, B. P., Rao, S., Galley, M., & McAuley, J. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.340 · Cited by 29
KACE: Generating Knowledge Aware Contrastive Explanations for Natural Language Inference
Chen, Q., Ji, F., Zeng, X., Li, F., Zhang, J., Chen, H., & Zhang, Y. (2021) · doi:10.18653/v1/2021.acl-long.196 · Cited by 21
“In recent years, pre-trained language models (De-vlin et al., 2019; Liu et al., 2019; Yang et al., 2019) have been widely adopted in many tasks of natu-ral language processing (Talmor et al., 2019; Choi et al., 2018; Bowman et al., 2015).”
MethodBenchmarking Machine Reading Comprehension: A Psychological Perspective
Sugawara, S., Stenetorp, P., & Aizawa, A. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.137 · Cited by 22
Question Rewriting? Assessing Its Importance for Conversational Question Answering
Raposo, G., Ribeiro, R., Martins, B., & Coheur, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-99739-7_23 · Cited by 19
“Conversational question answering extends traditional Question Answering (QA) by involving a sequence of interconnected questions and answers [2].”
BackgroundMG-ShopDial: A Multi-Goal Conversational Dataset for e-Commerce
Bernard, N., & Balog, K. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591883 · Cited by 24
How Far are We from Robust Long Abstractive Summarization?
Koh, H. Y., Ju, J., Zhang, H., Liu, M. Z., & Pan, S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.172 · Cited by 21
Explicit Query Rewriting for Conversational Dense Retrieval
Qian, H., & Dou, Z. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.311 · Cited by 26
“( 3) CA-NARD (Elgohary et al., 2019): The CANARD dataset is derived from the QuAC dataset (Choi et al., 2018).”
MethodAsk to Learn: A Study on Curiosity-driven Question Generation
Scialom, T., & Staiano, J. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.202 · Cited by 24
BERT-CoQAC: BERT-Based Conversational Question Answering in Context
Zaib, M., Tran, D. H., Sagar, S., Mahmood, A., Zhang, W. E., & Sheng, Q. Z. (2021) · Communications in computer and information science · doi:10.1007/978-981-16-0010-4_5 · Cited by 18
“Our model outperforms the baseline methods and BERT-HAE model on all the three metrics i.e. F1, HEQ-Q, and HEQ-D and answer our research questions as follows.”
BackgroundMethodInfluentialNOAHQA: Numerical Reasoning with Interpretable Graph Question Answering Dataset
Zhang, Q., Wang, L., Yu, S., Wang, S., Wang, Y., Jiang, J., & Lim, E. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.350 · Cited by 21
Multi-Task Learning using Dynamic Task Weighting for Conversational Question Answering
Kongyoung, S., Macdonald, C., & Ounis, I. (2020) · doi:10.18653/v1/2020.scai-1.3 · Cited by 16
“Recently, the advancement in neural language modeling such as BERT (Devlin et al., 2019), and the introduction of two large-scale datasets, namely CoQA (Reddy et al., 2019) and QuAC (Choi et al., 2018) have further boosted research in the ConvQA task.”
BackgroundMethodInfluentialDitch the Gold Standard: Re-evaluating Conversational Question Answering
Li, H., Gao, T., Goenka, M., & Chen, D. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.555 · Cited by 22
Dynamic Contexts for Generating Suggestion Questions in RAG Based Conversational Systems
Tayal, A., & Tyagi, A. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3651905 · Cited by 19
SD-QA: Spoken Dialectal Question Answering for the Real World
Faisal, F., Keshava, S., Alam, M. M. I., & Anastasopoulos, A. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.281 · Cited by 26
Novel regularization method for the class imbalance problem
Kim, B., Ko, Y., & Seo, J. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115974 · Cited by 17
Review Conversational Reading ComprehensionPreprint
Xu, H., Liu, B., Shu, L., & Yu, P. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00821 · Cited by 24
“Inspired by conversational reading comprehension (CRC) ( Reddy et al., 2018; Choi et al., 2018; 1 The dataset is available at https://github.com/howardhsu/RCRC .”
MethodXLTime: A Cross-Lingual Knowledge Transfer Framework for Temporal Expression Extraction
Cao, Y., Groves, W., Saha, T. K., Tetreault, J., Jaimes, A., Peng, H. M., & Yu, P. S. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.148 · Cited by 17
Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading
Gao, Y., Wu, C.-S., Joty, S., Xiong, C., Socher, R., King, I., Lyu, M. R.-T., & Hoi, S. C. H. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.88 · Cited by 18
Interactive Classification by Asking Informative Questions
Yu, L., Chen, H., Wang, S. I., Lei, T., & Artzi, Y. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.237 · Cited by 25
Conversing by Reading: Contentful Neural Conversation with On-demand Machine ReadingPreprint
Qin, L., Galley, M., Brockett, C., Liu, X., Gao, X., Dolan, B., Choi, Y., & Gao, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02738 · Cited by 16
“Questions are treated as entirely independent in these “single-turn” MRC models, so recent work (e.g., CoQA (Reddy et al., 2019) and QuAC (Choi et al., 2018)) focuses on multi-turn MRC, modeling sequences of questions and answers in a conversation.”
BackgroundSelf-citationSchlegel, V., Nenadić, G., & Batista-Navarro, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.14709 · Cited by 15
“…et al., 2017) 2018 PoiReviewQA (Mai et al., 2018), NarrativeQA (Kočiský et al., 2018), ReCoRd (Zhang et al., 2018), ARC (Clark et al., 2018), QuAC (Choi et al., 2018), emrQA (Pampari et al., 2018), ProPara (Dalvi et al., 2018), MedHop (Welbl et al., 2018), OBQA (Mihaylov et al., 2018), BioASQ…”
InfluentialRAST: Domain-Robust Dialogue Rewriting as Sequence Tagging
Hao, J., Song, L., Wang, L., Xu, K., Tu, Z., & Yu, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.402 · Cited by 23
“Recent years have witnessed increasing attention in conversation-based tasks, such as conversational question answering (Choi et al., 2018; Reddy et al., 2019; Sun et al., 2019), dialogue response generation (Li et al., 2017; Zhang et al., 2018; Wu et al., 2019; Zhou et al., 2020), dialogue state…”
BackgroundZero-shot Query Contextualization for Conversational Search
Krasakis, A. M., Yates, A., & Kanoulas, E. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531769 · Cited by 19
“Understanding the semantics of language has been empowered by the availability of large-scale datasets in a variety of tasks [3, 20, 29], which are lacking when it comes to conversational retrieval.”
BackgroundSearch-Oriented Conversational Query Editing
Mao, K., Dou, Z., Liu, B., Qian, H., Mo, F., Wu, X., Cheng, X., & Cao, Z. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.256 · Cited by 15
“…dataset for conversational question answering, which contains 14K information-seeking conversations with 80K query-answer pairs originated from the training set of CAsT-19 (Dalton et al., 2020), QuAC (Choi et al., 2018), and NQ (Choi et al., 2018) with manually generated follow-up queries.”
MethodDoc2Dial: A Framework for Dialogue Composition Grounded in Documents
Feng, S., Fadnis, K., Liao, Q. V., & Lastras, L. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i09.7089 · Cited by 15
“Our work is also largely related to conversational QA such as CoQA (Reddy, Chen, and Manning 2019) and QuAC (Choi et al. 2018).”
MethodMulti-task joint training model for machine reading comprehension
Li, F., Shan, Y., Mao, X., Ren, X., Liu, X., & Zhang, S. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.02.082 · Cited by 15
Probing Prior Knowledge Needed in Challenging Chinese Machine Reading Comprehension.Preprint
Sun, K., Yu, D., Yu, D., & Cardie, C. (2019) · arXiv (Cornell University) · Cited by 15
Vakulenko, S., Longpre, S., Tu, Z., & Anantha, R. (2020) · Proceedings of the 5th International Workshop on Search-Oriented Conversational AI (SCAI) · doi:10.18653/v1/2020.scai-1.2 · Cited by 14
CREPE: Open-Domain Question Answering with False Presuppositions
Yu, X., Min, S., Zettlemoyer, L., & Hajishirzi, H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.583 · Cited by 35
Self-citationSaving Dense Retriever from Shortcut Dependency in Conversational Search
Kim, S., & Kim, G. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.701 · Cited by 21
Shaikh, T. A., Rasool, T., & Mir, W. A. (2025) · Computer Standards & Interfaces · doi:10.1016/j.csi.2025.104005 · Cited by 18
Bernasconi, E., Redavid, D., & Ferilli, S. (2025) · Electronics · doi:10.3390/electronics14071481 · Cited by 14
Question Answering with BERT: designing a 3D virtual avatar for Cultural Heritage exploration
Farella, M., Chiazzese, G., & Bosco, G. L. (2022) · IEEE Mediterranean Electrotechnical Conference (MELECON) · doi:10.1109/melecon53508.2022.9843028 · Cited by 15
Improving zero-shot action recognition using human instruction with text description
Wu, N., Kera, H., & Kawamoto, K. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04808-w · Cited by 14
“Previously, question answering (Q&A) systems were mainly text-based Q&A [35, 36].”
BackgroundLearning to Identify Follow-Up Questions in Conversational Question Answering
Kundu, S., Lin, Q., & Ng, H. T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.90 · Cited by 16
“To create each instance in LIF from QuAC, we take the associated passage, the previous question-answer pairs till it says should ask a follow-up question, and the next question as the gold valid candidate follow-up question.”
BackgroundMethodInfluentialOtegi, A., Campos, J. A., Azkune, G., Soroa, A., & Agirre, E. (2020) · Workshop on NLP for COVID-19 (NLP-COVID19) · doi:10.18653/v1/2020.nlpcovid19-2.15 · Cited by 15
“The automatic evaluation revealed that the submitted system was not optimal, contradicting some design choices, such as using the QuAC dataset (Choi et al., 2018) to fine-tune the QA system in addition to the SQuAD 2.”
BackgroundMethodInfluentialWeakly-Supervised Open-Retrieval Conversational Question Answering
Qu, C., Liu, Y., Chen, C., Croft, W. B., Krishna, K., & Iyyer, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-72113-8_35 · Cited by 13
Self-citationConversational Entity Linking: Problem Definition and Datasets
Joko, H., Hasibi, F., Balog, K., & de Vries, A. P. (2021) · NORA - Norwegian Open Research Archives · Cited by 13
Automatic Essay Scoring: A Review on the Feature Analysis Techniques
Chassab, R. H., Zakaria, L. Q., & Tiun, S. B. (2021) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2021.0121028 · Cited by 20
MIMICS: A Large-Scale Data Collection for Search ClarificationPreprint
Zamani, H., Lueck, G., Chen, E., Quispe, R., Luu, F., & Craswell, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10174 · Cited by 13
“user interaction signals, and coverage. It is worth noting that a number of datasets related to conversational search has recently been created and released. They include CCPE-M [21], CoQA [25], QuAC [7], MISC [27], and the Conversation Assistance Track data created in TREC 2019 [11]. Although these datasets do not pa…”
BackgroundOpen-Retrieval Conversational Machine ReadingPreprint
Gao, Y., Li, J., Wu, C.-S., Lyu, M. R.-T., & King, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.08633 · Cited by 17
“Chen et al. [25] firstly introduces an open-domain setting for answering factoid conversational questions in QuAC [9].”
BackgroundInformation retrieval and question answering: A case study on COVID-19 scientific literature
Otegi, A., Vicente, I. S., Saralegi, X., Peñas, A., Lozano, B., & Agirre, E. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.108072 · Cited by 17
“Therefore, we need to assess our hypothesis that using QuAC [56] in addition to SQuAD 2.0 when fine-tuning SciBERT will improve system results, as QuAC is a conversational QA dataset containing a higher rate of non-factoid questions than SQuAD.”
InfluentialEvent-centric hierarchical hyperbolic graph for multi-hop question answering over knowledge graphs
Zhu, X., Gao, W., Li, T., Yao, W., & Deng, H. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.107971 · Cited by 12
Reference Knowledgeable Network for Machine Reading Comprehension
Zhao, Y., Zhang, Z., & Zhao, H. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3164219 · Cited by 13
Bi-directional CognitiveThinking Network for Machine Reading Comprehension
Peng, W., Hu, Y., Xing, L., Xie, Y., Yu, J., Sun, Y., & Wei, X. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.235 · Cited by 13
Biomedical Question Answering: A Survey of Methods and Datasets
Kaddari, Z., Mellah, Y., Berrich, J., Bouchentouf, T., & Belkasmi, M. G. (2020) · International Conference on Intelligent Computing in Data Sciences (ICDS) · doi:10.1109/icds50568.2020.9268742 · Cited by 12
Conversational Entity Linking: Problem Definition and Datasets
Joko, H., Hasibi, F., Balog, K., & de Vries, A. P. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3463258 · Cited by 21
“We made an extensive list of around 130 datasets, 2 extracted from ParlAI [43] and other dataset comparison lists [16, 36, 44].”
MethodInfluentialYou Don’t Need Labeled Data for Open-Book Question Answering
Gholami, S., & Noori, M. (2021) · Applied Sciences · doi:10.3390/app12010111 · Cited by 20
Multi-tasking Dialogue Comprehension with Discourse ParsingPreprint
He, Y., Zhang, Z., & Zhao, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03269 · Cited by 13
Socratic Question Generation: A Novel Dataset, Models, and Evaluation
Ang, B. H., Gollapalli, S. D., & Ng, S.-K. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.12 · Cited by 13
“, 2018), QuAC (Choi et al., 2018), and CoQA (Reddy et al.”
BackgroundMethodMultilingual Chatbot for Indian Languages
Singh, U., Vora, N., Lohia, P., Sharma, Y., Bhatia, A., & Tiwari, K. (2023) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt56998.2023.10307978 · Cited by 11
“For the purpose of context question-answering, the BERT model is fine-tuned on datasets such as SQuAD [18], Coqa [19], QuAC [20], etc.”
MethodKostric, I., & Balog, K. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657933 · Cited by 12
“The dialogues are based on questions from QuAC [5], TREC CAsT 2019 [6], and Google Natural Questions (NQ) [14], with TREC CAsT appearing only in the test set.”
Srivastava, A., & Memon, A. M. (2024) · IEEE Access · doi:10.1109/access.2024.3446854 · Cited by 14
“It is an evaluation metric that has been used in conversational question systems such as QuAC [53].”
BackgroundMethodInfluentialVisual Robustness Benchmark for Visual Question Answering (VQA)
Ishmam, M. F., Tashdeed, I., Saadat, T. A., Ashmafee, M. H., Kamal, A. R. M., & Hossain, M. A. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00645 · Cited by 21
“Visual Question Answering (VQA) involves answering any question based on a visual input provided as the context [5], extending the problem of contextual question answering [14].”
Evaluation of Question Answering Systems: Complexity of Judging a Natural Language
Farea, A., Yang, Z., Duong, K., Perera, N., & Emmert‐Streib, F. (2025) · ACM Computing Surveys · doi:10.1145/3744663 · Cited by 32
mt RAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems
Katsis, Y., Rosenthal, S., Fadnis, K., Gunasekara, C., Lee, Y., Popa, L., Shah, V., Zhu, H., Contractor, D., & Danilevsky, M. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.19 · Cited by 60
Conversational Machine Reading Comprehension for Vietnamese Healthcare Texts
Luu, S. T., Bui, M. N., Nguyen, L. D., Tran, K. V., Nguyen, K. V., & Nguyen, N. L.-T. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-88113-9_44 · Cited by 13
Ontology-based semantic data interestingness using BERT models
Abhilash, C. B., Mahesh, K., & Sanda, N. (2023) · Connection Science · doi:10.1080/09540091.2023.2190499 · Cited by 12
“For COVID-19-related question answering, cosine similarity measures are applied on the word embedding for categorising the top-K documents (Choi et al., 2018; Guo et al., 2020; Shen et al., 2020).”
MethodEkram, S. M. S., Rahman, A. A., Altaf, M. S., Islam, M. S., Rahman, M. M., Rahman, M. M., Hossain, M. A., & Kamal, A. R. M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.186 · Cited by 13
“Another conversational dataset is QuAC (Choi et al., 2018), where in a sample, a student asks a question about a hidden Wikipedia passage,”
BackgroundCopyBERT: A Unified Approach to Question Generation with Self-Attention
Varanasi, S., Amin, S., & Neumann, G. (2020) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2020.nlp4convai-1.3 · Cited by 12
“With more question answering (QA) datasets like SQuAD (Rajpurkar et al., 2016) that have been released recently (Trischler et al., 2016; Choi et al., 2018; Reddy et al., 2019; Yang et al., 2018), there has been an increased interest in QG, as these datasets can not only be used for creating QA…”
BackgroundDaNetQA: A Yes/No Question Answering Dataset for the Russian Language
Glushkova, T., Machnev, A., Fenogenova, A., Shavrina, T., Artemova, E., & Ignatov, D. I. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-72610-2_4 · Cited by 13
“most cases. To tackle this problem, we aim to nd an optimal predicted probability threshold for validation dataset. This is done in two steps: 1. For each threshold t in some discrete subset of range [0;1], \yes" answer precision(t) and recall(t) calculated; 2. F 1(t) = 2 precision(t) recall(t) precision(t) + rec…”
BackgroundMethodKuthy, K. D., Kannan, M., Ponnusamy, H. S., & Meurers, D. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.509 · Cited by 12
“In current research approaching QG in the context of question answering (QA), QA corpora such as SQuAD (Rajpurkar et al., 2016), Coqa (Reddy et al., 2019), or Quac (Choi et al., 2018) are typically used to train and evaluate the NQG models.”
MethodLLMs Get Lost In Multi-Turn Conversation
Laban, P., Hayashi, H., Zhou, Y., & Neville, J. (2025) · arXiv · Cited by 419
“Existing studies implemented such user simulation in different ways: relying on templates [12, 68, 39, 16], using an LLM [63, 46, 7, 48], involving human annotators [21, 7], or real users as part of a study [67, 38, 11].”
CAiRE in DialDoc21: Data Augmentation for Information Seeking Dialogue System
Xu, Y., Ishii, E., Winata, G. I., Lin, Z., Madotto, A., Liu, Z., Xu, P., & Fung, P. (2021) · doi:10.18653/v1/2021.dialdoc-1.6 · Cited by 11
“, 2019), QuAC (Choi et al., 2018), and DoQA (Campos et al.”
UNK-VQA: A Dataset and a Probe Into the Abstention Ability of Multi-Modal Large Models
Guo, Y., Jiao, F., Shen, Z., Nie, L., & Kankanhalli, M. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3437288 · Cited by 18
“Thesedatasetscovervariousaspects,including single-round QA [55], multi-round dialogues [56], [57], more challenging binary questions [58], and cloze translation [59].”
BackgroundGu, Y., Gui, X., & Li, D. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.10.107 · Cited by 10
Constructing Arabic Reading Comprehension Datasets: Arabic WikiReading and KaifLematha
Albilali, E., Al-Twairesh, N., & Hosny, M. (2022) · Language Resources and Evaluation · doi:10.1007/s10579-022-09577-5 · Cited by 14
“In QuAC (Choi et al., 2018), the questioner inquires on hidden Wikipedia text, and receives a text excerpt from the answerer.”
BackgroundŁajewska, W., & Balog, K. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615132 · Cited by 11
“Relevant annotations efforts include QuaC [4], which is a dataset”
Background
Metadata
- Authors
- Choi, E., He, H., Iyyer, M., Yatskar, M., Yih, W.-T., Choi, Y., Liang, P., & Zettlemoyer, L.
- Affiliations
- University of Washington (US)
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 2174-2184
- OpenAlex
- W2888302696
- Cited by
- 714
- References
- 28
- Citations by year
- 2026: 20 · 2025: 41 · 2024: 84 · 2023: 105 · 2022: 101 · 2021: 150 · 2020: 115 · 2019: 88 · 2018: 10
- FWCI
- 36.11
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Funders
- Allen Institute for Artificial Intelligence
- Defense Advanced Research Projects Agency
- Army Research Office (W911NF-15-1-, W911NF)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- information-seeking questions, open-ended questions, unanswerable questions, reading comprehension
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
