DialFact: A Benchmark for Fact-Checking in Dialogue
Open accessAbstract
Fact-checking is an essential tool to mitigate the spread of misinformation and disinformation.We introduce the task of fact-checking in dialogue, which is a relatively unexplored area.We construct DIALFACT, a testing benchmark dataset of 22,245 annotated conversational claims, paired with pieces of evidence from Wikipedia.There are three sub-tasks in DIALFACT: 1) Verifiable claim detection task distinguishes whether a response carries verifiable factual information; 2) Evidence retrieval task retrieves the most relevant Wikipedia snippets as evidence; 3) Claim verification task predicts a dialogue response to be supported, refuted, or not enough information.We found that existing fact-checking models trained on non-dialogue data like FEVER (Thorne et al., 2018) fail to perform well on our task, and thus, we propose a simple yet data-efficient solution to effectively improve fact-checking performance in dialogue.We point out unique challenges in DIALFACT such as handling the colloquialisms, coreferences and retrieval ambiguities in the error analysis to shed light on future research in this direction 1 .Dialogue Context: I have family in Ireland!Have you ever been there?Evidence: Ireland is an island in the North Atlantic.Non-Verifiable Response: I haven't been but want to!Verifiable Supported Response: I haven't.It is an island in the north Atlantic right?Verifiable Refuted Response: I haven't been.Isn't it somewhere in north Pacific?Verifiable NEI Response: I haven't been.I heard it's the most popular tourist location in Europe!
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior hallucination evaluation benchmarks are limited to single tasks (e.g., dialogue) and small models (e.g., DPR).
“Though these benchmarks can serve as decent evaluation platform, they are penurious in only focusing on single tasks (e.g., dialogue) and small models (e.g., DPR).”
From HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models (2023) · cites this paper as Gupta et al., 2022
The Task Background and Related Work sections describe DialFact as focused on dialogue fact-checking, so it supports the dialogue-specific example. But Section 5 lists three subtasks, and Section 5.2.1 treats DPR as an evidence-retrieval method; the paper does not establish that prior benchmarks focus only on single tasks or small models. (LLM, conf=0.82)
Experimental — accuracy not yet measuredAnother line of work focuses on evaluating the hallucination of models in different NLP tasks.
“Another line of work focusing on evaluating the hallucination of models in different NLP tasks”
From HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models (2023) · cites this paper as Gupta et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Kilgarriff, A. (2000) · Language · doi:10.2307/417141 · Cited by 11,595 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575 · see citing works
"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection
Wang, W. Y. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-2067 · Cited by 1,822 · see citing works
Shu, K., Mahudeswaran, D., Wang, S., Lee, D., & Liu, H. (2018) · Big Data · doi:10.1089/big.2020.0062 · Cited by 1,362 · see citing works
FEVER: a Large-scale Dataset for Fact Extraction and VERification
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1074 · Cited by 1,343 · see citing works
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304 · see citing works
AllenNLP: A Deep Semantic Natural Language Processing Platform
Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N. F., Peters, M. E., Schmitz, M., & Zettlemoyer, L. (2018) · Workshop for NLP Open Source Software (NLP-OSS) · doi:10.18653/v1/w18-2501 · Cited by 1,273 · see citing works
A Survey of Data Augmentation Approaches for NLP
Feng, S. Y., Gangal, V., Wei, J., Chandar, S., Vosoughi, S., Mitamura, T., & Hovy, E. H. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.84 · Cited by 802 · see citing works
Recipes for Building an Open-Domain Chatbot
Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Smith, E. M., Boureau, Y.-L., et al. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.24 · Cited by 784 · see citing works
A Stylometric Inquiry into Hyperpartisan and Fake News
Potthast, M., Kiesel, J., Reinartz, K., Bevendorff, J., & Stein, B. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1022 · Cited by 700 · see citing works
Towards a Human-like Open-Domain Chatbot
Adiwardana, D., Luong, M.-T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R., Yang, Z., Kulshreshtha, A., Nemade, G., Lu, Y., et al. (2020) · arXiv · doi:10.48550/arxiv.2001.09977 · Cited by 614 · see citing works
Wizard of Wikipedia: Knowledge-Powered Conversational agentsPreprint
Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., & Weston, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01241 · Cited by 610 · see citing works
Retrieval Augmentation Reduces Hallucination in Conversation
Shuster, K., Poff, S., Chen, M., Kiela, D., & Weston, J. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.320 · Cited by 575 · see citing works
A Survey on Automated Fact-Checking
Guo, Z., Schlichtkrull, M., & Vlachos, A. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00454 · Cited by 500 · see citing works
(2020) · arXiv (Cornell University) · Cited by 461 · see citing works
Fact or Fiction: Verifying Scientific Claims
Wadden, D., Lin, S., Lo, K., Wang, L. L., van Zuylen, M., Cohan, A., & Hajishirzi, H. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.609 · Cited by 382 · see citing works
KILT: a Benchmark for Knowledge Intensive Language Tasks
Petroni, F., Piktus, A., Fan, A., Lewis, P., Yazdani, M., De Cao, N., Thorne, J., Jernite, Y., Karpukhin, V., Maillard, J., et al. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.200 · Cited by 357 · see citing works
WinoGrande: an adversarial winograd schema challenge at scale
Sakaguchi, K., Bras, R. L., Bhagavatula, C., & Choi, Y. (2021) · Communications of the ACM · doi:10.1145/3474381 · Cited by 356 · see citing works
Dialogue Natural Language Inference
Welleck, S., Weston, J., Szlam, A., & Cho, K. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1363 · Cited by 249 · see citing works
TabFact: A Large-scale Dataset for Table-based Fact Verification
Chen, W., Wang, H., Chen, J., Zhang, Y., Wang, H., Li, S., Zhou, X., & Wang, W. Y. (2019) · arXiv · doi:10.48550/arxiv.1909.02164 · Cited by 239 · see citing works
Fine-grained Fact Verification with Kernel Graph Attention Network
Liu, Z., Xiong, C., Sun, M., & Liu, Z. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.655 · Cited by 220 · see citing works
Self-citationGEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification
Zhou, J., Han, X., Yang, C. H., Liu, Z., Wang, L., Li, C., & Sun, M. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1085 · Cited by 206 · see citing works
Internet-Augmented Dialogue Generation
Komeili, M., Shuster, K., & Weston, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.579 · Cited by 192 · see citing works
Beyond Goldfish Memory: Long-Term Open-Domain Conversation
Xu, J., Szlam, A., & Weston, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.356 · Cited by 188 · see citing works
UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification
Hanselowski, A., Zhang, H., Li, Z., Sorokin, D., Schiller, B., Schulz, C., & Gurevych, I. (2018) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/w18-5516 · Cited by 184 · see citing works
sense2vec - A Fast and Accurate Method for Word Sense Disambiguation In Neural Word EmbeddingsPreprint
Trask, A., Michalak, P., & Liu, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06388 · Cited by 175 · see citing works
Coreferential Reasoning Learning for Language Representation
Ye, D., Lin, Y., Du, J., Liu, Z., Li, P., Sun, M., & Liu, Z. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.582 · Cited by 174 · see citing works
Proactive Human-Machine Conversation with Explicit Conversation Goal
Wu, W., Guo, Z., Zhou, X., Wu, H. R., Zhang, X., Lian, R., & Wang, H. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1369 · Cited by 173 · see citing works
Towards Debiasing Fact Verification Models
Schuster, T., Shah, D., Yeo, Y. J. S., Roberto Filizzola Ortiz, D., Santus, E., & Barzilay, R. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1341 · Cited by 171 · see citing works
Beyond Goldfish Memory: Long-Term Open-Domain Conversation
Xu, J., Szlam, A., & Weston, J. (2021) · arXiv · doi:10.48550/arxiv.2107.07567 · Cited by 148 · see citing works
Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence
Schuster, T., Fisch, A., & Barzilay, R. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.52 · Cited by 126 · see citing works
HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verification
Jiang, Y., Bordia, S., Zhong, Z., Dognin, C., Singh, M., & Bansal, M. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.309 · Cited by 121 · see citing works
Honovich, O., Choshen, L., Aharoni, R., Neeman, E., Szpektor, I., & Abend, O. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.619 · Cited by 113 · cited 4× · see citing works
“In dialogue domain, related work either focus on evaluating factual consistency (Honovich et al., 2021; Qin et al., 2021) or consistent response generation (Rashkin et al.”
BackgroundInfluentialCOVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic
Saakyan, A., Chakrabarty, T., & Muresan, S. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.165 · Cited by 91 · see citing works
Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding
Dziri, N., Madotto, A., Zaïane, O., & Bose, A. J. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.168 · Cited by 81 · see citing works
Nakov, P., Martino, G. D. S., Elsayed, T., Barrón‐Cedeño, A., Míguez, R., Shaar, S., Alam, F., Haouari, F., Hasanain, M., Babulkov, N., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-72240-1_75 · Cited by 80 · see citing works
Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features
Rashkin, H., Reitter, D., Tomar, G. S., & Das, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.58 · Cited by 74 · see citing works
I like fish, especially dolphins: Addressing Contradictions in Dialogue Modeling
Nie, Y., Williamson, M., Bansal, M., Kiela, D., & Weston, J. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.134 · Cited by 58 · see citing works
FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured informationPreprint
Aly, R., Guo, Z., Schlichtkrull, M., Thorne, J., Vlachos, A., Christodoulopoulos, C., Cocarascu, O., & Mittal, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05707 · Cited by 57 · see citing works
A Review on Fact Extraction and Verification
Bekoulis, G., Papagiannopoulou, C., & Deligiannis, N. (2023) · ACM Computing Surveys · doi:10.1145/3485127 · Cited by 48 · see citing works
A Survey on Automated Fact-CheckingPreprint
Guo, Z., Schlichtkrull, M., & Vlachos, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.11896 · Cited by 42 · see citing works
Evidence-based Factual Error Correction
Thorne, J., & Vlachos, A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.256 · Cited by 39 · see citing works
DeSePtion: Dual Sequence Prediction and Adversarial Examples for Improved Fact-Checking
Hidey, C., Chakrabarty, T., Alhindi, T., Varia, S., Krstovski, K., Diab, M., & Muresan, S. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.761 · Cited by 39 · see citing works
Team DOMLIN: Exploiting Evidence Enhancement for the FEVER Shared Task
Stammbach, D., & Neumann, G. (2019) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/d19-6616 · Cited by 37 · see citing works
(2019) · doi:10.18653/v1/d19-1292 · Cited by 36 · see citing works
Generating Label Cohesive and Well-Formed Adversarial Claims
Atanasova, P., Wright, D., & Augenstein, I. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.256 · Cited by 36 · see citing works
How Robust are Fact Checking Systems on Colloquial Claims?
Kim, B., Kim, H., Hong, S., & Kim, G. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.121 · Cited by 22 · see citing works
GEM: Generative Enhanced Model for adversarial attacks
Niewiński, P., Pszona, M., & Janicka, M. (2019) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/d19-6604 · Cited by 22 · see citing works
FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured information
Aly, R., Guo, Z., Schlichtkrull, M., Thorne, J. P., Vlachos, A., Christodoulopoulos, C., Cocarascu, O., & Mittal, A. (2021) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.4911507 · Cited by 20 · see citing works
DanFEVER: claim verification dataset for Danish
Nørregaard, J., & Derczynski, L. (2021) · DSpace repository (University of Tartu) · doi:10.6084/m9.figshare.14380970 · Cited by 19 · see citing works
Managing ambiguities across utterances in dialogue
DeVault, D., & Stone, M. (2007) · Cited by 19 · see citing works
Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation
Gupta, P., Tsvetkov, Y., & Bigham, J. P. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.338 · Cited by 16 · see citing works
Self-citation(2021) · arXiv (Cornell University) · Cited by 13 · see citing works
Plot-guided Adversarial Example Construction for Evaluating Open-domain Story Generation
Ghazarian, S., Liu, Z., Akash, S., Weischedel, R., Galstyan, A., & Peng, N. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.343 · Cited by 12 · see citing works
Qin, L., Xie, T., Huang, S., Chen, Q., Xiao, X., & Che, W. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.182 · Cited by 10 · see citing works
Automatic Fact-Guided Sentence Modification
Shah, D., Schuster, T., & Barzilay, R. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6406 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- George A Miller. 1998. WordNet: An electronic lexical database. MIT press.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
Cited by
Sorted by citations
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · see citing works
HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
Li, J., Cheng, X., Zhao, X., Nie, J.-Y., & Wen, J.-R. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.397 · Cited by 343 · see citing works
Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10896-y · Cited by 141 · see citing works
AlignScore: Evaluating factual consistency with a unified alignment function
Zha, Y., Yang, Y., Li, R., & Hu, Z. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.634 · Cited by 81 · see citing works
Irfan, B., Kuoppamäki, S., Hosseini, A., & Skantze, G. (2025) · Autonomous Robots · doi:10.1007/s10514-025-10190-y · Cited by 75 · see citing works
Measuring Attribution in Natural Language Generation Models
Rashkin, H., Nikolaev, V., Lamm, M., Aroyo, L., Collins, M., Das, D., Petrov, S., Tomar, G. S., Turc, I., & Reitter, D. (2023) · Computational Linguistics · doi:10.1162/coli_a_00486 · Cited by 71 · see citing works
TRUE: Re-evaluating Factual Consistency Evaluation
Honovich, O., Aharoni, R., Herzig, J., Taitelbaum, H., Kukliansy, D., Cohen, V., Scialom, T., Szpektor, I., Hassidim, A., & Matias, Y. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.287 · Cited by 75 · see citing works
FaithDial : A Faithful Benchmark for Information-Seeking Dialogue
Dziri, N., Kamalloo, E., Milton, S., Zai͏̈ane, O. R., Yu, M., Ponti, E. M., & Reddy, S. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00529 · Cited by 68 · see citing works
TRUE: Re-evaluating Factual Consistency Evaluation
Honovich, O., Aharoni, R., Herzig, J., Taitelbaum, H., Kukliansy, D., Cohen, V., Scialom, T., Szpektor, I., Hassidim, A., & Matias, Y. (2022) · Proceedings of the Second DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering · doi:10.18653/v1/2022.dialdoc-1.19 · Cited by 56 · see citing works
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark
Dziri, N., Rashkin, H., Linzen, T., & Reitter, D. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00506 · Cited by 67 · see citing works
Semnani, S. J., Yao, V., Zhang, H., & Lam, M. S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.157 · Cited by 52 · see citing works
InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning
Gupta, P., Jiao, C., Yeh, Y.-T., Mehri, S., Eskénazi, M., & Bigham, J. P. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.33 · Cited by 46 · see citing works
Self-citationMissing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation
Glockner, M., Hou, Y., & Gurevych, I. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.397 · Cited by 34 · see citing works
WiCE: Real-world entailment for claims in Wikipedia
Kamoi, R., Goyal, T., Rodriguez, J., & Durrett, G. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.470 · Cited by 26 · see citing works
A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detection
Yang, S., Sun, R., & Wan, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.256 · Cited by 32 · see citing works
Diving Deep into Modes of Fact Hallucinations in Dialogue Systems
Das, S., Saha, S., & Srihari, R. K. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.48 · Cited by 27 · see citing works
Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters
Schuster, T., Chen, S., Buthpitiya, S., Fabrikant, A., & Metzler, D. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.28 · Cited by 17 · see citing works
Target-Guided Dialogue Response Generation Using Commonsense and Data Augmentation
Gupta, P., Jhamtani, H., & Bigham, J. P. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.97 · Cited by 18 · see citing works
Self-citationMeasuring Attribution in Natural Language Generation Models
Rashkin, H., Nikolaev, V. A., Lamm, M., Aroyo, L. M., Collins, M., Das, D., Petrov, S., Tomar, G. S., Turc, I., & Reitter, D. (2023) · Computational Linguistics · doi:10.1162/coli_a_00490 · Cited by 16 · see citing works
Eliciting Knowledge from Large Pre-Trained Models for Unsupervised Knowledge-Grounded Conversation
Li, Y., Zhao, J., Lyu, M. R.-T., & Wang, L. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.721 · Cited by 14 · see citing works
Leveraging Large Language Models for Automated Dialogue Analysis
Finch, S. E., Paek, E. S., & Choi, J. D. (2023) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2023.sigdial-1.20 · Cited by 12 · see citing works
Evaluating LLM-based Agents for Multi-turn Conversations: A Survey
Guan, S., Wang, J., 渡辺, 将., Zhu, B., Lou, J., & Xiong, H. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3793671 · Cited by 28 · see citing works
Evaluating Synthetic Data Generation from User Generated Text
Chim, J., Ive, J., & Liakata, M. (2024) · Computational Linguistics · doi:10.1162/coli_a_00540 · Cited by 10 · see citing works
Don’t Forget Your ABC’s: Evaluating the State-of-the-Art in Chat-Oriented Dialogue Systems
Finch, S. E., Finch, J. D., & Choi, J. D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.839 · Cited by 10 · see citing works
SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation
Clark, E., Rijhwani, S., Gehrmann, S., Maynez, J., Aharoni, R., Nikolaev, V. A., Sellam, T., Siddhant, A., Das, D., & Parikh, A. P. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.584 · Cited by 12 · see citing works
CDConv: A Benchmark for Contradiction Detection in Chinese Conversations
Zheng, C., Zhou, J., Zheng, Y., Peng, L., Guo, Z., Wu, W., Niu, Z.-Y., Wu, H., & Huang, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.2 · Cited by 9 · see citing works
Xue, B., Wang, W., Wang, H., Mi, F., Wang, R., Wang, Y., Shang, L., Jiang, X., Liu, Q., & Wong, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.525 · Cited by 11 · see citing works
Fast and Accurate Factual Inconsistency Detection Over Long Documents
Lattimer, B. M., Chen, P. M., Zhang, X., & Yang, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.105 · Cited by 10 · see citing works
Detecting Dialogue Hallucination Using Graph Neural Networks
Furumai, K., Wang, Y., Shinohara, M., Ikeda, K., Yu, Y., & Kato, T. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00128 · Cited by 5 · see citing works
Automated Fact-Checking in Dialogue: Are Specialized Models Needed?
Chamoun, E., Saeidi, M., & Vlachos, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.993 · Cited by 4 · see citing works
WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning
Wu, W., Li, W., Xiao, X., Liu, J., Li, S., & Lyu, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.18 · Cited by 7 · see citing works
q2d: Turning Questions into Dialogs to Teach Models How to Search
Bitton, Y., Cohen-Ganor, S., Hakimi, I., Lewenberg, Y., Aharoni, R., & Weinreb, E. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.843 · Cited by 5 · see citing works
With a Little Push, NLI Models can Robustly and Efficiently Predict Faithfulness
Steen, J., Opitz, J., Frank, A., & Markert, K. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.79 · Cited by 3 · see citing works
CLEVA: Chinese Language Models EVAluation Platform
Li, Y., Zhao, J., Zheng, D., Hu, Z.-Y., Chen, Z., Su, X., Huang, Y., Huang, S., Lin, D., Lyu, M. R.-T., et al. (2023) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2023.emnlp-demo.17 · Cited by 3 · see citing works
QaDialMoE: Question-answering Dialogue based Fact Verification with Mixture of Experts
Wang, L., Zhang, P., Lu, X., Zhang, L., Yan, C., & Zhang, C. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.229 · Cited by 3 · see citing works
CREDIFY: contextualized retrieval of evidence for open-domain fact verification
Nasir, A., Wasim, M., & Nasir, S. (2025) · Knowledge and Information Systems · doi:10.1007/s10115-025-02400-x · Cited by 2 · see citing works
A framework to synthetically generate fine-grained hallucinated data
Saxena, A., Aggarwal, D., Badathala, N., & Bhattacharyya, P. (2025) · Language Resources and Evaluation · doi:10.1007/s10579-025-09864-x · Cited by 4 · see citing works
Interactively Learning Social Media Representations Improves News Source Factuality Detection
Mehta, N., & Goldwasser, D. (2023) · doi:10.18653/v1/2023.findings-ijcnlp.27 · Cited by 1 · see citing works
Chen, Y.-C., & Huang, H. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i22.34534 · Cited by 1 · see citing works
Wang, Z., Li, L., Chen, L., Xu, X., Li, M., & Su, Y. (2025) · International Conference on Wireless Communication, Networking and Internet of Things (WCNIoT) · doi:10.1109/wcniot67424.2025.11381315 · Cited by 1 · see citing works
DialGuide: Aligning Dialogue Model Behavior with Developer Guidelines
Gupta, P., Liu, Y., Jin, D., Hedayatnia, B., Gella, S., Liu, S., Lange, P. L., Hirschberg, J., & Hakkani-Tür, D. Z. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.935 · Cited by 1 · see citing works
Self-citationNaser, M. Z. (2026) · Language Resources and Evaluation · doi:10.1007/s10579-026-09938-4 · Cited by 2 · see citing works
CESAR: Automatic Induction of Compositional Instructions for Multi-turn Dialogs
Aksu, T., Hazarika, D., Mehri, S., Kim, S., Hakkani-Tür, D. Z., Liu, Y., & Namazifar, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.717
Xia, W., Zhong, Y., Gong, L., Yang, Y., Zhao, S., & Cui, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04549-2_25
MIRAGE: A Metrics lIbrary for Rating hAllucinations in Generated tExt
Vendeville, B., Магдановна, Е. Л., Loor, P. D., & Kamps, J. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761644
Yang, K., Liu, D., & Wang, Z. (2026) · Synthesis lectures on human language technologies · doi:10.1007/978-3-032-06264-2_10
Liu, Y., Qian, Z., Li, P., & Zhu, Q. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729293
MDFDU: A Multi-Task Deceptive-Factual Dialogue Understanding Framework
Liu, Y., Qian, Z., Li, P., & Zhu, Q. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11728516
- Authors
- Gupta, P., Wu, C.-S., Liu, W., & Xiong, C.
- Affiliations
- Salesforce (United States) (US)
- Venue
- Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2022
- Pages
- 3785-3801
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W3207604732
- Cited by
- 48
- References
- 58
- Citations by year
- 2026: 5 · 2025: 7 · 2024: 2 · 2023: 21 · 2022: 13
- FWCI
- 33.42
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- evidence retrieval, claim verification, misinformation, disinformation, colloquial vocabulary, coreference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
