The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?
Open accessAbstract
Nouha Dziri, Sivan Milton, Mo Yu, Osmar Zaiane, Siva Reddy. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“LLMs are also capable of producing falsehoods [94] or bias [95].”
From Deanthropomorphising NLP: Can a language model be conscious? (2024) · cites this paper as [94]
The cited source says what the claim attributes to it.
In the citing paper
“Prior work has shown that retrieval-augmented models generate text that either includes additional information outside the retrieved documents (Dziri et al., 2022), ignores the documents altogether (Krishna et al., 2021), or even contradicts the documents (Longpre et al., 2021).”
From RARR: Researching and revising what language models say, using language models (2023) · cites this paper as Dziri et al., 2022
In the citing paper
“Hallucination issues in healthcare chatbots arise when responses appear factually accurate but lack a validity5,31–33.”
From Foundation metrics for evaluating effectiveness of healthcare conversations powered by generative AI (2024) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
(2014) · Cited by 33,419
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Sequence Level Training with Recurrent Neural Networks
Ranzato, M., Chopra, S., Auli, M., & Zaremba, W. (2015) · arXiv · doi:10.48550/arxiv.1511.06732 · Cited by 1,426
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781
Wizard of Wikipedia: Knowledge-Powered Conversational agentsPreprint
Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., & Weston, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01241 · Cited by 755
CTRL: A Conditional Transformer Language Model for Controllable Generation
Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., & Socher, R. (2019) · arXiv · doi:10.48550/arxiv.1909.05858 · Cited by 751
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.416 · Cited by 643
Self-citationRetrieval Augmentation Reduces Hallucination in Conversation
Shuster, K., Poff, S., Chen, M., Kiela, D., & Weston, J. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.320 · Cited by 575
TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational AgentsPreprint
Wolf, T., Sanh, V., Chaumond, J., & Delangue, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08149 · Cited by 495
The importance of assessing clinical phenomena in Mechanical Turk research.
Hall, K. A. A., Çek, D., Shaw, A. M., & Timpano, K. R. (2015) · Psychological Assessment · doi:10.1037/pas0000217 · Cited by 311
OpenDialKG: Explainable Conversational Reasoning with Attention-based Walks over Knowledge Graphs
Moon, S., Shah, P., Kumar, A., & Subba, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1081 · Cited by 306
Durmus, E., He, H., & Diab, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.454 · Cited by 306
Describing talk : a taxonomy of verbal response modes
Stiles, W. B. (1992) · Sage Pub. eBooks · Cited by 250
A Dataset for Document Grounded Conversations
Zhou, K., Prabhumoye, S., & Black, A. W. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1076 · Cited by 220
Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversations
Gopalakrishnan, K., Hedayatnia, B., Chen, Q., Gottardi, A., Kwatra, S., Venkatesh, A., Gabriel, R., & Hakkani-Tür, D. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-3079 · Cited by 207
The Curious Case of Hallucinations in Neural Machine Translation
Raunak, V., Menezes, A., & Junczys-Dowmunt, M. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.92 · Cited by 142
On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation
Wang, C., & Sennrich, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.326 · Cited by 118
Improved Natural Language Generation via Loss Truncation
Kang, D. D., & Hashimoto, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.66 · Cited by 85
Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding
Dziri, N., Madotto, A., Zaïane, O., & Bose, A. J. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.168 · Cited by 81
Self-citationIncreasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features
Rashkin, H., Reitter, D., Tomar, G. S., & Das, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.58 · Cited by 74
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark
Dziri, N., Rashkin, H., Linzen, T., & Reitter, D. (2021) · arXiv · doi:10.48550/arxiv.2105.00071 · Cited by 47
Self-citationFocused Attention Improves Document-Grounded Generation
Prabhumoye, S., Hashimoto, K., Zhou, Y., Black, A. W., & Salakhutdinov, R. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.338 · Cited by 41
Measuring Attribution in Natural Language Generation Models
Rashkin, H., Nikolaev, V., Lamm, M., Aroyo, L., Collins, M., Das, D., Petrov, S., Tomar, G. S., Turc, I., & Reitter, D. (2021) · arXiv · doi:10.48550/arxiv.2112.12870 · Cited by 34
(2021) · arXiv (Cornell University) · Cited by 13
Self-citationJi, Y., & Gao, S. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 10
Research works printed but not linked: 6
- Emily M Bender, Timnit Gebru, Angelina McMillan- Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Confer- ence on Fairness, Accountability, and Transparency, pages 610-623.
- Harry Bunt, Volha Petukhova, Emer Gilmartin, Cather- ine Pelachaud, Alex Fang, Simon Keizer, and Lau- rent Prévot. 2020. The ISO standard for dialogue act annotation, second edition. In Proceedings of the 12th Language Resources and Evaluation Con- ference, pages 549-558, Marseille, France. European Language Resources Association.
- Paul Grice. 1989. Studies in the Way of Words. Harvard University Press.
- Sabrina J Mielke, Arthur Szlam, Y-Lan Boureau, and Emily Dinan. 2020. Linguistic calibration through metacognition: aligning dialogue agent re- sponses with expected correctness. arXiv preprint arXiv:2012.14983. · arXiv
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
- Svitlana Vakulenko, Evangelos Kanoulas, and Maarten de Rijke. 2021. A large-scale analysis of mixed ini- tiative in information-seeking dialogues for conversa- tional search. arXiv preprint arXiv:2104.07096. 5277 · arXiv
Cited by
Sorted by citations
Explainability for Large Language Models: A Survey
Zhao, H., Chen, H., Yang, F., Liu, N., Deng, H., Cai, H., Wang, S., Yin, D., & Du, M. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3639372 · Cited by 681
Generative artificial intelligence
Banh, L., & Strobel, G. (2023) · Electronic Markets · doi:10.1007/s12525-023-00680-1 · Cited by 879
Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015)
Ojha, A. K., Doğruöz, A. S., Da San Martino, G., Madabushi, H. T., Kumar, R., & Sartori, E. (2015) · doi:10.18653/v1/s15-2 · Cited by 232
Foundation metrics for evaluating effectiveness of healthcare conversations powered by generative AI
Abbasian, M., Khatibi, E., Azimi, I., Oniani, D., Shakeri Hossein Abad, Z., Thieme, A., Sriram, R., Yang, Z., Wang, Y., Lin, B., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01074-z · Cited by 213
The clinicians’ guide to large language models: a general perspective with a focus on hallucinations
Roustan, D., & Bastardot, F. (2025) · Interactive Journal of Medical Research · doi:10.2196/59823 · Cited by 191
Metadata
- Authors
- Dziri, N., Milton, S., Yu, M., Zaiane, O., & Reddy, S.
- Affiliations
- University of Alberta (CA)
- McGill University (CA)
- Corresponding author
- Dziri, Nouha
- ORCID
- Venue
- Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Published
- 2022
- Pages
- 5271-5285
- OpenAlex
- W4287854593
- Cited by
- 121
- References
- 30
- Citations by year
- 2026: 16 · 2025: 24 · 2024: 33 · 2023: 42 · 2022: 5
- FWCI
- 14.38
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- Funders
- Canadian Institute for Advanced Research
- Alberta Machine Intelligence Institute
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- hallucinations, dialogue model, large language models, data bias, conversational AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
