CoChat

A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions

Open access
Huang, Lei, Yu, Weijiang, Ma, Weitao, Zhong, Weihong, Feng, Zhangyin, Wang, Haotian, Chen, Qianglong, Peng, Weihua, Feng, Xiaocheng, Qin, Bing, Liu, Ting · 2025 · ACM Transactions on Information Systems

The DOI matches its Crossref record; 496 of the 496 references it cites were found in a registry or an index; 110 of the 234 sources its claims cite were checked for support.

Abstract

The emergence of large language models (LLMs) has marked a significant breakthrough in natural language processing (NLP), fueling a paradigm shift in information acquisition. Nevertheless, LLMs are prone to hallucination, generating plausible yet nonfactual content. This phenomenon raises significant concerns over the reliability of LLMs in real-world information retrieval (IR) systems and has attracted intensive research to detect and mitigate such hallucinations. Given the open-ended general-purpose attributes inherent to LLMs, LLM hallucinations present distinct challenges that diverge from prior task-specific models. This divergence highlights the urgency for a nuanced understanding and comprehensive overview of recent advances in LLM hallucinations. In this survey, we begin with an innovative taxonomy of hallucination in the era of LLM and then delve into the factors contributing to hallucinations. Subsequently, we present a thorough overview of hallucination detection methods and benchmarks. Our discussion then transfers to representative methodologies for mitigating LLM hallucinations. Additionally, we delve into the current limitations faced by retrieval-augmented LLMs in combating hallucinations, offering insights for developing more robust IR systems. Finally, we highlight the promising research directions on LLM hallucinations, including hallucination in large vision-language models and understanding of knowledge boundaries in LLM hallucinations.

Overview

CoChat Summary

TL;DR

This survey proposes a taxonomy of LLM hallucination into factuality and faithfulness, then reviews causes, detection methods, benchmarks, mitigation strategies, and RAG-specific challenges. It also outlines open directions including hallucinations in large vision-language models and understanding knowledge boundaries in LLMs.

Key findings

  • Redefines LLM hallucination as factuality hallucination (factual contradiction: entity-error, relation-error; factual fabrication: unverifiability, overclaim) and faithfulness hallucination (instruction inconsistency, context inconsistency, logical inconsistency).
  • Identifies causes across data (misinformation/biases, knowledge boundary, inferior alignment data), training (pre-training, SFT, RLHF), and inference (imperfect decoding, over-confidence, softmax bottleneck, reasoning failure).
  • Reviews detection and benchmarks: factuality detection via fact-checking and uncertainty estimation; faithfulness detection via fact-based, classifier-based, QA-based, uncertainty-based, and LLM-based metrics. Reported benchmark sizes include TruthfulQA (817 questions), HaluEval (30,000 task-specific and 5,000 general samples), FreshQA (600 hand-crafted questions), and HaluEval 2.0 (8,770 questions across five domains).
  • Mitigation is organized by cause: data filtering, model editing, RAG; pre-training/alignment fixes; and factuality/faithfulness enhanced decoding. RAG still faces retrieval failure and generation bottleneck.

Methods

  • Design: a survey/overview that proposes a taxonomy and organizes literature on LLM hallucination principles, causes, detection, benchmarks, mitigation, RAG challenges, and future directions.
  • Data or sample: no primary dataset or new experiments; synthesizes existing research and benchmarks such as TruthfulQA, HaluEval, FreshQA, and HaluEval 2.0.
  • Main analysis: conceptual categorization and qualitative synthesis; no quantitative meta-analysis is reported.

Limitations

  • The paper does not explicitly state limitations of the survey itself.
  • It states limitations of reviewed methods: RAG can be impacted by irrelevant retrievals and shallow retriever-generator interactions; fact-checking solely via LLM parametric knowledge may yield inaccurate assessments; classifier-based faithfulness metrics are constrained by scarcity of annotated data.
  • Text makes evident: as a survey, it presents no original empirical validation or new benchmark results.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3703155

Cites

496 of 496 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

76 Valid · 5 Not valid · 48 Not checked · 108 Own results and interpretations

102 more claims · 41 with their cited sources checked

Create a free account to see all 237 claims and to verify the 13 references still open.

See all claims — free account
  • Probably not a substantial claim. 97
  • It may state two separate claims as one. 55
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

21 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 496 references

Research works printed but not linked: 33

  1. ArXiv. 2023. arxiv dataset. Retrieved from https://www.kaggle.com/datasets/Cornell-University/arxiv/versions/134
  2. Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems. Corinna Cortes, Neil D. Lawrence, Daniel D. Lee, Masashi Sugiyama, and Roman Garnett (Eds.), 1171–1179. Retrieved from https://proceedings.neurips.cc/paper/2015/hash/e995f98d56967d946471af29d7bf99f1-Abstract.html
  3. Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from trillions of tokens. In Proceedings of theInternational Conference on Machine Learning (ICML ’22). Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (Eds.), Proceedings of Machine Learning Research, Vol. 162, PMLR, 2206–2240. Retrieved from https://proceedings.mlr.press/v162/borgeaud22a.html
  4. 10.5555/3495724.3495883 · doi:10.5555/3495724.3495883
  5. 10.5555/3600270.3602050 · doi:10.5555/3600270.3602050
  6. Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. In Proceedings of the 31st International Conference on Neural Information Processing Systems. Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.), 4299–4307. Retrieved from https://proceedings.neurips.cc/paper/2017/hash/d5e2c0adad503c91f91df240d0cd4e49-Abstract.html
  7. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach to controlled text generation. In Proceedings of the 8th International Conference on Learning Representations (ICLR ’20). OpenReview.net. Retrieved from https://openreview.net/forum?id=H1edEyBKDS
  8. Bikash Gyawali, Lucas Anastasiou, and Petr Knoth. 2020. Deduplication of scholarly documents using locality sensitive hashing and word embeddings. In Proceedings of the 12th Language Resources and Evaluation Conference. European Language Resources Association, 901–910. Retrieved from https://aclanthology.org/2020.lrec-1.113
  9. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In Proceedings of the 8th International Conference on Learning Representations (ICLR ’20). OpenReview.net. Retrieved from https://openreview.net/forum?id=rygGQyrFvH
  10. Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, and Zhang Xiong. 2023d. Transformer-Patcher: One mistake worth one neuron. In Proceedings of the 11th International Conference on Learning Representations (ICLR ’23). OpenReview.net. Retrieved from https://openreview.net/pdf?id=4oYUGeGBPm
  11. Gautier Izacard Mathilde Caron Lucas Hosseini Sebastian Riedel Piotr Bojanowski Armand Joulin and Edouard Grave. 2022. Unsupervised dense information retrieval with contrastive learning. Transactions on Machine Learning Research 2022 (2022). Retrieved from https://openreview.net/forum?id=jKN1pXi7b0
  12. Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. Large language models struggle to learn long-tail knowledge. In Proceedings of the International Conference on Machine Learning (ICML ’23). Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (Eds.), Proceedings of Machine Learning Research, Vol. 202, PMLR, 15696–15707. Retrieved from https://proceedings.mlr.press/v202/kandpal23a.html
  13. Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020. Generalization through memorization: Nearest neighbor language models. In Proceedings of the 8th International Conference on Learning Representations (ICLR ’20). OpenReview.net. Retrieved from https://openreview.net/forum?id=HklBjCEKvH
  14. 10.5555/3600270.3601883 · doi:10.5555/3600270.3601883
  15. Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. In Proceedings of the 31st International Conference on Neural Information Processing Systems. Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.), 6402–6413. Retrieved from https://proceedings.neurips.cc/paper/2017/hash/9ef2ed4b7fd2c810847ffa5fa85bce38-Abstract.html
  16. 10.5555/3600270.3602776 · doi:10.5555/3600270.3602776
  17. 10.5555/3495724.3496517 · doi:10.5555/3495724.3496517
  18. Jinming Li, Wentao Zhang, Tian Wang, Guanglei Xiong, Alan Lu, and Gerard Medioni. 2023. GPT4Rec: A generative framework for personalized recommendation and user interests interpretation. In Proceedings of the SIGIR Workshop on eCommerce Co-located with the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 23). Surya Kallumadi, Yubin Kim, Tracy Holloway King, Shervin Malmasi, Maarten de Rijke, and Jacopo Tagliabue (Eds.), CEUR Workshop Proceedings, Vol. 3589, CEUR-WS.org. Retrieved from https://ceur-ws.org/Vol-3589/paper_2.pdf
  19. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out. Association for Computational Linguistics, 74–81. Retrieved from https://aclanthology.org/W04-1013
  20. 10.5555/3600270.3601532 · doi:10.5555/3600270.3601532
  21. Kevin Meng, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. 2023. Mass-editing memory in a transformer. In Proceedings of the 11th International Conference on Learning Representations (ICLR ’23). OpenReview.net. Retrieved from https://openreview.net/pdf?id=MkbcAHIYgyS
  22. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. 2022. Fast model editing at scale. In Proceedings of the 10th International Conference on Learning Representations (ICLR ’22). OpenReview.net. Retrieved from https://openreview.net/forum?id=0DcZxeWfOPt
  23. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. In NeurIPS. Retrieved from http://papers.nips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html
  24. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
  25. Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitriy Pyrkin, Sergei Popov, and Artem Babenko. 2020. Editable neural networks. In Proceedings of the8th International Conference on Learning Representations (ICLR ’20). OpenReview.net. Retrieved from https://openreview.net/forum?id=HJedXaEtvS
  26. 10.5555/3495724.3495977 · doi:10.5555/3495724.3495977
  27. Chenmien Tan, Ge Zhang, and Jie Fu. 2024. Massive editing for large language model via meta learning. In Proceedings of the 12th International Conference on Learning Representations. Retrieved from https://openreview.net/forum?id=L6L1CJQ2PE
  28. Zirui Wang Jiahui Yu Adams Wei Yu Zihang Dai Yulia Tsvetkov and Yuan Cao. 2022. SimVLM: Simple visual language model pretraining with weak supervision. In Proceedings of the 10th International Conference on Learning Representations (ICLR ’22). OpenReview.net. Retrieved from https://openreview.net/forum?id=GUrhfTuf_3
  29. 10.5555/3600270.3602070 · doi:10.5555/3600270.3602070
  30. Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, and William W. Cohen. 2018. Breaking the softmax bottleneck: A high-rank RNN language model. In Proceedings of the6th International Conference on Learning Representations (ICLR ’18). OpenReview.net. Retrieved from https://openreview.net/forum?id=HkwZSG-CZ
  31. Zhilin Yang, Thang Luong, Ruslan Salakhutdinov, and Quoc V. Le. 2019. Mixtape: Breaking the softmax bottleneck efficiently. In Proceedings of the 33rd International Conference on Neural Information Processing Systems (NeurIPS ’19). Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (Eds.), 15922–15930. Retrieved from https://proceedings.neurips.cc/paper/2019/hash/512fc3c5227f637e41437c999a2d3169-Abstract.html
  32. Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. BARTScore: Evaluating generated text as text generation. In Proceedings of the 35th International Conference on Neural Information Processing Systems (NeurIPS ’21). Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (Eds.), 27263–27277. Retrieved from https://proceedings.neurips.cc/paper/2021/hash/e4d2b6e6fdeca3e60e0f1a62fee3d9dd-Abstract.html
  33. Hugh Zhang, Daniel Duckworth, Daphne Ippolito, and Arvind Neelakantan. 2021. Trading off diversity and quality in natural language generation. In Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval). Association for Computational Linguistics, 25–33. Retrieved from https://aclanthology.org/2021.humeval-1.3

Cited by

Items 301–400 of 2,243

Metadata

Authors
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., & Liu, T.
Affiliations
  • Harbin Institute of Technology (CN)
  • Huawei Technologies (China) (CN)
Venue
ACM Transactions on Information Systems
Published
2025-01-24
Volume
43
Issue
2
Pages
1-55
ISSN
  • 1046-8188 (ISSN-L)
  • 1558-2868
OpenAlex
W4404534210
Cited by
4,688
References
229
Citations by year
2026: 2433 · 2025: 1643
FWCI
504.87
Type
journal article
Language
en
Open access
Locations
  • ACM Transactions on Information Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
LLM hallucination, large language models, hallucination taxonomy, hallucination detection, hallucination mitigation, retrieval-augmented generation, information retrieval systems, factual reliability, large vision-language models, knowledge boundaries

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-27, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-27 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.