Survey of Hallucination in Natural Language Generation
Open accessThe DOI matches its Crossref record; 355 of the 360 references it cites were found in a registry or an index, and 4 are not checked yet; 224 of the 256 sources its claims cite were checked for support.
Abstract
Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of sequence-to-sequence deep learning technologies such as Transformer-based language models. This advancement has led to more fluent and coherent NLG, leading to improved development in downstream tasks such as abstractive summarization, dialogue generation, and data-to-text generation. However, it is also apparent that deep learning based generation is prone to hallucinate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios. To address this issue, many studies have been presented in measuring and mitigating hallucinated texts, but these have never been reviewed in a comprehensive manner before. In this survey, we thus provide a broad overview of the research progress and challenges in the hallucination problem in NLG. The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions, and (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, and machine translation. This survey serves to facilitate collaborative efforts among researchers in tackling the challenge of hallucinated texts in NLG.
Verified 2026-11-19 · 4 references not checked yet
Overview
CoChat Summary
TL;DR
This survey provides a broad overview of hallucination in natural language generation, covering definitions, contributors, metrics, mitigation methods, and task-specific progress across summarization, dialogue, GQA, data-to-text, machine translation, vision-language generation, and LLMs. It distinguishes intrinsic from extrinsic hallucination and argues that hallucination must be treated differently across tasks and sources.
Key findings
- The survey categorizes hallucination into intrinsic hallucination, where generated output contradicts the source, and extrinsic hallucination, where output cannot be verified from the source; extrinsic hallucination is not always erroneous because it may come from factually correct external information.
- It reports prevalence examples from cited work: 25% of summaries generated by state-of-the-art abstractive summarization systems have hallucinated content; 62% of first sentences in WIKIBIO have additional information not stated in the corresponding infobox; and 74% of samples in the QMSum dataset consist of inconsistent facts.
- It reviews hallucination metrics as statistical, model-based (IE-based, QA-based, NLI-based, faithfulness classification, LM-based), and human evaluation, and mitigation methods as data-related and modeling/inference methods including architecture changes, planning/sketching, RL, multi-task learning, controllable generation, and post-processing.
- For LLMs, it notes that RLHF reduced the hallucination rate from 41% to 21% in closed-domain tasks for InstructGPT, and discusses reference-dependent and reference-free hallucination metrics plus mitigation via data refinement, RAG, safety fine-tuning, model editing, decoding methods, chain-of-thought variants, post-processing, and ensembling.
Methods
- Literature survey design: the paper reviews existing research on hallucination in NLG rather than conducting new experiments or primary data collection.
- Coverage: general NLG hallucination topics (definitions, contributors, metrics, mitigation, future directions) plus task-specific sections on abstractive summarization, dialogue generation, generative question answering, data-to-text generation, neural machine translation, vision-language generation, and large language models.
- Analysis: synthesizes and compares definitions, metrics, mitigation approaches, and future directions across tasks, with examples in tables and figures; the LLM section was updated in Jan 2024.
Limitations
- The paper states that GQA has no standard definition or categorization of hallucination and no automatic metric specifically for hallucination in GQA; it also notes that open-domain dialogue lacks a standard hallucination metric and that VL hallucination research is still in its early stage.
- The paper states that extrinsic hallucination has been somewhat overlooked and is more challenging to reduce, and that most existing mitigation methods do not distinguish between intrinsic and extrinsic hallucination.
- Evident from the text: as a survey, it does not provide new empirical validation of metrics or mitigation methods, and its LLM coverage is explicitly updated only through Jan 2024, so later work is not included.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
124 Valid · 26 Not valid · 19 Not checked · 135 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Furthermore, perturbation, where the insertions of most common tokens are placed at the beginning of the sentence, seems to be the most successful in hallucination mitigation.”
What the sources say
- Not checked — no full text of the sourcecites [133] [133] Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fannjiang, and David Sussillo. 2019. Hallucinations in Neural Machine Translation. ICLR (2019). -
- Disputedcites [215] [215] Vikas Raunak, Arul Menezes, and Marcin Junczys-Dowmunt. 2021. The Curious Case of Hallucinations in Neural Machine Translation. (4 2021), 1172-1183. https://arxiv.org/abs/2104.06683v1 - sourceExperimental — accuracy not yet measured
In the passage describing the perturbation procedure (page unavailable), the paper says that inserting common tokens at the beginning is used to generate hallucinations, not mitigate them. Its prevention section instead discusses approaches such as robust learning and corpus-level filtering. (LLM, conf=0.92)
Checked by a reasoning model — may be wrong.
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“While there are works on the visual extension of faithfulness-oriented RLHF [303], these essentially involve labeling more data, which is expensive and difficult to scale.”
What the sources say
- Disputedcites [303] [303] Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, and Tat-Seng Chua. 2023. RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback. CoRR abs/2312.00849 (2023). https://doi.org/10.48550/ARXIV.2312.00849 arXiv:2312.00849 - sourceExperimental — accuracy not yet measured
RLHF-V does use human annotation for visual-model alignment, but it makes the contrary point about needing more labeled data. The Fine-grained Correctional Human Preference Collection section says its corrections improve data efficiency, and Section 4.3 reports comparable hallucination rates using only 200 preference examples versus an order of magnitude more ranking-labeled examples. Our copy does not establish a separate claim about annotation cost or scalability. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“Nevertheless, as illustrated by Falke et al. [66], off-the-shelf NLI models tend to transfer poorly to the abstractive summarization task.”
What the sources say
- Supportedcites [66] [66] Tobias Falke, Leonardo FR Ribeiro, Prasetya Ajie Utama, Ido Dagan, and Iryna Gurevych. 2019. Ranking Generated Summaries by Correctness: An Interesting but Challenging Application for Natural Language Inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . 2214-2220. - source
The Introduction states that models trained on NLI datasets transfer poorly to the paper’s downstream task of ranking generated summaries by correctness. The Conclusions likewise say that out-of-the-box NLI models do not perform well on that task. (LLM, conf=0.94)
- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Such wrong association results in generation with facts mixed up between two similar entities [60, 233].”
What the sources say
- Partly supportedcites [60] [60] Nouha Dziri, Andrea Madotto, Osmar Zaiane, and Avishek Joey Bose. 2021. Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding. EMNLP (2021). - sourceExperimental — accuracy not yet measured
The GPT2-KG passage describes an incorrect association between entities, and G Error Analysis gives an example of a generated response assigning a book to the wrong author. These support the claim that facts can be mixed up, but the paper does not say the entities involved are similar. The supplied passages do not provide page numbers. (LLM, conf=0.83)
- Supportedcites [233] [233] Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval Augmentation Reduces Hallucination in Conversation. EMNLP (2021). - source
The Introduction states that language models often mix up facts between two similar entities when generating factually incorrect statements. This directly supports the cited description of the resulting error. (LLM, conf=0.92)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“As a result, the target reference may contain information that cannot be supported by the source [198, 268].”
What the sources say
- Supportedcites [198] [198] Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. ToTTo: A Controlled Table-To-Text Generation Dataset. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . 1173-1186. - source
The Dataset section says that reference targets selected from web sentences may contain information that cannot be inferred from the source. The Data Annotation Process section also describes deleting phrases unsupported by table cells. (LLM, conf=0.93)
- Supportedcites [268] [268] Hongmin Wang. 2019. Revisiting Challenges in Data-to-Text Generation with Fact Grounding. In Proceedings of the 12th International Conference on Natural Language Generation . 311-322. - source
The Introduction reports that about 40% of human-written game-summary content cannot be directly mapped to the input boxscore records and draws on information beyond those inputs. Section 2.1 likewise describes reference content not licensed by the boxscore table. This supports the claim that a target reference may contain information the source does not support. (LLM, conf=0.93)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
117 more claims · 74 with their cited sources checked
Create a free account to see all 304 claims and to verify the 2 references still open.
- Borderline: one of the tests for a substantial claim is unclear. 1
- It may state two separate claims as one. 90
- Probably not a substantial claim. 142
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Hallucination can lead to serious consequences such as incorrect or misleading code, security risks, and ethical issues [186].”
From A survey on large language model based autonomous agents (2024) · cites this paper as [186]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“citation fabrication constitutes a form of extrinsic hallucination in the absence of retrieval augmentation, since the model generates references not grounded in any provided source material [4].”
From How LLMs Cite and Why It Matters: A Cross-Model Audit of Reference Fabrication in AI-Assisted Academic Writing and Methods to Detect Phantom Citations (2026) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent taxonomic work distinguishes at least seven cases [15,18].”
From Hallucination as epistemic pollution: epistemic injustice and credibility in futures-oriented science education (2026) · cites this paper as [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs struggle to generate accurate and up-to-date responses on current topics, due to outdated knowledge, lack of domain-specific expertise, or hallucination13–17.”
From Leveraging long context in retrieval augmented language models for medical question answering (2025) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Fine-tuning can enrich LLMs with domain knowledge, enhance their specific ability, improve the fairness and reliability of the outputs, and prevent certain damage caused by hallucination [26].”
From FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning (2024) · cites this paper as [26]
Experimental — accuracy not yet measured
41 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Extrinsic Object Hallucination
“CHAIR (Caption Hallucination Assessment with Image Relevance) metric, which calculates what proportion of object words generated are actually in the image according to the ground truth captions.”
§ Extrinsic Object Hallucination
“CHAIR 𝑖 measures per-instance object hallucination, i.e. what fraction of object instances in each generated caption are hallucinated.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 7.1 Hallucination Definition in Abstractive Summarization
“For instance, in Table 1, given the input article shown in the caption, an example of intrinsic hallucination is ' The Ebola vaccine was rejected by the FDA in 2019, '”
§ 2.1 Categorization
“in the abstractive summarization task from Table 1, the generated summary ' The first Ebola vaccine was approved in 2021 ' contradicts the source content”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Survey of Hallucination in Natural Language Generation
“The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine translation, and visual-language generation; and (3) hallucinations in large language models (LLMs)”
§ 1 INTRODUCTION
“The third part discusses this phenomenon in LLMs in Section 13.”
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“Recently, the rapid development of NLG has captured the imagination of many thanks to the advances in deep learning technologies, especially Transformer [189]-based models like BERT [29], BART [100], GPT-2 [149], and GPT-3 [16].”
BackgroundAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“This work investigates the performances of GPT-3 [13], GPT-Neo/J [141], GPT-2 [105] and a T5-based model [106].”
BackgroundAdvances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 8,890
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · cited 1×
“Recently, the development of NLG has conquered the imagination of many thanks to the advances in deep learning technologies, especially the Transformer [159]-based models like BERT [24], BART [83], GPT-2 [127], and GPT3 [13].”
BackgroundLewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816 · cited 1×
“Recently, the development of NLG has conquered the imagination of many thanks to the advances in deep learning technologies, especially the Transformer [159]-based models like BERT [24], BART [83], GPT-2 [127], and GPT3 [13].”
Background
Research works printed but not linked: 16
- [4] Joshua Albrecht and Rebecca Hwa. 2007. A Re-examination of Machine Learning Approaches for Sentence-Level MT Evaluation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics . Association for Computational Linguistics, Prague, Czech Republic, 880-887. https://aclanthology.org/P07-1111 -
- [16] Andrzej Białecki, Robert Muir, Grant Ingersoll, and Lucid Imagination. 2012. Apache lucene 4. In SIGIR 2012 workshop on open source information retrieval . 17. -
- [53] Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. 2023. Improving Factuality and Reasoning in Language Models through Multiagent Debate. In Forty-first International Conference on Machine Learning . -
- [121] Philipp Koehn and Rebecca Knowles. 2017. Six Challenges for Neural Machine Translation. In First Workshop on Neural Machine Translation . Association for Computational Linguistics, 28-39. -
- [145] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation. In ICML . -
- [149] Tian Li, Ahmad Beirami, Maziar Sanjabi, and Virginia Smith. 2020. Tilted Empirical Risk Minimization. In International Conference on Learning Representations . -
- [164] Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. 2018. Neural Baby Talk. In CVPR . -
- [165] Yukun Ma, Khanh Linh Nguyen, Frank Z Xing, and Erik Cambria. 2020. A Survey on Empathetic Dialogue Systems. Information Fusion 64 (2020), 50-70. -
- [175] Marianna J. Martindale, Marine Carpuat, Kevin Duh, and Paul McNamee. 2019. Identifying Fluently Inadequate Output in Neural and Statistical Machine Translation. In MTSummit . -
- [179] Kathleen McKeown. 1992. Text Generation . Cambridge University Press. -
- [209] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. OpenAI Blog 1, 8 (2019), 9. -
- [238] Hui Su, Xiaoyu Shen, Sanqiang Zhao, Zhou Xiao, Pengwei Hu, Cheng Niu, and Jie Zhou. 2020. Diversifying Dialogue Generation with Non-Conversational Text. In 58th Annual Meeting of the Association for Computational Linguistics . ACL, 7087-7097. -
- [256] Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li. 2017. Neural Machine Translation with Reconstruction. In Thirty-First AAAI Conference on Artificial Intelligence . -
- [261] Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. 2023. A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation. CoRR abs/2307.03987 (2023). https://doi.org/10.48550/arXiv.2307.03987 arXiv:2307.03987 - · arXiv · doi:10.48550/arxiv.2307.03987 · Paper page
- [285] Zeqiu Wu, Michel Galley, Chris Brockett, Yizhe Zhang, Xiang Gao, Chris Quirk, Rik Koncel-Kedziorski, Jianfeng Gao, Hannaneh Hajishirzi, Mari Ostendorf, et al. 2021. A Controllable Model of Grounded Response Generation. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 35. 14085-14093. -
- [291] Xinnuo Xu, Ondřej Dušek, Verena Rieser, and Ioannis Konstas. 2021. AGGGEN: Ordering and Aggregating while Generating. roceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL2021) (2021). -
Cited by
Items 1,301–1,400 of 4,844
Factored Verification: Detecting and Reducing Hallucination in Summaries of Academic Papers
George, C., & Stuhlmüller, A. (2023) · Workshop on Information Extraction from Scientific Publications (WIESP) · doi:10.18653/v1/2023.wiesp-1.13 · Cited by 22
OntoFact: Unveiling Fantastic Fact-Skeleton of LLMs via Ontology-Driven Reinforcement Learning
Shang, Z., Ke, W., Xiu, N., Wang, P., Liu, J., Li, Y., Luo, Z., & Ji, K. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29859 · Cited by 20
Wang, H., & Shu, K. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679821 · Cited by 35
Generative artificial intelligence in finance
OECD (2023) · OECD artificial intelligence papers · doi:10.1787/ac7149cc-en · Cited by 22
Bane and boon of hallucinations in context of generative AIPreprint
Patel, H. (2024) · doi:10.36227/techrxiv.171198062.20183635/v1 · Cited by 13
Meta-Cognitive Memory Policy Optimization for Long-Horizon LLM Agents
Liu, Z., Hao, Z., Chen, Y., Wang, H., Hou, J., Ding, R., Yang, Y., Ji, W., Xia, W., & Liu, F. (2026) · arXiv · Cited by 12
“Despite this advantage, recursive summarization inherently accumulates semantic noise introduced by LLMs, which can induce cascading hallucinations and ultimately degrade long-horizon collapse (Zhang et al., 2025b; Ji et al., 2023; Sheng et al., 2026).”
SYNFAC-EDIT: Synthetic Imitation Edit Feedback for Factual Alignment in Clinical Summarization
Mishra, P., Yao, Z., Vashisht, P., Ouyang, F., Wang, B., Mody, V. D., & Yu, H. (2024) · arXiv · doi:10.48550/arxiv.2402.13919 · Cited by 7
Evolving techniques and emerging schemes: prospective teachers’ transformation of ChatGPT
Davis, J. D. (2025) · International Journal of Science and Mathematics Education · doi:10.1007/s10763-024-10533-8 · Cited by 6
“These false results have been referred to as hallucinations (Ji et al., 2023), but this suggests some type of sensory response that does not align with reality.”
RIGHT: retrieval-augmented generation for mainstream hashtag recommendation
Fan, R.-Z., Fan, Y., Chen, J., Guo, J., Zhang, R., & Cheng, X. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-56027-9_3 · Cited by 6
Hamilton, K., Longo, L., & Božić, B. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3651909 · Cited by 13
Comparative Analysis of Performance of Large Language Models in Urogynecology
Yadav, G. S., Pandit, K. M., Connell, P. T., Erfani, H., & Nager, C. W. (2024) · Urogynecology · doi:10.1097/spv.0000000000001545 · Cited by 8
From outputs to insights: a survey of rationalization approaches for explainable text classification
Guzman, E. M., Schlegel, V., & Batista-Navarro, R. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1363531 · Cited by 13
A Hierarchical Context Augmentation Method to Improve Retrieval-Augmented LLMs on Scientific Papers
Che, T.-Y., Mao, X.-L., Lan, T., & Huang, H. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671847 · Cited by 16
Causal-CoG: A Causal-Effect Look at Context Generation for Boosting Multi-Modal Language Models
Zhao, S., Li, Z., Lu, Y., Yuille, A., & Wang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01267 · Cited by 20
Toward Facilitating Search in VR With the Assistance of Vision Large Language Models
Liu, C., Cheung, C. S., Xu, M., Zhang, Z., Su, M., & Fan, M. (2024) · ACM Symposium on Virtual Reality Software and Technology (VRST) · doi:10.1145/3641825.3687742 · Cited by 9
NoTeeline: Supporting Real-Time, Personalized Notetaking with LLM-Enhanced Micronotes
Huq, F., Samee, A., Lin, D. C.-E., Tang, A. X., & Bigham, J. P. (2025) · Proceedings of the 30th International Conference on Intelligent User Interfaces · doi:10.1145/3708359.3712086 · Cited by 7
Learning to generate and evaluate fact-checking explanations with transformers
Feher, D., Khered, A., Zhang, H., Batista-Navarro, R., & Schlegel, V. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109492 · Cited by 15
Enhancing Efficiency and Flexibility of Rapid Prototyping for Scalable Multimodal Intelligent Agents
Chandrasekaran, M. (2024) · Artificial Intelligence for Business (AIxB) · doi:10.1109/aixb62249.2024.00019 · Cited by 15
“For example, research has shown that agents can exhibit hallucinatory behavior when given simplistic instructions during code generation tasks, resulting in misleading or incorrect code, ethical issues and security risks [12].”
Development of an Expert Chatbot for Digital Forensics Using RAG Model Implementation
Kim, M., Kim, D., Park, Y., & Jeong, D. (2024) · International Conference on Platform Technology and Service · doi:10.1109/platcon63925.2024.10830748 · Cited by 6
Herbert, D. N., Westendorf, J., Farmer, M., & Reeder, B. (2025) · Journal of Studies on Alcohol and Drugs · doi:10.15288/jsad.24-00319 · Cited by 7
Lighter And Better: Towards Flexible Context Adaptation For Retrieval Augmented Generation
Wu, C., Shao, N., Liu, Z., Xiao, S., Li, C., Zhang, C., Wang, S., & Lian, D. (2025) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3701551.3703580 · Cited by 18
Wan, X., Wang, R., Zhao, J., et al. (2025) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.70017 · Cited by 5
“Second, the risk of hallucination, where models generate plausible but factually incorrect conclusions, can compromise reliability, such as overestimating clarity scores by inferring implicit information not explicitly stated in guidelines [77].”
CogSys: Efficient and Scalable Neurosymbolic Cognition System via Algorithm-Hardware Co-Design
Wan, Z., Yang, H., Raj, R., Liu, C.-K., Samajdar, A., Raychowdhury, A., & Krishna, T. (2025) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca61900.2025.00064 · Cited by 9
“This undermines the model output trustworthiness in cognitive and safety-critical applications [31], [39], [84].”
Hierarchical Index Retrieval-Driven Wireless Network Intent Translation With LLM
Wang, J., Guo, L., Wu, J., Yan, C., Sun, H., Zhang, L., Zhuang, Z., Qi, Q., & Liao, J. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3564937 · Cited by 11
“The black-box nature of LLM makes their reasoning processes difficult to interpret [33], [34].”
Evaluating LLMs for visualization generation and understanding
Khan, S. R., Chandak, V., & Mukherjea, S. (2025) · Discover Data · doi:10.1007/s44248-025-00036-4 · Cited by 12
Quantifying Bias in Text-to-Image Generative Models
Vice, J., Akhtar, N., Hartley, R. I., & Mian, A. S. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3572115 · Cited by 21
“Instead, it has been proposed as a method for image outpainting [55] and sometimes, hallucination is even discussed as a tool to improve generative models [55], [56], or an artefact to mitigate [57], [58].”
Li, Y., Branco, P., Hoole, A. M., Marwah, M., Koduvely, H. M., Jourdan, G.-V., & Jou, S. (2025) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp61157.2025.00191 · Cited by 31
“This distinction is vital for verifying the reliability of LLMs, as reliance exclusively on pre-trained patterns often leads to poor generalization [25] and inconsistent interoperability [26], especially when addressing semantic and structural variations in code within real-world scenarios.”
Behind the Counter: Exploring Motivations and Barriers of Online Counterspeech Writing
Ping, K., Kumar, A., Ding, X., & Rho, E. H. R. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3745769 · Cited by 17
ChatSOS: Vector database augmented generative question answering assistant in safety engineering
Tang, H., Chen, D., Chu, Q., & Liu, Z. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128644 · Cited by 8
Long Context vs. RAG: Strategies for Processing Long Documents in LLMs
Li, X., Bai, Y., Jin, B., Zhu, F., Pan, L., & Cao, Y. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3731690 · Cited by 16
Improving LLM Outputs Against Jailbreak Attacks With Expert Model Integration
Tsmindashvili, T., Kolkhidashvili, A., Kurtskhalia, D., Maghlakelidze, N., Mekvabishvili, E., Dentoshvili, G., Shamilov, O., Gachechiladze, Z., Saporta, S., & Choladze, D. D. (2025) · IEEE Access · doi:10.1109/access.2025.3592458 · Cited by 9
“An important advancement in AI technology has been brought about by the creation of LLMs (Large Language Models) like GPT variations (f.e InstructGPT [2], LLAMA models [3], Mistral [4], and Mixtral [5]) presenting new concerns and challenges about their safe deployment [6].”
Assessing the potential of generative agents in crowdsourced fact-checking
Costabile, L., Orlando, G. M., Gatta, V. L., & Moscato, V. (2025) · Online Social Networks and Media · doi:10.1016/j.osnem.2025.100326 · Cited by 14
“Incorporating the complete content of each news article within a single prompt presents practical challenges due to the limited context window of any LLM-powered agent and the increased risk of generating hallucinations [49].”
Lin, K., Rasool, A., Surabhi, S., Mutlu, O. C., Zhang, H., Wall, D. P., & Washington, P. Y. (2025) · JMIR AI · doi:10.2196/75030 · Cited by 9
Bozkurt, A. (2025) · Open Praxis · doi:10.55982/openpraxis.17.3.794 · Cited by 8
Catch me if you search: When contextual web search results affect the detection of hallucinations
Nahar, M., Lee, E.-J., Park, J. W., & Lee, D. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108763 · Cited by 7
Kadian, A., Merhi, M. I., Gupta, M., Ghafoori, A., & Dennehy, D. (2025) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2025.3606352 · Cited by 9
The dual‐role clinician teacher: A human bridge for socially responsible AI use
Brouwer, H. J., Barry, M., & GROOT, E. D. (2025) · Medical Education · doi:10.1111/medu.70035 · Cited by 6
“5 And, as Holm 6 with the students about their own epistemological challenges while using GenAI, they create rich opportunities for bidirectional learning.”
Clinical pathway-aware large language models for reliable and transparent medical dialogue
Wu, J., Wu, X., Zheng, Y., & Yang, J. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104942 · Cited by 14
Chen, Y., Wen, B., & Zulkernine, F. H. (2025) · JMIR AI · doi:10.2196/75932 · Cited by 11
“One of the critical limitations is hallucination, where LLMs generate content that is incoherent or factually incorrect [14,15].”
Transcription Accuracy of Automatic Speech Recognition for Orthodontic Clinical Records
O’Kane, R., Stonehouse‐Smith, D., Ota, L. C. U., Patel, R., Johnson, N., Slipper, C., Seehra, J., Papageorgiou, S. N., & Cobourne, M. T. (2025) · Journal of Dental Research · doi:10.1177/00220345251382452 · Cited by 11
“In particular, mistranscriptions are recognition errors in output that distort spoken words, whereas hallucinations invent or contradict information not found in the original spoken text (Ji et al. 2023).”
Smutný, Z., & Sudzina, F. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2573037 · Cited by 9
“…as data misuse, data poisoning, amplification of existing bias, mis-information proliferation (Capraro et al., 2024) or artificial hallucination (Ji et al., 2023) have also come to the fore; as well as ethical issues (Steinerov � a & Ondri � sov � a, 2024) involving, for instance, copy-right…”
Xia, M., Bayerlein, R., Chemli, Y., Liu, X., Ouyang, J., Lin, M., Fakhri, G. E., Badawi, R. D., Li, Q., & Liu, C. (2025) · Journal of Nuclear Medicine · doi:10.2967/jnumed.125.270653 · Cited by 19
“Automatic hallucination detectors ( 44 ), trained on hallucination benchmark datasets ( 4 ), have recently been explored in large (vision) language models to reduce the burden of human evaluation (section IV of the supplemental material).”
Passi, S., Dhanorkar, S., & Vorvoreanu, M. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_98 · Cited by 15
Large language model performance versus human expert ratings in automated suicide risk assessment
Thomas, J., Elyoseph, Z., Kuchinke, L., & Meinlschmidt, G. (2025) · Scientific Reports · doi:10.1038/s41598-025-22402-7 · Cited by 15
How is language intelligence evolving? A multi-dimensional survey of large language models
Chen, X., Zhou, L., Chen, J., Wang, G., & Li, X. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.130637 · Cited by 11
Propaganda by prompt: Tracing hidden linguistic strategies in large language models
Barfar, A., & Sommerfeldt, L. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104403 · Cited by 10
Medical QA dialogue datasets in RAG systems performance evaluation and ChatGPT optimization
Muhetaer, M., Yusupu, A., Yifan, W., Mutalipu, M., & Fan, H. (2025) · Scientific Reports · doi:10.1038/s41598-025-28015-4 · Cited by 8
“This results in the so-called “hallucination” problem 3 .”
Temur, K. T., & Mağat, G. (2026) · Scientific Reports · doi:10.1038/s41598-025-34773-y · Cited by 8
“Unlike rule-based or image-trained diagnostic systems, LLMs generate outputs based on probabilistic language patterns rather than explicit medical validation, which may result in the presentation of inaccurate diagnostic suggestions without clear indicators of uncertainty 17 .”
An LLM-based multi-agent system for remote sensing analysis
Sun, Z., Zhou, Y., & Yang, J. (2026) · Big Earth Data · doi:10.1080/20964471.2025.2600178 · Cited by 12
“Challenges also arise from LLMs’ inherent uncertainty, known as hallucination (Huang et al., 2024; Ji et al., 2023; Zhang et al., 2023), worsened by cascading hallucinations in multi-agent systems, where errors propagate—e.g., an inappropriate NDVI threshold led to inaccurate green space…”
Chaabene, N. E. H. B., & Hammami, H. (2026) · Smart Learning Environments · doi:10.1186/s40561-025-00423-z · Cited by 8
The comfort of automation: why cognitive sovereignty matters in AI-driven life sciences
Branda, F., & Ciccozzi, M. (2026) · Artificial Intelligence in the Life Sciences · doi:10.1016/j.ailsci.2026.100158 · Cited by 14
A Survey of Multimodal Hallucination Evaluation and Detection
Chen, Z., Min, Y., Zhang, J., Yan, B., Wang, J., Wang, X., & Shan, S. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02756-9 · Cited by 46
“This phenomenon has garnered significant attention across multiple fields, including natural language generation (Ji et al., 2023), computer vision (Rohrbach et al., 2018; Yu et al., 2018), and multi-modal modeling (Bai et al., 2024).”
Pangestu, F. N. N., Hamidah, I., Widaningsih, L., Abdullah, A. G., & Saputra, N. A. (2026) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2026.100581 · Cited by 21
Gong, E. J., Bang, C. S., & Shin, Y. S. (2026) · Bioengineering · doi:10.3390/bioengineering13030365 · Cited by 19
“The phenomenon of hallucination, where LLMs generate plausible but entirely fabricated information, poses significant risks in medical contexts where accuracy directly affects patient care decisions [5,13].”
Winslow, B. D., Garibay, Ö. Ö., Goyal, T., Koon, S., Margetis, G., Salvendy, G., Shneiderman, B., Tayebi, A., & Vardoulakis, L. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2641703 · Cited by 17
ChatGPT's Astonishing Fabrications About Percy Ludgate
Randell, B., & Coghlan, B. (2023) · IEEE Annals of the History of Computing · doi:10.1109/mahc.2023.3272989 · Cited by 6
“” [4] (The term “ hallucination ” has come into use by the AI community for such responses, which are not unique to ChatGPT [5].)”
BackgroundDirected Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generation
Huang, F., Ke, P., & Huang, M. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00582 · Cited by 17
“Previous studies empirically found that AR generated texts may lose relevance to the input sentences, which is also known as hallucination (Maynez et al., 2020; Ji et al., 2022) or off-prompt errors (Dou et al.”
BackgroundUsing ChatGPT with Confidence for Biodiversity-Related Information Tasks
Elliott, M. J., & Fortes, J. A. B. (2023) · Biodiversity Information Science and Standards · doi:10.3897/biss.7.112926 · Cited by 7
Exploring Text Decoding Methods for Portuguese Legal Text Generation
Sakiyama, K. M., Montanari, R., Malaquias, R. D., Nogueira, R. F., & Romero, R. A. F. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-45368-7_5 · Cited by 6
Unlearning Descartes: Sentient AI is a Political Problem
Hull, G. (2023) · Journal of Social Computing · doi:10.23919/jsc.2023.0020 · Cited by 9
Alevizos, V., Georgousis, I., Simasiku, A., Messinis, A., Karypidou, S., & Malliarou, D. (2024) · International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) · doi:10.1109/acdsa59508.2024.10467463 · Cited by 14
“Insofar as AI systems like AutoRepo streamline and enhance efficiency in complex tasks, they also bring to the fore the issue of potential mistakes and 'hallucinations'[21][22]; instances where AI systems generate inaccurate or misleading information.”
BackgroundKunz, J. (2024) · Linköping studies in science and technology. Dissertations · doi:10.3384/9789180754712 · Cited by 8
Scalable integration of multiomic single-cell data using generative adversarial networks
Giansanti, V., Giannese, F., Botrugno, O. A., Gandolfi, G., Balestrieri, C., Antoniotti, M., Tonon, G., & Cittaro, D. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae300 · Cited by 8
“Semisupervised training of MOWGAN, when possible, reduces the possibility of hallucination, a well-known issue that affects deep generative models (Ji et al. 2022), and allows the computation of good-quality and biologically meaningful data.”
Why Do Fishes Die? Lab Assignment Created Using ChatGPT
Trčková, K., Maršálek, R., Žáček, M., & Teplá, M. (2024) · Journal of Chemical Education · doi:10.1021/acs.jchemed.4c00130 · Cited by 9
Enhancing the Rationale-Input Alignment for Self-explaining Rationalization
Liu, W., Wang, H., Wang, J., Deng, Z., Zhang, Y., Wang, C., & Li, R. (2024) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00176 · Cited by 19
“However, LLMs sometimes exhibit unpredictable failure modes [31] or hallucination reasoning [32], making this kind of generative explanation not trustworthy enough in some high-stakes scenarios.”
BackgroundNatural Language Generation for Visualizations: State of the Art, Challenges and Future Directions
Hoque, E., & Islam, M. S. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15266 · Cited by 32
Stoykova, R., Porter, K., & Beka, T. (2024) · Forensic Science International Synergy · doi:10.1016/j.fsisyn.2024.100563 · Cited by 17
Artificial Intelligence (AI) and Men’s Health Clinic Efficiency and Clinic Billing
Kinachtchouk, N., & Canes, D. (2024) · Current Urology Reports · doi:10.1007/s11934-024-01252-2 · Cited by 6
Artificial Doctors: Performance of Chatbots as a Tool for Patient Education on Keratoconus
Özcan, Z. Ö., Doğan, L., & Yılmaz, İ. E. (2024) · Eye & Contact Lens Science & Clinical Practice · doi:10.1097/icl.0000000000001160 · Cited by 10
“36 For this reason, even if the content is accurate and of high quality, the content is often fi ctional.”
InfluentialReducing extrinsic hallucination in multimodal abstractive summaries with post-processing technique
Rafi, S. M., Laitonjam, L., & Das, R. (2025) · Neural Computing and Applications · doi:10.1007/s00521-024-10895-8 · Cited by 10
“Extrinsic hallucinations [21–23] can lead to summaries that appear more similar to the reference summaries but may not faithfully represent the source text.”
EEG-Based Music Emotion Prediction Using Supervised Feature Extraction for MIDI Generation
Morales, Ó. W. G., Pérez-Nastar, H. D., Álvarez-Meza, A. M., Torres-Cardona, H. F., & Castellanos-Dominguez, G. (2025) · Sensors · doi:10.3390/s25051471 · Cited by 11
“Nonetheless, integrating DL models with EEG BCI devices imposes additional constraints, particularly in capturing fine-grained temporal and spatial neural patterns [42,43], as well as addressing individual variability, which hinders model generalization for controllable deep data generation [44,45].”
Can LLMs revolutionize text mining in chemistry? A comparative study with domain-specific tools
Kumari, M., Chauhan, R., & Garg, P. (2025) · Comput. Stand. Interfaces · doi:10.1016/j.csi.2025.103997 · Cited by 8
Negotiating with LLMs: Prompt Hacks, Skill Gaps, and Reasoning Deficits
Schneider, J., Haag, S., & Kruse, L. C. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-83845-3_15 · Cited by 34
Ateia, S., & Kruschwitz, U. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88720-8_1 · Cited by 14
Global agricultural adaptation case database and trend analysis based on large language models
Jing-Wen, Z., ZHANG, X.-Y., & Xin, M. (2025) · Advances in Climate Change Research · doi:10.1016/j.accre.2025.03.013 · Cited by 10
Context and Layers in Harmony: A Unified Strategy for Mitigating LLM Hallucinations
Yu, S., Kim, G., & Kang, S. (2025) · Mathematics · doi:10.3390/math13111831 · Cited by 11
Experiences with Content Development and Assessment Design in the Era of GenAI
Sharma, A., Shailendra, S., & Kadel, R. (2025) · International Conference on Computer Science, Engineering, and Education (CSEE) · doi:10.1109/csee64583.2025.00008 · Cited by 9
“This is often termed as hallucination [11], [12].”
Lu, X., Mahesh, A., Shen, Z., Dudley, M., Sano, L., & Wang, X. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98417-4_28 · Cited by 11
‘Beach’ to ‘Bitch’: Inadvertent Unsafe Transcription of Kids’ Content on YouTube
Ramesh, K., KhudaBukhsh, A. R., & Kumar, S. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i11.21470 · Cited by 9
CDConv: A Benchmark for Contradiction Detection in Chinese Conversations
Zheng, C., Zhou, J., Zheng, Y., Peng, L., Guo, Z., Wu, W., Niu, Z.-Y., Wu, H., & Huang, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.2 · Cited by 21
A comprehensive survey of personal knowledge graphs
Chakraborty, P., & Sanyal, D. K. (2023) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1513 · Cited by 16
“LLMs, however, are notorious for hallucinating, or generating false and unreliable responses to questions and prompts about knowledge that they do not come across during training (Ji et al., 2023).”
BackgroundTackling Hallucinations in Neural Chart Summarization
Islam, S. O. U., Škrjanec, I., Dušek, O., & Demberg, V. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.30 · Cited by 13
The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models
Hagström, L., Saynova, D., Norlund, T., Johansson, M., & Johansson, R. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.332 · Cited by 6
Dancing Between Success and Failure: Edit-level Simplification Evaluation using SALSA
Heineman, D., Dou, Y., Maddela, M., & Xu, W. H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.211 · Cited by 45
Critic-Driven Decoding for Mitigating Hallucinations in Data-to-text Generation
Lango, M., & Dušek, O. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.172 · Cited by 27
Li, S., Zhou, G., Li, Z., Lu, J., & Huang, N. (2023) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs60937.2023.00032 · Cited by 9
Artificial Intelligence for Climate Smart Forestry: A Forward Looking Vision
Luo, F., Liu, L., Wang, G. G., Kumar, V., Ashton, M. S., Abernethy, J., Afghah, F., Browning, M. H. E. M., Coyle, D. R., Dames, P., et al. (2023) · IEEE International Conference on Cognitive Machine Intelligence (CogMI) · doi:10.1109/cogmi58952.2023.00011 · Cited by 10
“However, the hallucination shortcoming of LLMs [135] makes it possible for them to generate answers that may cause significant economic loss.”
BackgroundWang, Z., Chen, Y., Guo, X., Li, Y., Li, P., Li, C., Ye, X., & Sakurai, T. (2024) · Current Bioinformatics · doi:10.2174/0115748936285493240307071916 · Cited by 7
Face4Rag: Factual Consistency Evaluation for Retrieval Augmented Generation in Chinese
Xu, Y., Cai, T., Jiang, J., & Song, X. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671656 · Cited by 24
Self-citationGenerative AI for the Maritime Environments
Reddy, G. P., Sinha, S., & Park, S. (2024) · International Conference on Ubiquitous and Future Networks · doi:10.1109/icufn61752.2024.10625100 · Cited by 10
“These models occasionally exhibit a phenomenon known as hallucination, where they generate errors or inaccuracies [20].”
BackgroundKhawaldeh, A. M. (2024) · International Journal for the Semiotics of Law - Revue internationale de Sémiotique juridique · doi:10.1007/s11196-024-10199-z · Cited by 7
“Lies by ChatGPT or similar systems seem believable to lay citizens given the large number of references and sources to factual content [ 2 , 19 , 20 ].”
Complex QA and language models hybrid architectures, SurveyPreprint
Daull, X., Bellot, P., Bruno, E., Martin, V., & Murisasco, É. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.09051 · Cited by 44
Langer, M., Baum, K. L., & Schlicker, N. (2023) · doi:10.31234/osf.io/ke256 · Cited by 6
Procko, T., Davidoff, A., Elvira, T., & Ochoa, O. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4594836 · Cited by 8
Do People Trust Humans More Than ChatGPT?Preprint
Buchanan, J. A., & Hickman, W. C. (2024) · Social Science Research Network · doi:10.2139/ssrn.4635674 · Cited by 41
“In a survey article, Ji et al. (2023) state that “deep learning based generation is prone to halluci-nate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios.””
BackgroundTowards Faithful and Robust LLM Specialists for Evidence-Based Question-AnsweringPreprint
Schimanski, T., Ni, J., Kraus, M., Ash, E., & Leippold, M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4728973 · Cited by 12
Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise
Wang, R. E., Ribeiro, A. T., Robinson, C. D., Loeb, S., & Demszky, D. (2024) · arXiv · doi:10.48550/arxiv.2410.03017 · Cited by 6
“LMs are trained on the Web (e.g., Wikipedia) that is vastly different from real-world interactions (e.g., K-12 interactions), and current methods like fine-tuning and prompt engineering struggle to close this gap (Frieder et al., 2023, Ji et al., 2023, Singer, 2023, Wang and Demszky, 2023).”
Uncertainty in Natural Language Generation: From Theory to Applications
Baan, J., Daheim, N., Ilia, E., Ulmer, D., Li, H.-S., Fernández, R., Plank, B., Sennrich, R., Zerva, C., & Aziz, W. (2023) · arXiv · doi:10.48550/arxiv.2307.15703 · Cited by 6
“Likewise, those about NLG do not discuss uncertainty (Gatt and Krahmer, 2018; Celikyilmaz et al., 2020; Ji et al., 2022; Erdem et al., 2022; Dong et al., 2022).”
BackgroundUnlocking electronic health records: a hybrid graph RAG approach to safe clinical AI for patient QA
Thio, S., Lewis, M., Denaxas, S., & Dobson, R. J. B. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1780700 · Cited by 6
“Research by Zubiaga and others highlights that without access to external veridical knowledge, LLMs are prone to “hallucinations” generating plausible but factually incorrect information (7, 8, 16, 17).”
Salem, A., & Sumi, K. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1493348 · Cited by 5
Metadata
- Authors
- Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P.
- Affiliations
- Hong Kong University of Science and Technology (HK)
- ORCID
- Venue
- ACM Computing Surveys
- Published
- 2023-03-03
- Volume
- 55
- Issue
- 12
- Pages
- 1-38
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4309674289
- Semantic Scholar
- 3def68bd0f856886d34272840a7f81588f2bc082
- Cited by
- 5,025
- References
- 111
- Citations by year
- 2026: 2296 · 2025: 1441 · 2024: 817 · 2023: 453
- FWCI
- 224.36
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Computing Surveys (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- hallucinations, natural language generation, hallucination mitigation, abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine translation, large language models, transformer models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.




