Survey of Hallucination in Natural Language Generation
Open accessThe DOI matches its Crossref record; 355 of the 360 references it cites were found in a registry or an index, and 4 are not checked yet; 224 of the 256 sources its claims cite were checked for support.
Abstract
Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of sequence-to-sequence deep learning technologies such as Transformer-based language models. This advancement has led to more fluent and coherent NLG, leading to improved development in downstream tasks such as abstractive summarization, dialogue generation, and data-to-text generation. However, it is also apparent that deep learning based generation is prone to hallucinate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios. To address this issue, many studies have been presented in measuring and mitigating hallucinated texts, but these have never been reviewed in a comprehensive manner before. In this survey, we thus provide a broad overview of the research progress and challenges in the hallucination problem in NLG. The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions, and (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, and machine translation. This survey serves to facilitate collaborative efforts among researchers in tackling the challenge of hallucinated texts in NLG.
Verified 2026-11-19 · 4 references not checked yet
Overview
CoChat Summary
TL;DR
This survey provides a broad overview of hallucination in natural language generation, covering definitions, contributors, metrics, mitigation methods, and task-specific progress across summarization, dialogue, GQA, data-to-text, machine translation, vision-language generation, and LLMs. It distinguishes intrinsic from extrinsic hallucination and argues that hallucination must be treated differently across tasks and sources.
Key findings
- The survey categorizes hallucination into intrinsic hallucination, where generated output contradicts the source, and extrinsic hallucination, where output cannot be verified from the source; extrinsic hallucination is not always erroneous because it may come from factually correct external information.
- It reports prevalence examples from cited work: 25% of summaries generated by state-of-the-art abstractive summarization systems have hallucinated content; 62% of first sentences in WIKIBIO have additional information not stated in the corresponding infobox; and 74% of samples in the QMSum dataset consist of inconsistent facts.
- It reviews hallucination metrics as statistical, model-based (IE-based, QA-based, NLI-based, faithfulness classification, LM-based), and human evaluation, and mitigation methods as data-related and modeling/inference methods including architecture changes, planning/sketching, RL, multi-task learning, controllable generation, and post-processing.
- For LLMs, it notes that RLHF reduced the hallucination rate from 41% to 21% in closed-domain tasks for InstructGPT, and discusses reference-dependent and reference-free hallucination metrics plus mitigation via data refinement, RAG, safety fine-tuning, model editing, decoding methods, chain-of-thought variants, post-processing, and ensembling.
Methods
- Literature survey design: the paper reviews existing research on hallucination in NLG rather than conducting new experiments or primary data collection.
- Coverage: general NLG hallucination topics (definitions, contributors, metrics, mitigation, future directions) plus task-specific sections on abstractive summarization, dialogue generation, generative question answering, data-to-text generation, neural machine translation, vision-language generation, and large language models.
- Analysis: synthesizes and compares definitions, metrics, mitigation approaches, and future directions across tasks, with examples in tables and figures; the LLM section was updated in Jan 2024.
Limitations
- The paper states that GQA has no standard definition or categorization of hallucination and no automatic metric specifically for hallucination in GQA; it also notes that open-domain dialogue lacks a standard hallucination metric and that VL hallucination research is still in its early stage.
- The paper states that extrinsic hallucination has been somewhat overlooked and is more challenging to reduce, and that most existing mitigation methods do not distinguish between intrinsic and extrinsic hallucination.
- Evident from the text: as a survey, it does not provide new empirical validation of metrics or mitigation methods, and its LLM coverage is explicitly updated only through Jan 2024, so later work is not included.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
124 Valid · 26 Not valid · 19 Not checked · 135 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Furthermore, perturbation, where the insertions of most common tokens are placed at the beginning of the sentence, seems to be the most successful in hallucination mitigation.”
What the sources say
- Not checked — no full text of the sourcecites [133] [133] Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fannjiang, and David Sussillo. 2019. Hallucinations in Neural Machine Translation. ICLR (2019). -
- Disputedcites [215] [215] Vikas Raunak, Arul Menezes, and Marcin Junczys-Dowmunt. 2021. The Curious Case of Hallucinations in Neural Machine Translation. (4 2021), 1172-1183. https://arxiv.org/abs/2104.06683v1 - sourceExperimental — accuracy not yet measured
In the passage describing the perturbation procedure (page unavailable), the paper says that inserting common tokens at the beginning is used to generate hallucinations, not mitigate them. Its prevention section instead discusses approaches such as robust learning and corpus-level filtering. (LLM, conf=0.92)
Checked by a reasoning model — may be wrong.
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“While there are works on the visual extension of faithfulness-oriented RLHF [303], these essentially involve labeling more data, which is expensive and difficult to scale.”
What the sources say
- Disputedcites [303] [303] Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, and Tat-Seng Chua. 2023. RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback. CoRR abs/2312.00849 (2023). https://doi.org/10.48550/ARXIV.2312.00849 arXiv:2312.00849 - sourceExperimental — accuracy not yet measured
RLHF-V does use human annotation for visual-model alignment, but it makes the contrary point about needing more labeled data. The Fine-grained Correctional Human Preference Collection section says its corrections improve data efficiency, and Section 4.3 reports comparable hallucination rates using only 200 preference examples versus an order of magnitude more ranking-labeled examples. Our copy does not establish a separate claim about annotation cost or scalability. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“Nevertheless, as illustrated by Falke et al. [66], off-the-shelf NLI models tend to transfer poorly to the abstractive summarization task.”
What the sources say
- Supportedcites [66] [66] Tobias Falke, Leonardo FR Ribeiro, Prasetya Ajie Utama, Ido Dagan, and Iryna Gurevych. 2019. Ranking Generated Summaries by Correctness: An Interesting but Challenging Application for Natural Language Inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . 2214-2220. - source
The Introduction states that models trained on NLI datasets transfer poorly to the paper’s downstream task of ranking generated summaries by correctness. The Conclusions likewise say that out-of-the-box NLI models do not perform well on that task. (LLM, conf=0.94)
- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Such wrong association results in generation with facts mixed up between two similar entities [60, 233].”
What the sources say
- Partly supportedcites [60] [60] Nouha Dziri, Andrea Madotto, Osmar Zaiane, and Avishek Joey Bose. 2021. Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding. EMNLP (2021). - sourceExperimental — accuracy not yet measured
The GPT2-KG passage describes an incorrect association between entities, and G Error Analysis gives an example of a generated response assigning a book to the wrong author. These support the claim that facts can be mixed up, but the paper does not say the entities involved are similar. The supplied passages do not provide page numbers. (LLM, conf=0.83)
- Supportedcites [233] [233] Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval Augmentation Reduces Hallucination in Conversation. EMNLP (2021). - source
The Introduction states that language models often mix up facts between two similar entities when generating factually incorrect statements. This directly supports the cited description of the resulting error. (LLM, conf=0.92)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“As a result, the target reference may contain information that cannot be supported by the source [198, 268].”
What the sources say
- Supportedcites [198] [198] Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. ToTTo: A Controlled Table-To-Text Generation Dataset. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . 1173-1186. - source
The Dataset section says that reference targets selected from web sentences may contain information that cannot be inferred from the source. The Data Annotation Process section also describes deleting phrases unsupported by table cells. (LLM, conf=0.93)
- Supportedcites [268] [268] Hongmin Wang. 2019. Revisiting Challenges in Data-to-Text Generation with Fact Grounding. In Proceedings of the 12th International Conference on Natural Language Generation . 311-322. - source
The Introduction reports that about 40% of human-written game-summary content cannot be directly mapped to the input boxscore records and draws on information beyond those inputs. Section 2.1 likewise describes reference content not licensed by the boxscore table. This supports the claim that a target reference may contain information the source does not support. (LLM, conf=0.93)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
117 more claims · 74 with their cited sources checked
Create a free account to see all 304 claims and to verify the 2 references still open.
- Borderline: one of the tests for a substantial claim is unclear. 1
- It may state two separate claims as one. 90
- Probably not a substantial claim. 142
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Hallucination can lead to serious consequences such as incorrect or misleading code, security risks, and ethical issues [186].”
From A survey on large language model based autonomous agents (2024) · cites this paper as [186]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“citation fabrication constitutes a form of extrinsic hallucination in the absence of retrieval augmentation, since the model generates references not grounded in any provided source material [4].”
From How LLMs Cite and Why It Matters: A Cross-Model Audit of Reference Fabrication in AI-Assisted Academic Writing and Methods to Detect Phantom Citations (2026) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent taxonomic work distinguishes at least seven cases [15,18].”
From Hallucination as epistemic pollution: epistemic injustice and credibility in futures-oriented science education (2026) · cites this paper as [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs struggle to generate accurate and up-to-date responses on current topics, due to outdated knowledge, lack of domain-specific expertise, or hallucination13–17.”
From Leveraging long context in retrieval augmented language models for medical question answering (2025) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Fine-tuning can enrich LLMs with domain knowledge, enhance their specific ability, improve the fairness and reliability of the outputs, and prevent certain damage caused by hallucination [26].”
From FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning (2024) · cites this paper as [26]
Experimental — accuracy not yet measured
41 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Extrinsic Object Hallucination
“CHAIR (Caption Hallucination Assessment with Image Relevance) metric, which calculates what proportion of object words generated are actually in the image according to the ground truth captions.”
§ Extrinsic Object Hallucination
“CHAIR 𝑖 measures per-instance object hallucination, i.e. what fraction of object instances in each generated caption are hallucinated.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 7.1 Hallucination Definition in Abstractive Summarization
“For instance, in Table 1, given the input article shown in the caption, an example of intrinsic hallucination is ' The Ebola vaccine was rejected by the FDA in 2019, '”
§ 2.1 Categorization
“in the abstractive summarization task from Table 1, the generated summary ' The first Ebola vaccine was approved in 2021 ' contradicts the source content”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Survey of Hallucination in Natural Language Generation
“The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine translation, and visual-language generation; and (3) hallucinations in large language models (LLMs)”
§ 1 INTRODUCTION
“The third part discusses this phenomenon in LLMs in Section 13.”
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“Recently, the rapid development of NLG has captured the imagination of many thanks to the advances in deep learning technologies, especially Transformer [189]-based models like BERT [29], BART [100], GPT-2 [149], and GPT-3 [16].”
BackgroundAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“This work investigates the performances of GPT-3 [13], GPT-Neo/J [141], GPT-2 [105] and a T5-based model [106].”
BackgroundAdvances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 8,890
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · cited 1×
“Recently, the development of NLG has conquered the imagination of many thanks to the advances in deep learning technologies, especially the Transformer [159]-based models like BERT [24], BART [83], GPT-2 [127], and GPT3 [13].”
BackgroundLewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816 · cited 1×
“Recently, the development of NLG has conquered the imagination of many thanks to the advances in deep learning technologies, especially the Transformer [159]-based models like BERT [24], BART [83], GPT-2 [127], and GPT3 [13].”
Background
Research works printed but not linked: 16
- [4] Joshua Albrecht and Rebecca Hwa. 2007. A Re-examination of Machine Learning Approaches for Sentence-Level MT Evaluation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics . Association for Computational Linguistics, Prague, Czech Republic, 880-887. https://aclanthology.org/P07-1111 -
- [16] Andrzej Białecki, Robert Muir, Grant Ingersoll, and Lucid Imagination. 2012. Apache lucene 4. In SIGIR 2012 workshop on open source information retrieval . 17. -
- [53] Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. 2023. Improving Factuality and Reasoning in Language Models through Multiagent Debate. In Forty-first International Conference on Machine Learning . -
- [121] Philipp Koehn and Rebecca Knowles. 2017. Six Challenges for Neural Machine Translation. In First Workshop on Neural Machine Translation . Association for Computational Linguistics, 28-39. -
- [145] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation. In ICML . -
- [149] Tian Li, Ahmad Beirami, Maziar Sanjabi, and Virginia Smith. 2020. Tilted Empirical Risk Minimization. In International Conference on Learning Representations . -
- [164] Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. 2018. Neural Baby Talk. In CVPR . -
- [165] Yukun Ma, Khanh Linh Nguyen, Frank Z Xing, and Erik Cambria. 2020. A Survey on Empathetic Dialogue Systems. Information Fusion 64 (2020), 50-70. -
- [175] Marianna J. Martindale, Marine Carpuat, Kevin Duh, and Paul McNamee. 2019. Identifying Fluently Inadequate Output in Neural and Statistical Machine Translation. In MTSummit . -
- [179] Kathleen McKeown. 1992. Text Generation . Cambridge University Press. -
- [209] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. OpenAI Blog 1, 8 (2019), 9. -
- [238] Hui Su, Xiaoyu Shen, Sanqiang Zhao, Zhou Xiao, Pengwei Hu, Cheng Niu, and Jie Zhou. 2020. Diversifying Dialogue Generation with Non-Conversational Text. In 58th Annual Meeting of the Association for Computational Linguistics . ACL, 7087-7097. -
- [256] Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li. 2017. Neural Machine Translation with Reconstruction. In Thirty-First AAAI Conference on Artificial Intelligence . -
- [261] Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. 2023. A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation. CoRR abs/2307.03987 (2023). https://doi.org/10.48550/arXiv.2307.03987 arXiv:2307.03987 - · arXiv · doi:10.48550/arxiv.2307.03987 · Paper page
- [285] Zeqiu Wu, Michel Galley, Chris Brockett, Yizhe Zhang, Xiang Gao, Chris Quirk, Rik Koncel-Kedziorski, Jianfeng Gao, Hannaneh Hajishirzi, Mari Ostendorf, et al. 2021. A Controllable Model of Grounded Response Generation. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 35. 14085-14093. -
- [291] Xinnuo Xu, Ondřej Dušek, Verena Rieser, and Ioannis Konstas. 2021. AGGGEN: Ordering and Aggregating while Generating. roceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL2021) (2021). -
Cited by
Items 801–900 of 4,844
AI to Train AI: Using ChatGPT to Improve the Accuracy of a Therapeutic Dialogue System
Gabor-Siatkowska, K., Sowański, M., Rzatkiewicz, R., Stefaniak, I., Kozłowski, M., & Janicki, A. (2023) · Electronics · doi:10.3390/electronics12224694 · Cited by 20
Akhtar, M., Shankarampeta, A., Gupta, V., Patil, A., Cocarascu, O., & Simperl, E. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.1028 · Cited by 17
Exploring Large Language Models for Multi-Modal Out-of-Distribution Detection
Dai, Y. Q., Lang, H., Zeng, K., Huang, F., & Li, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.351 · Cited by 38
Potential merits and flaws of large language models in epilepsy care: A critical review
Diessen, E. G. A. L. V., van Amerongen, R. A., Zijlmans, M., & Otte, W. M. (2024) · Epilepsia · doi:10.1111/epi.17907 · Cited by 26
Two artificial intelligence models underperform on examinations in a veterinary curriculum
Coleman, M. C., & Moore, J. N. (2024) · Journal of the American Veterinary Medical Association · doi:10.2460/javma.23.12.0666 · Cited by 65
“25 Irrespective of the overall performance, the consistency of results is also important for educators and students to rec - ognize.”
BackgroundThe Intersection of Generative AI and Healthcare: Addressing Challenges to Enhance Patient Care
Albaroudi, E., Mansouri, T., & Alameer, A. (2024) · International Women in Data Science Conference at Prince Sultan University (WiDS PSU) · doi:10.1109/wids-psu61003.2024.00039 · Cited by 29
CorpusLM: Towards a Unified Language Model on Corpus for Knowledge-Intensive Tasks
Li, X., Dou, Z.-C., Zhou, Y.-J., & Liu, F.-C. (2024) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657778 · Cited by 35
On Early Detection of Hallucinations in Factual Question Answering
Snyder, B., Moisescu, M., & Zafar, M. B. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671796 · Cited by 98
Next‐generation human‐robot interaction with ChatGPT and robot operating system
Koubâa, A., Ammar, A., & Boulila, W. (2024) · Software Practice and Experience · doi:10.1002/spe.3377 · Cited by 34
“29 In one of our experiments, we observed a clear example of hallucination, as shown in Figure 11.”
Beyond Scores: A Modular RAG-Based System for Automatic Short Answer Scoring with Feedback
Fateen, M., Wang, B., & Mine, T. (2024) · IEEE Access · doi:10.1109/access.2024.3508747 · Cited by 37
Ullah, A., Meng, Y., & Mendes, J. F. F. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104201 · Cited by 17
Navigating Complexity in Software Engineering: A Prototype for Comparing GPT-n Solutions
Treude, C. (2023) · IEEE/ACM International Workshop on Bots in Software Engineering (BotSE) · doi:10.1109/botse59190.2023.00008 · Cited by 32
“One problem with large language models is that they can sometimes “hallucinate” [24].”
BackgroundChatGPT and GPTZero in Research and Social Media: A Sentiment-and Topic-based AnalysisPreprint
Heumann, M., Kraschewski, T., & Breitner, M. H. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4467646 · Cited by 21
Duolingo English Test Responsible AI Standards
Burstein, J. C. (2023) · doi:10.46999/vcae5025 · Cited by 36
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ac4sp · Cited by 157
MisCaltral: Reducing Numeric Hallucinations of Mistral with Precision Numeric CalculationPreprint
Chen, S.-W., & Hsu, H.-J. (2023) · Research Square · doi:10.21203/rs.3.rs-3789011/v1 · Cited by 18
Developing a Framework for Auditing Large Language Models using Human-in-the-loop
Amirizaniani, M., Lavergne, A., Snell Okada, E., Chadha, A., Roosta, T., & Shah, C. (2025) · Proceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region · doi:10.1145/3767695.3769514 · Cited by 16
The Internal State of an LLM Knows When It's Lying
Azaria, A., & Mitchell, T. (2023) · arXiv · doi:10.48550/arxiv.2304.13734 · Cited by 179
“However, when composing a response, LLMs tend to hallucinate facts and provide inaccurate information (Ji et al., 2023).”
BackgroundPoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models
Zou, W., Geng, R., Wang, B., & Jia, J. (2024) · arXiv · doi:10.48550/arxiv.2402.07867 · Cited by 16
“…are pre-trained on past data (e.g., the cutoff date for the pre-training data of GPT-4 is April 2023 [2]); they exhibit hallucination behaviors [4] (e.g., generate inaccurate content); they could have gaps of knowledge in particular domains (e.g., medical domain), especially when the data is…”
BackgroundDetecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents
Rao, D., Wong, E., & Callison-Burch, C. (2026) · arXiv · doi:10.48550/arxiv.2604.03173 · Cited by 16
Chan, S., Li, J., Yao, B., Mahmood, A., Huang, C.-M., Jimison, H., Mynatt, E. D., & Wang, D. (2023) · arXiv · doi:10.48550/arxiv.2310.05853 · Cited by 16
International AI Safety Report
Bengio, Y., Mindermann, S., Privitera, D., Besiroglu, T., Bommasani, R., Casper, S., Choi, Y., Fox, P., Garfinkel, B., Goldfarb, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.17805 · Cited by 16
Self-citationModel Editing Harms General Abilities of Large Language Models: Regularization to the Rescue
Gu, J.-C., Xu, H.-X., Ma, J.-Y., Lu, P., Ling, Z.-H., Chang, K.-W., & Peng, N. (2024) · arXiv · doi:10.48550/arxiv.2401.04700 · Cited by 16
Kaniwa, Y., Kuribayashi, M., Kayukawa, S., Sato, D., Takagi, H., Asakawa, C., & Morishima, S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3676492 · Cited by 32
Understanding the impact of AI Hallucinations on the university community
Kamel, H. (2024) · Cybrarians Journal · doi:10.70000/cj.2024.73.622 · Cited by 26
Agent-Based Modelling Meets Generative AI in Social Network Simulations
Ferraro, A., Galli, A., Gatta, V. L., Postiglione, M., Orlando, G. M., Russo, D., Riccio, G., Romano, A., & Moscato, V. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-78541-2_10 · Cited by 69
Improving AI in CS50: Leveraging Human Feedback for Better Learning
Liu, R., Zhao, J., Xu, B. W., Perez, C., Zhukovets, Y., & Malan, D. J. (2025) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3641554.3701945 · Cited by 30
Sun, S., Zhang, K., Li, J., Yu, M., Hou, K., Wang, Y., & Cheng, X. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104147 · Cited by 25
Reducing hallucinations of large language models via hierarchical semantic piece
Liu, Y., Yang, Q., Tang, J.-W., Guo, T., Chen, W., Pan, L., Sai, X., Gao, X., Li, Z., Liu, J., et al. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-01833-9 · Cited by 26
“In practice, LLMs often generate hallucinations in their outputs, where the generated text is fluent but disconnected from the objective facts or the context [6].”
SEMINT: an LLM-empowered long-term vessel trajectory prediction framework
Chen, N., Yang, A., Wu, H., Chen, L., Xiong, W., & Jing, N. (2025) · International Journal of Geographical Information Systems · doi:10.1080/13658816.2025.2487990 · Cited by 22
Classifying legal interpretations using large language models
Dugac, G., & Altwicker, T. (2025) · Artificial Intelligence and Law · doi:10.1007/s10506-025-09447-9 · Cited by 30
AI Narrative Modeling: How Machines’ Intelligence Reproduces Archetypal Storytelling
Kabashkin, I., Zervina, O., & Mišņevs, B. (2025) · Information · doi:10.3390/info16040319 · Cited by 15
“The resources required to train an LLM prevent these models from easily expanding their memory, leading to what are called “hallucinations” [10] when generating unknown content.”
Prabhudesai, S., Kasi, A. P., Mansingh, A., Antar, A. D., Shen, H., & Banović, N. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713714 · Cited by 35
The bewitching AI: The Illusion of Communication with Large Language Models
Bottazzi Grifoni, E., & Ferrario, R. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00893-6 · Cited by 15
“8 We argue that emphasizing communication is the most efficacious approach also to the alignment problem (Ji et al., 2023a), though a full discussion lies beyond our present scope.”
InfluentialCAT+: Investigating and Enhancing Audio-Visual Understanding in Large Language Models
Ye, Q., Yu, Z., Shao, R., Cui, Y., Kang, X., Liu, X., Torr, P. H. S., & Cao, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3582389 · Cited by 42
“Indeed, the problem of hallucinations has long been defined in the field of natural language processing as generated content that deviates from the intended meaning and makes no sense [18], [38], [62], [82].”
When generative AI messes up:How politeness and attribution shape user reactions to hallucinations
Kim, H., Seo, S., & Lee, S. W. (2025) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2025.102958 · Cited by 26
Yu, M., Liu, Z., Long, T., Dong, L., Deng, L., Kong, X., & Sun, J. (2025) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-025-00545-x · Cited by 28
“Additionally, GenAI tools can generate hallucinations, which are confident yet factually inaccurate responses, significantly increasing the risk of spreading unverified or misleading information (Ji et al., 2023).”
Khoo, T. L., Lee, T. S., Bee, S., Ma, C., & Zhang, Y.-Y. (2025) · Processes · doi:10.3390/pr13092680 · Cited by 23
LLM4DSR: Leveraging Large Language Model for Denoising Sequential Recommendation
Wang, B., Liu, F., Zhang, C., Chen, J., Wu, Y., Zhou, S., Lou, X., Wang, J., Feng, Y., Chen, C., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3762182 · Cited by 68
Reflecting on LLM Support in Reflexive Thematic Analysis: An Exploratory Study
Vikan, M., Aryan, R., Kannelønning, M. S., Riegler, M., & Danielsen, S. O. (2025) · Qualitative Health Research · doi:10.1177/10497323251365211 · Cited by 23
“Owing to the increased quality and perceived credibility of LLM responses, errors become challenging to identify, and the risk of taking false facts into the analysis increases (Ji et al., 2023).”
Monks, T., Harper, A. L., & Heather, A. (2025) · Journal of the Operational Research Society · doi:10.1080/01605682.2025.2554751 · Cited by 21
“Hallucination is a major limitation of generative AI and hence is an active area of research (Ji et al., 2023).”
InfluentialLee, J., Kim, J.-Y., Kim, H., Lee, I., & Ryu, S. (2025) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2025.11.007 · Cited by 23
AIOps for log anomaly detection in the era of LLMs: A systematic literature review
Cabello, M. D. L. C., Sales, T. P., & Machado, M. (2025) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2025.200608 · Cited by 51
Adejumo, A. A., Oyelere, S. S., Sanusi, I. T., & Suhonen, J. T. (2026) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2026.100570 · Cited by 29
“Intrinsic hallucinations occur when generated content contradicts source information, whereas extrinsic hallucinations produce information unrelated to the source and neither supported nor refuted by it (Ji et al., 2023; Patel, 2024).”
Interactive Table Synthesis With Natural Language
Huang, Y., Zhou, Y., Chen, R., Pan, C., Shu, X., Weng, D., & Wu, Y. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3329120 · Cited by 20
“Incremental Refinement of Synthesized Tables: Experimental studies have demonstrated that LLMs can produce inaccurate or unfaithful results in practice [23], [24], [25], including in scenarios related to data transformation [23].”
Ding, Z., Smith, A., Zhang, W., Tetreault, J., & Jaimes, A. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.217 · Cited by 45
Towards a Hybrid Security Framework for Phishing Awareness Education and Defense
Loh, P. K. K., Lee, A. Z. Y., & Balachandran, V. (2024) · Future Internet · doi:10.3390/fi16030086 · Cited by 32
Measuring and Modifying Factual Knowledge in Large Language Models
Pezeshkpour, P. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00122 · Cited by 52
“Furthermore, recent studies (Goodrich et al., 2019; Shuster et al., 2021; Ji et al., 2023) utilize factual knowledge to detect and mitigate hallucination in the generated output of LLMs.”
BackgroundLarge Language Model Powered Agents in the Web
Deng, Y., Zhang, A., Lin, Y., Chen, X., Wen, J.-R., & Chua, T. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3641240 · Cited by 33
Artificial intelligence hallucinations in anaesthesia: Causes, consequences and countermeasures
Gondode, P. G., Duggal, S., & Mahor, V. (2024) · Indian Journal of Anaesthesia · doi:10.4103/ija.ija_203_24 · Cited by 24
“[3] Model complexity: A highly complex AI system tasked with monitoring vital signs during surgery may exhibit hallucinatory responses when encountering unusual physiological patterns.”
Boosting Conversational Question Answering with Fine-Grained Retrieval-Augmentation and Self-Check
Ye, L., Lei, Z., Yin, J., Chen, Q., Zhou, J., & He, L. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657980 · Cited by 51
Abernethy, N. J. (2024) · AI and Ethics · doi:10.1007/s43681-024-00575-7 · Cited by 20
MyVLM: Personalizing VLMs for User-Specific Queries
Alaluf, Y., Richardson, E., Tulyakov, S., Aberman, K., & Cohen‐Or, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72624-8_5 · Cited by 85
Richter, E., Spitzer, M., Morgan, A., Frede, L., Weidlich, J., & Moeller, K. (2025) · Trends in Neuroscience and Education · doi:10.1016/j.tine.2025.100255 · Cited by 20
“…however, such queries pose particular challenges, as LLMs are prone to hallucinations by generating incorrect content or answers (e.g., [68,69]) and sycophantic behavior , which means they tend to align their responses with users beliefs or implied assumptions, even if these are factually…”
COPEN: Probing Conceptual Knowledge in Pre-trained Language Models
Peng, H. M., Wang, X., Hu, S., Jin, H., Hou, L., Li, J., Liu, Z., & Liu, Q. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.335 · Cited by 68
ChatClimate: Grounding Conversational AI in Climate SciencePreprint
Vaghefi, S. A., Wang, Q., Muccione, V., Ni, J., Kraus, M., Bingler, J., Schimanski, T., Senni, C. C., Webersinke, N., Huggel, C., et al. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4414628 · Cited by 32
Retrieval Augmented Generation for Large Language Models in Healthcare: A Systematic ReviewPreprint
Amugongo, L. M., Mascheroni, P., Brooks, S. G., Doering, S., & Seidel, J. (2024) · Preprints.org · doi:10.20944/preprints202407.0876.v1 · Cited by 15
Narrative Player: Reviving Data Narratives With Visuals
Shao, Z., Shen, L., Li, H., Shan, Y., Qu, H., Wang, Y., & Chen, S. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3530512 · Cited by 15
“When prompting LLM, considering its hallucination nature [28], we first try to achieve a comprehensive data fact set for each clause by utilizing three sessions, each using the same prompt with temperature 0.3 and generating three fact candidates per clause.”
MethodInfluentialTowards Faithful and Robust LLM Specialists for Evidence-Based Question-Answering
Schimanski, T., Ni, J., Kraus, M., Ash, E., & Leippold, M. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.105 · Cited by 15
RTLFixer: Automatically Fixing RTL Syntax Errors with Large Language Models
Tsai, Y.-D., Liu, M., & Ren, H. (2023) · arXiv · doi:10.48550/arxiv.2311.16543 · Cited by 15
Gender, Age, and Technology Education Influence the Adoption and Appropriation of LLMs
Draxler, F., Buschek, D., Tavast, M., Hämäläinen, P., Schmidt, A., Kulshrestha, J., & Welsch, R. (2023) · arXiv · doi:10.48550/arxiv.2310.06556 · Cited by 15
Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models
Amayuelas, A., Wong, K., Pan, L., Chen, W., & Wang, W. (2023) · arXiv · doi:10.48550/arxiv.2305.13712 · Cited by 15
“Consequently, these models may exhibit overconfidence by generating responses that are unwarranted or inaccurate, commonly referred to as "hallucinations" (Ji et al., 2023), which produce nonsensical or unfaithful text.”
BackgroundPark, S.-A., & Kim, H. Y. (2025) · Women s Health Nursing · doi:10.4069/whn.2025.06.18 · Cited by 14
Artificial intelligence and the dawn of an algorithmic divide
Eder, M., & Sjøvaag, H. (2024) · Frontiers in Communication · doi:10.3389/fcomm.2024.1453251 · Cited by 26
“On the one hand, journalists need to be aware of the algorithm’s influence on their gatekeeping role within the newsgathering process (Cools et al., 2021) and that generative AI can produce false information (Ji et al., 2023).”
Users do not trust recommendations from a large language model more than AI-sourced snippets
McGrath, M. J., Cooper, P. S., & Duenser, A. (2024) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1456098 · Cited by 15
Seki, T., Kawazoe, Y., Ito, H., Akagi, Y., Takiguchi, T., & Ohe, K. (2025) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2025.1458289 · Cited by 25
“Visual question answering (VQA) entails providing a relevant answer based on an image and natural language query, necessitating image interpretation and intricate reasoning (15).”
LLM-CDM: A Large Language Model Enhanced Cognitive Diagnosis for Intelligent Education
Chen, X., Zhang, J., Zhou, T., & Zhang, F. (2025) · IEEE Access · doi:10.1109/access.2025.3549309 · Cited by 20
“Although LLMs have broad application prospects in education, they also face the ‘‘hallucination’’ problem [20], [21], [22], [23], [24], in which the model may generate information that sounds reasonable but is inaccurate or fabricated.”
Greussing, E., Guenther, L., Baram‐Tsabari, A., Dabran‐Zivan, S., Jonas, E., Klein-Avraham, I., Taddicken, M., Agergaard, T. E., Beets, B., Brossard, D., et al. (2025) · Journal of Science Communication · doi:10.22323/2.24020205 · Cited by 22
Alhusban, M. I., Khatatbeh, I. N., & Alshurafat, H. (2025) · Competitiveness Review An International Business Journal incorporating Journal of Global Competitiveness · doi:10.1108/cr-06-2024-0121 · Cited by 32
Hondjeu, A. R. M., Zhao, Z. Y., Newton, L., Ajenkar, A., Hladkowicz, E., Ladha, K. S., Wijeysundera, D. N., & McIsaac, D. I. (2025) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-025-02980-w · Cited by 27
“55 Hallucinations can be broadly categorized into two types: 1.”
REIS: A High-Performance and Energy-Efficient Retrieval System with In-Storage Processing
Chen, K., Nadig, R., Frouzakis, M., Ghiasi, N. M., Liang, Y., Mao, H., Park, J., Sadrosadati, M., & Mutlu, O. (2025) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3695053.3731116 · Cited by 26
“Third, REIS leverages an ANNS engine that uses the existing computational resources inside the storage system, without requiring hardware modifications.”
Hu, W., & Shao, Z. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100735 · Cited by 24
Wang, P., Liu, Z., Li, Y., Holmes, J., Shu, P., Zhang, L., Li, X., Li, Q., Laughlin, B., Toesca, D. A. S., et al. (2025) · Medical Physics · doi:10.1002/mp.17985 · Cited by 23
Sun, Z., Yim, W.-W., Uzuner, Ö., Xia, F., & Yetişgen, M. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104866 · Cited by 26
An, W., Tian, F., Leng, S., Nie, J., Lin, H., Wang, Q., Chen, P., Zhang, X., & Lu, S. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02784 · Cited by 155
“Hallucination [28, 55, 76] in the NLP community is defined as the generation of incorrect or nonsensical content.”
Wu, X., Tian, Y., Chen, Y., Ye, P., Cui, X., Jia, J., Li, S., Liu, J., & Niu, W. (2025) · Applied Sciences · doi:10.3390/app15169096 · Cited by 34
The Impact of AI on Students’ Reading, Critical Thinking, and Problem-Solving Skills
Hassen, M. Z. (2025) · American Journal of Education and Information Technology · doi:10.11648/j.ajeit.20250902.12 · Cited by 28
AI-generated content in cross-domain applications: Research trends, challenges and propositions
Li, J., Qu, L., Cai, T., Zhao, Z., Haldar, N. A. H., Krishna, A., Kong, X., Macau, F., Chakraborty, T., Deroy, A., et al. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114634 · Cited by 34
“While black-box self-consistency checks and white-box activation-based signals can improve calibration, they remain insu ffi ciently reliable for deployment at platform scale [47, 76].”
(Generative) AI in Financial Economics
Mo, H., & Ouyang, S. (2025) · Journal of Chinese Economic and Business Studies · doi:10.1080/14765284.2025.2569006 · Cited by 46
Tres escenarios para la IA en educación: del apoyo responsable a la cocreación
García‐Peñalvo, F. J. (2025) · Education in the Knowledge Society (EKS) · doi:10.14201/eks.32932 · Cited by 19
Jadhav, R., Meshram, V., Bhosle, A. A., Patil, K., Dash, S., & Jadhav, S. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1690616 · Cited by 28
“…analyses also link the generation of synthetic misinformation to broader issues of hallucination and factual unreliability in large language models (Ji et al., 2023), underscoring the need for adversarial evaluation of detectors trained on content generated by LLM. Transformers are vulnerable to…”
Bias in AI systems: integrating formal and socio-technical approaches
Ahmad, A., Vallès, Y., & Idaghdour, Y. (2026) · Frontiers in Big Data · doi:10.3389/fdata.2025.1686452 · Cited by 33
Ding, C., Wang, Z., Tan, Z., & Zhang, X. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132862 · Cited by 20
SafeText: A Benchmark for Exploring Physical Safety in Language Models
Levy, S., Allaway, E., Subbiah, M., Chilton, L. B., Patton, D. U., McKeown, K. R., & Wang, W. Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.154 · Cited by 69
Chatbot Confabulations Are Not Hallucinations—Reply
Brender, T. D. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.3875 · Cited by 14
CarExpert: Leveraging Large Language Models for In-Car Conversational Question Answering
Rony, M. R. A. H., Suess, C., Bhat, S. R., Sudhi, V., Schneider, J., Vogel, M., Teucher, R., Friedl, K. E., & Sahoo, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.56 · Cited by 22
Zhou, Z., Yang, T., & Hu, K. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385748 · Cited by 22
“For content that cannot be answered, it frequently generates seemingly reasonable but actually incorrect information, known as the “hallucination” [3] problem.”
BackgroundCherumanal, S. P., Tian, L., Abushaqra, F. M., de Paula, A. F. M., Ji, K., Ali, H., Hettiachchi, D., Trippas, J. R., Scholer, F., & Spina, D. (2024) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3627508.3638309 · Cited by 23
Manakul, P., Liusie, A., & Gales, M. (2023) · Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.ijcnlp-main.4 · Cited by 14
Saleshat: A LLM-Based Social Robot for Human-Like Sales Conversations
Hanschmann, L., Gnewuch, U., & Maedche, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-54975-5_4 · Cited by 28
Fan, H., Fuh, J. Y. H., Lu, W. F., Kumar, A., & Li, B. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.01.125 · Cited by 15
Akademik Yazımda Yapay Zekâ Kullanımının Etik Açıdan İncelenmesi: ChatGPT Örneği
Büyükada, S. (2024) · Rize İlahiyat Dergisi · doi:10.32950/rid.1337208 · Cited by 27
Can Vision-Language Models be a Good Guesser? Exploring VLMs for Times and Location Reasoning
Zhang, G., Zhang, Y., Zhang, K., & Volker Tresp, C. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00069 · Cited by 37
“Please zoom in for a better view. on contexts, such as in-context demonstrations and instructions, and expose the hallucination problem [14].”
BackgroundDetecting and Mitigating the Ungrounded Hallucinations in Text Generation by LLMs
Wei, Z., Guo, D., Huang, D., Zhang, Q., Zhang, S., Jiang, K., & Li, R. (2023) · International Conference on Artificial Intelligence, Systems and Network Security (AISNS) · doi:10.1145/3661638.3661653 · Cited by 16
APPLICATIONS OF MULTIMODAL GENERATIVE ARTIFICIAL INTELLIGENCE IN A REAL-WORLD RETINA CLINIC SETTING
Ghalibafan, S., Gonzalez, D. J. T., CAI, L. Z., Chou, B. G., Panneerselvam, S., Barrett, S. C., Djulbegovic, M. B., & Yannuzzi, N. A. (2024) · Retina · doi:10.1097/iae.0000000000004204 · Cited by 17
“24 27 Generative Pre-trained Transformer 4 with vision often gave incorrect answers when symptoms were not directly related to the diagnosis or were observed in the fellow eye.”
Prompts and Large Language Models: A New Tool for Drafting, Reviewing and Interpreting Contracts?
Wang, B. T. (2024) · Law Technology and Humans · doi:10.5204/lthj.3483 · Cited by 24
Striking the Balance in Using LLMs for Fact-Checking: A Narrative Literature Review
Dierickx, L., van Dalen, A., Opdahl, A. L., & Lindén, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-71210-4_1 · Cited by 26
Wang, X., Huang, L., Xu, S., & Lü, K. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c00882 · Cited by 26
Artificial Intelligence: Tool or Teammate?
Díaz, B., & Delgado, C. (2024) · Journal of Research in Science Teaching · doi:10.1002/tea.21993 · Cited by 14
Embodied AI-Guided Interactive Digital Teachers for Education
Zhao, Z., Yin, Z., Sun, J., & Hui, P. (2024) · doi:10.1145/3680533.3697070 · Cited by 26
“Moreover, current LLMs face problems such as domain knowledge gaps, hallucination, and factuality problems [Ji et al. 2023].”
Metadata
- Authors
- Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P.
- Affiliations
- Hong Kong University of Science and Technology (HK)
- ORCID
- Venue
- ACM Computing Surveys
- Published
- 2023-03-03
- Volume
- 55
- Issue
- 12
- Pages
- 1-38
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4309674289
- Semantic Scholar
- 3def68bd0f856886d34272840a7f81588f2bc082
- Cited by
- 5,025
- References
- 111
- Citations by year
- 2026: 2296 · 2025: 1441 · 2024: 817 · 2023: 453
- FWCI
- 224.36
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Computing Surveys (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- hallucinations, natural language generation, hallucination mitigation, abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine translation, large language models, transformer models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.




