Survey of Hallucination in Natural Language Generation
Open accessThe DOI matches its Crossref record; 355 of the 360 references it cites were found in a registry or an index, and 4 are not checked yet; 224 of the 256 sources its claims cite were checked for support.
Abstract
Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of sequence-to-sequence deep learning technologies such as Transformer-based language models. This advancement has led to more fluent and coherent NLG, leading to improved development in downstream tasks such as abstractive summarization, dialogue generation, and data-to-text generation. However, it is also apparent that deep learning based generation is prone to hallucinate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios. To address this issue, many studies have been presented in measuring and mitigating hallucinated texts, but these have never been reviewed in a comprehensive manner before. In this survey, we thus provide a broad overview of the research progress and challenges in the hallucination problem in NLG. The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions, and (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, and machine translation. This survey serves to facilitate collaborative efforts among researchers in tackling the challenge of hallucinated texts in NLG.
Verified 2026-11-19 · 4 references not checked yet
Overview
CoChat Summary
TL;DR
This survey provides a broad overview of hallucination in natural language generation, covering definitions, contributors, metrics, mitigation methods, and task-specific progress across summarization, dialogue, GQA, data-to-text, machine translation, vision-language generation, and LLMs. It distinguishes intrinsic from extrinsic hallucination and argues that hallucination must be treated differently across tasks and sources.
Key findings
- The survey categorizes hallucination into intrinsic hallucination, where generated output contradicts the source, and extrinsic hallucination, where output cannot be verified from the source; extrinsic hallucination is not always erroneous because it may come from factually correct external information.
- It reports prevalence examples from cited work: 25% of summaries generated by state-of-the-art abstractive summarization systems have hallucinated content; 62% of first sentences in WIKIBIO have additional information not stated in the corresponding infobox; and 74% of samples in the QMSum dataset consist of inconsistent facts.
- It reviews hallucination metrics as statistical, model-based (IE-based, QA-based, NLI-based, faithfulness classification, LM-based), and human evaluation, and mitigation methods as data-related and modeling/inference methods including architecture changes, planning/sketching, RL, multi-task learning, controllable generation, and post-processing.
- For LLMs, it notes that RLHF reduced the hallucination rate from 41% to 21% in closed-domain tasks for InstructGPT, and discusses reference-dependent and reference-free hallucination metrics plus mitigation via data refinement, RAG, safety fine-tuning, model editing, decoding methods, chain-of-thought variants, post-processing, and ensembling.
Methods
- Literature survey design: the paper reviews existing research on hallucination in NLG rather than conducting new experiments or primary data collection.
- Coverage: general NLG hallucination topics (definitions, contributors, metrics, mitigation, future directions) plus task-specific sections on abstractive summarization, dialogue generation, generative question answering, data-to-text generation, neural machine translation, vision-language generation, and large language models.
- Analysis: synthesizes and compares definitions, metrics, mitigation approaches, and future directions across tasks, with examples in tables and figures; the LLM section was updated in Jan 2024.
Limitations
- The paper states that GQA has no standard definition or categorization of hallucination and no automatic metric specifically for hallucination in GQA; it also notes that open-domain dialogue lacks a standard hallucination metric and that VL hallucination research is still in its early stage.
- The paper states that extrinsic hallucination has been somewhat overlooked and is more challenging to reduce, and that most existing mitigation methods do not distinguish between intrinsic and extrinsic hallucination.
- Evident from the text: as a survey, it does not provide new empirical validation of metrics or mitigation methods, and its LLM coverage is explicitly updated only through Jan 2024, so later work is not included.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
124 Valid · 26 Not valid · 19 Not checked · 135 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Furthermore, perturbation, where the insertions of most common tokens are placed at the beginning of the sentence, seems to be the most successful in hallucination mitigation.”
What the sources say
- Not checked — no full text of the sourcecites [133] [133] Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fannjiang, and David Sussillo. 2019. Hallucinations in Neural Machine Translation. ICLR (2019). -
- Disputedcites [215] [215] Vikas Raunak, Arul Menezes, and Marcin Junczys-Dowmunt. 2021. The Curious Case of Hallucinations in Neural Machine Translation. (4 2021), 1172-1183. https://arxiv.org/abs/2104.06683v1 - sourceExperimental — accuracy not yet measured
In the passage describing the perturbation procedure (page unavailable), the paper says that inserting common tokens at the beginning is used to generate hallucinations, not mitigate them. Its prevention section instead discusses approaches such as robust learning and corpus-level filtering. (LLM, conf=0.92)
Checked by a reasoning model — may be wrong.
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“While there are works on the visual extension of faithfulness-oriented RLHF [303], these essentially involve labeling more data, which is expensive and difficult to scale.”
What the sources say
- Disputedcites [303] [303] Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, and Tat-Seng Chua. 2023. RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback. CoRR abs/2312.00849 (2023). https://doi.org/10.48550/ARXIV.2312.00849 arXiv:2312.00849 - sourceExperimental — accuracy not yet measured
RLHF-V does use human annotation for visual-model alignment, but it makes the contrary point about needing more labeled data. The Fine-grained Correctional Human Preference Collection section says its corrections improve data efficiency, and Section 4.3 reports comparable hallucination rates using only 200 preference examples versus an order of magnitude more ranking-labeled examples. Our copy does not establish a separate claim about annotation cost or scalability. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“Nevertheless, as illustrated by Falke et al. [66], off-the-shelf NLI models tend to transfer poorly to the abstractive summarization task.”
What the sources say
- Supportedcites [66] [66] Tobias Falke, Leonardo FR Ribeiro, Prasetya Ajie Utama, Ido Dagan, and Iryna Gurevych. 2019. Ranking Generated Summaries by Correctness: An Interesting but Challenging Application for Natural Language Inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . 2214-2220. - source
The Introduction states that models trained on NLI datasets transfer poorly to the paper’s downstream task of ranking generated summaries by correctness. The Conclusions likewise say that out-of-the-box NLI models do not perform well on that task. (LLM, conf=0.94)
- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Such wrong association results in generation with facts mixed up between two similar entities [60, 233].”
What the sources say
- Partly supportedcites [60] [60] Nouha Dziri, Andrea Madotto, Osmar Zaiane, and Avishek Joey Bose. 2021. Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding. EMNLP (2021). - sourceExperimental — accuracy not yet measured
The GPT2-KG passage describes an incorrect association between entities, and G Error Analysis gives an example of a generated response assigning a book to the wrong author. These support the claim that facts can be mixed up, but the paper does not say the entities involved are similar. The supplied passages do not provide page numbers. (LLM, conf=0.83)
- Supportedcites [233] [233] Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval Augmentation Reduces Hallucination in Conversation. EMNLP (2021). - source
The Introduction states that language models often mix up facts between two similar entities when generating factually incorrect statements. This directly supports the cited description of the resulting error. (LLM, conf=0.92)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“As a result, the target reference may contain information that cannot be supported by the source [198, 268].”
What the sources say
- Supportedcites [198] [198] Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. ToTTo: A Controlled Table-To-Text Generation Dataset. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . 1173-1186. - source
The Dataset section says that reference targets selected from web sentences may contain information that cannot be inferred from the source. The Data Annotation Process section also describes deleting phrases unsupported by table cells. (LLM, conf=0.93)
- Supportedcites [268] [268] Hongmin Wang. 2019. Revisiting Challenges in Data-to-Text Generation with Fact Grounding. In Proceedings of the 12th International Conference on Natural Language Generation . 311-322. - source
The Introduction reports that about 40% of human-written game-summary content cannot be directly mapped to the input boxscore records and draws on information beyond those inputs. Section 2.1 likewise describes reference content not licensed by the boxscore table. This supports the claim that a target reference may contain information the source does not support. (LLM, conf=0.93)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
117 more claims · 74 with their cited sources checked
Create a free account to see all 304 claims and to verify the 2 references still open.
- Borderline: one of the tests for a substantial claim is unclear. 1
- It may state two separate claims as one. 90
- Probably not a substantial claim. 142
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Hallucination can lead to serious consequences such as incorrect or misleading code, security risks, and ethical issues [186].”
From A survey on large language model based autonomous agents (2024) · cites this paper as [186]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“citation fabrication constitutes a form of extrinsic hallucination in the absence of retrieval augmentation, since the model generates references not grounded in any provided source material [4].”
From How LLMs Cite and Why It Matters: A Cross-Model Audit of Reference Fabrication in AI-Assisted Academic Writing and Methods to Detect Phantom Citations (2026) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent taxonomic work distinguishes at least seven cases [15,18].”
From Hallucination as epistemic pollution: epistemic injustice and credibility in futures-oriented science education (2026) · cites this paper as [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs struggle to generate accurate and up-to-date responses on current topics, due to outdated knowledge, lack of domain-specific expertise, or hallucination13–17.”
From Leveraging long context in retrieval augmented language models for medical question answering (2025) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Fine-tuning can enrich LLMs with domain knowledge, enhance their specific ability, improve the fairness and reliability of the outputs, and prevent certain damage caused by hallucination [26].”
From FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning (2024) · cites this paper as [26]
Experimental — accuracy not yet measured
41 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Extrinsic Object Hallucination
“CHAIR (Caption Hallucination Assessment with Image Relevance) metric, which calculates what proportion of object words generated are actually in the image according to the ground truth captions.”
§ Extrinsic Object Hallucination
“CHAIR 𝑖 measures per-instance object hallucination, i.e. what fraction of object instances in each generated caption are hallucinated.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 7.1 Hallucination Definition in Abstractive Summarization
“For instance, in Table 1, given the input article shown in the caption, an example of intrinsic hallucination is ' The Ebola vaccine was rejected by the FDA in 2019, '”
§ 2.1 Categorization
“in the abstractive summarization task from Table 1, the generated summary ' The first Ebola vaccine was approved in 2021 ' contradicts the source content”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Survey of Hallucination in Natural Language Generation
“The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine translation, and visual-language generation; and (3) hallucinations in large language models (LLMs)”
§ 1 INTRODUCTION
“The third part discusses this phenomenon in LLMs in Section 13.”
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“Recently, the rapid development of NLG has captured the imagination of many thanks to the advances in deep learning technologies, especially Transformer [189]-based models like BERT [29], BART [100], GPT-2 [149], and GPT-3 [16].”
BackgroundAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“This work investigates the performances of GPT-3 [13], GPT-Neo/J [141], GPT-2 [105] and a T5-based model [106].”
BackgroundAdvances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 8,890
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · cited 1×
“Recently, the development of NLG has conquered the imagination of many thanks to the advances in deep learning technologies, especially the Transformer [159]-based models like BERT [24], BART [83], GPT-2 [127], and GPT3 [13].”
BackgroundLewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816 · cited 1×
“Recently, the development of NLG has conquered the imagination of many thanks to the advances in deep learning technologies, especially the Transformer [159]-based models like BERT [24], BART [83], GPT-2 [127], and GPT3 [13].”
Background
Research works printed but not linked: 16
- [4] Joshua Albrecht and Rebecca Hwa. 2007. A Re-examination of Machine Learning Approaches for Sentence-Level MT Evaluation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics . Association for Computational Linguistics, Prague, Czech Republic, 880-887. https://aclanthology.org/P07-1111 -
- [16] Andrzej Białecki, Robert Muir, Grant Ingersoll, and Lucid Imagination. 2012. Apache lucene 4. In SIGIR 2012 workshop on open source information retrieval . 17. -
- [53] Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. 2023. Improving Factuality and Reasoning in Language Models through Multiagent Debate. In Forty-first International Conference on Machine Learning . -
- [121] Philipp Koehn and Rebecca Knowles. 2017. Six Challenges for Neural Machine Translation. In First Workshop on Neural Machine Translation . Association for Computational Linguistics, 28-39. -
- [145] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation. In ICML . -
- [149] Tian Li, Ahmad Beirami, Maziar Sanjabi, and Virginia Smith. 2020. Tilted Empirical Risk Minimization. In International Conference on Learning Representations . -
- [164] Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. 2018. Neural Baby Talk. In CVPR . -
- [165] Yukun Ma, Khanh Linh Nguyen, Frank Z Xing, and Erik Cambria. 2020. A Survey on Empathetic Dialogue Systems. Information Fusion 64 (2020), 50-70. -
- [175] Marianna J. Martindale, Marine Carpuat, Kevin Duh, and Paul McNamee. 2019. Identifying Fluently Inadequate Output in Neural and Statistical Machine Translation. In MTSummit . -
- [179] Kathleen McKeown. 1992. Text Generation . Cambridge University Press. -
- [209] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. OpenAI Blog 1, 8 (2019), 9. -
- [238] Hui Su, Xiaoyu Shen, Sanqiang Zhao, Zhou Xiao, Pengwei Hu, Cheng Niu, and Jie Zhou. 2020. Diversifying Dialogue Generation with Non-Conversational Text. In 58th Annual Meeting of the Association for Computational Linguistics . ACL, 7087-7097. -
- [256] Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li. 2017. Neural Machine Translation with Reconstruction. In Thirty-First AAAI Conference on Artificial Intelligence . -
- [261] Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. 2023. A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation. CoRR abs/2307.03987 (2023). https://doi.org/10.48550/arXiv.2307.03987 arXiv:2307.03987 - · arXiv · doi:10.48550/arxiv.2307.03987 · Paper page
- [285] Zeqiu Wu, Michel Galley, Chris Brockett, Yizhe Zhang, Xiang Gao, Chris Quirk, Rik Koncel-Kedziorski, Jianfeng Gao, Hannaneh Hajishirzi, Mari Ostendorf, et al. 2021. A Controllable Model of Grounded Response Generation. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 35. 14085-14093. -
- [291] Xinnuo Xu, Ondřej Dušek, Verena Rieser, and Ioannis Konstas. 2021. AGGGEN: Ordering and Aggregating while Generating. roceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL2021) (2021). -
Cited by
Items 2,501–2,600 of 4,844
Yu, J., Li, M., Qin, X., et al. (2026) · Journal of Robotic Surgery · doi:10.1007/s11701-026-03874-9 · Cited by 1
“Hallucination—confidently stated but factually incorrect information—remains a persistent challenge [16 ], and the transparency of AI-generated content, including disclosure of limitations and source attribution, is frequently inadequate [17].”
Cho, A.-Y., Eo, S., Jeon, M., Kim, J., Ihm, J., & Seo, D. (2026) · JMIR Medical Education · doi:10.2196/91809 · Cited by 2
“Second, all outputs should be verified against the original record before clinical use, given the potential for errors when documentation is incomplete or absent [12,13].”
Architectural Transferability in Bounded AI: Five Conditions for Regulated Decision Domains
Melville, G., Ghiassi, D., Inthathirath, S., & Yeomans, J. S. (2026) · AI · doi:10.3390/ai7090366 · Cited by 1
Chatbots and psychiatry: Beware of hallucinations of artificial intelligence
Vizoso, J. E. I., & Pastor, A. B. (2023) · Psiquiatría Biológica · doi:10.1016/j.psiq.2023.100394 · Cited by 1
Dialogue Rewriting via Skeleton-Guided Generation
Xin, C., Lin, H., Wu, S., Han, X., Chen, B., Dai, W., Chen, S., Wang, B., & Sun, L. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26619 · Cited by 2
“Unfortunately, they will occasionally generate text that is nonsensical, or unfaithful to the provided source input, which is referred as hallucination (Raunak, Menezes, and Junczys-Dowmunt 2021; Dziri et al. 2021; Ji et al. 2022).”
BackgroundFactual or Contextual? Disentangling Error Types in Entity Description Generation
Goyal, N., Nenkova, A., & Daumé, H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.463 · Cited by 7
“Language models often generate erroneous information, not supported by source and/or background documents (Ji et al., 2023), often termed as hallucinations.”
BackgroundUsing contradictions improves question answering systems
Fortier-Dubois, E., & Rosati, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.72 · Cited by 8
“Safety in NLP systems is unresolved, particularly in biomedical and scientific contexts where hallucination, overconfidence, and other problems are major obstacles to deployment (Ji et al., 2022; Kell et al., 2021).”
BackgroundA Pipeline for Generating, Annotating and Employing Synthetic Data for Real World Question Answering
Maufe, M., Ravenscroft, J., Procter, R., & Liakata, M. (2022) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2022.emnlp-demos.9 · Cited by 3
Genetic Generative Information Retrieval
Kulkarni, H., Young, Z., Goharian, N., Frieder, O., & MacAvaney, S. (2023) · ACM Symposium on Document Engineering (DocEng) · doi:10.1145/3573128.3609340 · Cited by 2
“Hallucination is a common phenomenon in deep learning based generative models [12].”
BackgroundTowards Effective Automatic Evaluation of Generated Reflections for Motivational Interviewing
Wu, Z., Helaoui, R., Recupero, D. R., & Riboni, D. (2023) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3610661.3616127 · Cited by 2
“This focus is motivated by the fact that 1) recent LMs struggle with coherence [5]; 2) coherence is the pre-requisite for being a “good refection” — a refection should be a sensible response before being assessed w.”
BackgroundReranking for a Polish Medical Search Engine
Pokrywka, J., Jassem, K., Wierzchoń, P., Badylak, P., & Kurzyp, G. (2023) · Annals of Computer Science and Information Systems · doi:10.15439/2023f1627 · Cited by 2
“This is mainly due to the phenomenon of artificial hallucination [10], [11], [12].”
BackgroundEnhancing Reliability and User Experience in Conversational Agents
Aunimo, L. (2023) · World Symposium on Digital Intelligence for Systems and Machines (DISA) · doi:10.1109/disa59116.2023.10308922 · Cited by 4
“One of the main challenges with CAs based on generative pretrained transformers (also called large language models) such as those used in ChatGPT by Open AI 1 is that they may produce erroneous answers (Ji et al., 2023) or safe answers with very little information content.”
BackgroundBeyond Quantity: Research with Subsymbolic AI
Sudmann, A., Echterhölter, A., Ramsauer, M., Retkowski, F., Schröter, J., & Waibel, A. (2023) · AI critique · doi:10.1515/9783839467664 · Cited by 14
Contrastive Learning for Inference in Dialogue
Ishii, E., Xu, Y., Wilie, B., Ji, Z., Lovenia, H., Chung, W., & Fung, P. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.631 · Cited by 1
Self-citationHallucination Detection for Grounded Instruction Generation
Zhao, L., Nguyen, K., & Daumé, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.266 · Cited by 9
The Scope of In-Context Learning for the Extraction of Medical Temporal Constraints
Seegmiller, P., Gatto, J., Basak, M., Cook, D. J., Ghasemzadeh, H. G., Stankovic, J. A., & Preum, S. M. (2023) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi57859.2023.00107 · Cited by 4
“A single human annotator then categorizes each model error, providing one possible reason for each failed MTC The most error type is hallucination , an error common in LLMs such as GPT-3 [14].”
BackgroundLow-Resource Efficient Multi-Stage Tuning Strategy for Biomedical Question Answering Task
Wang, B., Du, Y., Jin, X., Yan, R., & Zhang, Q. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385678 · Cited by 2
“However, the training data and training strategies used in commercialized Large Language Models (LLM) are not public (e.g., ChatGPT), and the unpublic training data may contain false information that makes the models raise "hallucinations"[2], producing harmful results.”
BackgroundContext-based Fact-checking Using Knowledge Graph
Lee, H. J., & Sohn, M. M. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386121 · Cited by 3
“Hallucination in the content generated by LLMs refers to the phenomenon in which the generative AIs produce contextually unrelated or factually incorrect information as if it was a correct answer [7].”
BackgroundiTOChat: a Mobile Bot Recommending Engineering Departments and Open Campus Events
Eguchi, N., Nakao, I., Saito, K., Juntao, W., Motomatsu, H., & Mine, T. (2023) · International Conference on Mobile Computing and Ubiquitous Network (ICMU) · doi:10.23919/icmu58504.2023.10412254 · Cited by 1
“An inherent characteristic of ChatGPT is its attempt to answer such queries, which leads to the occurrence of “Hallucination” (plausible misinformation) [6].”
BackgroundOn Computing Paradigms - Where Will Large Language Models Be Going
Wu, X., Zhu, X., Baralis, E., Lu, R., Kumar, V., Rutkowski, L., & Tang, J. (2023) · Proceedings · doi:10.1109/icdm58522.2023.00211 · Cited by 2
“Thus, the LLMs, when prompted with contemporary user queries for which they have not been trained, may hallucinate by fabricating outputs that are unanchored to factual reality [18].”
BackgroundElephant: LLM System for Accurate Recantations
Chinedu-Eneh, P., & Nguyen, T. T. (2024) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc60891.2024.10427759 · Cited by 1
الشريف, د. م. س., & الله, د. م. ن. ع. ع. (2024) · Maǧallaẗ al-'ulūm al-Qānūniyyaẗ wa-al-Iqtiṣādiyyaẗ/Maǧallaẗ al-'ulūm al-Qānūniyyaẗ wa-al-Iqtiṣādiyyaẗ · doi:10.21608/jelc.2024.342115 · Cited by 1
Duże modele językowe i możliwości ich wykorzystania w terroryzmie dżihadystycznym i przestępczości
Puczyńska, J., Podhajski, M., Wojtasik, K., & Michalak, T. (2024) · Terroryzm · doi:10.4467/27204383ter.24.004.19392 · Cited by 2
Halucynacje chatbotów a prawda: główne nurty debaty i ich interpretacje
Kreft, J., Boguszewicz-Kreft, M., & Cyrek, B. (2024) · Roczniki Nauk Społecznych · doi:10.18290/rns2024.0009 · Cited by 5
Generative AI as an educational resource
Strain, S., Watson, A., & Hale, M. (2024) · ASEE Southeast Section Conference · doi:10.18260/1-2--45532 · Cited by 1
Vers l’XAI sémantique – la troisième vague de l’intelligence artificielle explicable
Bollaert, M., & Coppin, G. (2024) · Ingénierie cognitique · doi:10.21494/iste.op.2024.1147 · Cited by 1
Author's reply: “Assessing ChatGPT and perplexity AI performance”
Pellegrino, R., & Gravina, A. G. (2024) · Digestive and Liver Disease · doi:10.1016/j.dld.2024.04.007 · Cited by 1
Die Kunst im Zeitalter der künstlichen Intelligenz: Walter Benjamins Kunsttheorie im 21. Jahrhundert
Stevanovic, D. (2024) · Digital humanities · doi:10.1515/9783839470039-005 · Cited by 1
Streamlining event extraction with a simplified annotation framework
Saetia, C., Thonglong, A., Amornchaiteera, T., Chalothorn, T., Taerungruang, S., & Buabthong, P. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1361483 · Cited by 9
“However, LLMs primarily favored well-resourced language with large updated training corpora, which may lead to hallucination problems, especially in lower-resource languages, in which the training corpora are not abundantly available (Ji et al., 2023).”
BackgroundSalley, C., Mohammadi, N., & Taylor, J. E. (2024) · Journal of Infrastructure Systems · doi:10.1061/jitse4.iseng-2407 · Cited by 1
Interactive and Explainable Robot Learning: A Comprehensive Review
Seraj, E., Lee, K. M., Zaidi, Z., Xiao, Q., Li, Z., Nascimento, A., van Waveren, S., Tambwekar, P., Paleja, R., Das, D., et al. (2024) · Foundations and Trends in Robotics · doi:10.1561/2300000081 · Cited by 7
Ojeda, L. A. M., Montero, S., & Mehta, A. (2024) · Annals of Plastic Surgery · doi:10.1097/sap.0000000000003935 · Cited by 2
Human-in-the-Loop AI Summaries of NEJM AI Grand Rounds
Manrai, A. K., & Beam, A. L. (2024) · NEJM AI · doi:10.1056/aie2400581 · Cited by 3
Argasiński, J. K., Grabska-Gradzińska, I., Przystalski, K., Ochab, J. K., & Walkowiak, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-63775-9_20 · Cited by 3
Kang, M.-S., & Jeon, D.-I. (2024) · Journal of Digital Contents Society · doi:10.9728/dcs.2024.25.6.1601 · Cited by 2
PHÁT TRIỂN TRỢ LÝ ẢO THÔNG MINH BẰNG MÔ HÌNH NGÔN NGỮ LỚN HỖ TRỢ GIẢNG DẠY
Oanh, T. N. O. T. N., Tuấn, B. C. T. B. C., Phương, N. V. P. N. V., Bảo, H. N. N. B. H. N. N., Long, N. S. T. L. N. S. T., Thắng, B. H. T. B. H., & Thơ, Q. T. (2024) · TẠP CHÍ KHOA HỌC TRƯỜNG ĐẠI HỌC QUỐC TẾ HỒNG BÀNG · doi:10.59294/hiujs.khqg.2024.001 · Cited by 2
“Tuy nhiên, những LLM này có thể chứa đựng "ảo giác" [6] do hạn chế trong việc tiếp cận thông tin mới nhất, chính xác hoặc thiếu kiến thức chuyên môn trong các lĩnh vực.”
Ambiguous Entity Oriented Targeted Document Detection
Shen, W., & Wen, H. (2024) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00072 · Cited by 2
“Additionally, to address the hallucination problem [6] of large language models (e.g., GPT-3 [7] and PaLM [8]), a series of recent research endeavors named retrieval augmented generation (RAG) [9]–[13] have attempted to retrieve external Web documents to augment large language models and help…”
BackgroundAssembling Fragmented Domain Knowledge: A LLM-Powered QA System for Taiwan Cinema
Kuo, E.-C., & Su, Y.-H. (2024) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec60901.2024.10612108 · Cited by 2
“…such as “ChatGPT” from OpenAI and “Gemini” from Google Inc., has shown that while these general-purpose bots possess broad knowledge, they may not always provide precise answers to specialized queries and are prone to occasional misinformation, a phenomenon known as AI hallucination [5] [6] [7].”
BackgroundHomeopathic Poisoning of RAG Systems
Addad, B., & Kapusta, K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-68738-9_28 · Cited by 4
Dutta, T., & Liu, X. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650810 · Cited by 3
“For Natural Language Generation (NLG), hallucination can be formally defined as the state of a generated text which is nonsensical or unfaithful to the provided source content (Ji et al. [2023]).”
BackgroundToFC: Tree-of-Fact with Continued Best-First Search for Commonsense Reasoning
Dai, L., & Qin, K. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10649921 · Cited by 1
“However, the output of the language model is disturbed by hallucinations and is not very reliable [25] [26].”
Background“No Free Lunch” when using Large Language Models to Verify Self-Generated Programs
Zilberman, S., & Cheng, H. C. B. (2024) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw60967.2024.00018 · Cited by 4
“Although LLMs produce seemingly coherent responses, recent work has revealed the presence of ‘ hallucinations ’ (i.e., responses that are nonsensical or unfaithful to the real world) in LLM-generated text [10].”
BackgroundWatanabe, K., Tanaka, S., Vargo, A. W., et al. (2024) · doi:10.1145/3675094.3677575 · Cited by 1
New “ArchAIval” Practices: Using GPT for OCR and Historical Narration of Index Cards
Ströbel, P., Clematide, S., Meyer, J., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72440-4_18 · Cited by 1
A Generative Benchmark Creation Framework for Detecting Common Data Table Versions
Fox, D. C., Khatiwada, A., & Shraga, R. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679157 · Cited by 1
Zhan, B., Duan, Y., & Yan, S. (2024) · International Conference on Computational Intelligence and Applications (ICCIA) · doi:10.1109/iccia62557.2024.10719061 · Cited by 2
“This phenomenon presents a considerable challenge to the reliability of LLMs in real-world applications[6, 7].”
BackgroundZhuang, X., Mao, X. H., Jiang, Y.-H., Wu, H., Zhao, S., Cai, L., Liu, S., Chen, Y., Song, Y., Jia, C., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-9434-8_18 · Cited by 4
Solving Mystery Planning Problems Using Category Theory, Functors, and Large Language Models
Jha, S. K., Jha, S., Ewetz, R., & Velasquez, A. (2024) · IEEE International Conference on Assured Autonomy (ICAA) · doi:10.1109/icaa64256.2024.00028 · Cited by 1
“A significant challenge in the deployment of LLMs is their propensity to produce hallucinations, which are outputs that are factually incorrect or nonsensical [7].”
Divekar, R. R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-76809-5_4 · Cited by 2
Alucinaciones de la inteligencia artificial: impacto en tecnología, política y sociedad
Huidobro, C. B. (2024) · Revista Estrategia Poder y Desarrollo · doi:10.25062/2955-0289.4847 · Cited by 3
SLLIM-Rank: A Multi-Stage Item-to-Item Recommendation Model using Learning-to-Rank
Ahmadi, K., Gathwala, A., Osajima, J. S., et al. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10826085 · Cited by 1
“Quality checks are performed at each feature generation step to ensure GPT output quality and prevent hallucinations [21].”
Beyond the Bar: Generative AI as a Transformative Component in Legal Document Review
Yang, E., Omrani, R., Curtin, E., Emory, T. S., Gray, L., Pickens, J., Reff, N., Traylor, C., Underwood, S., Lewis, D. D., et al. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10826089 · Cited by 7
Towards Blockchain-based AI Alignment Using a Proof of Personhood Consensus Mechanism
Neulinger, A., & Sparer, L. (2024) · International Conference on Blockchain Computing and Applications (BCCA) · doi:10.1109/bcca62388.2024.10844393 · Cited by 1
“Classifying, filtering and limiting the noise in ML model input data by correctly annotating it reduces the AI safety risks [62].”
InfluentialResponses of Artificial Intelligence Chatbots to Testosterone Replacement Therapy: Patients Beware!
Pabla, H., Lange, A., Nadiminty, N., & Sindhwani, P. (2025) · Société Internationale d’Urologie Journal · doi:10.3390/siuj6010013 · Cited by 1
“Along with the advancements, it has been discovered that these AI models often generate false or misleading information and present it as a fact, a phenomenon popularly known as the “hallucination effect” [8,9].”
Hybrid Feature Fusion for Enhancing Medical Document Embedding
Zhu, Y., Liu, X., Pang, T., Xiao, X., Zhang, X., & Fan, C. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888956 · Cited by 1
“Nevertheless, these models occasionally generate inaccurate or entirely fabricated information, a phenomenon known as “hallucination” [2].”
KARLM: Enhancing LLM-based Recommendation Systems with Knowledge Bases
Song, Z., Chen, D., Shen, X., Zhou, X., Qi, J., & Zhou, Y. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888867 · Cited by 1
“LLMs usually encounter hallucinations, leading to the generation of nonsensical or unfaithful content compared to the provided source content [17].”
Wissigkeit, T., Lope-Nicholls, D., Stindl, C., & Iurgel, I. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-85663-1_9 · Cited by 1
Keasler, P. M., Chan, J. C. Y., & Sng, B. L. (2025) · International Journal of Obstetric Anesthesia · doi:10.1016/j.ijoa.2025.104353 · Cited by 2
Efficient and Deployable Knowledge Infusion for Open-World Recommendations via Large Language Models
Xi, Y., Liu, W., Lin, J., Weng, M., Cai, X., Zhu, H. C., Zhu, J., Chen, B., Tang, R., Yu, Y. H., et al. (2025) · ACM Transactions on Recommender Systems · doi:10.1145/3725894 · Cited by 14
Zhou, Y., Wei, J., Sun, Y., & Du, W. (2025) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00004-7 · Cited by 5
“Such hallucinations can be particularly detrimental when summarizing scientific and biomedical literature, where precision and reliability are paramount (Ji et al. 2023).”
Wee, P., & Baghdadi, R. (2024) · IEEE/ACM International Conference on Big Data Computing, Applications and Technologies (BDCAT) · doi:10.1109/bdcat63179.2024.00048 · Cited by 3
“With respect to models, Ji et al. [18] point out that the training and inference of models are known to contribute to hallucination through a variety of factors, including Imperfect Representation Learning, Erroneous Decoding, Exposure Bias and Parametric Knowledge Bias.”
InfluentialTowards a Probabilistic Framework for Analyzing and Improving LLM-Enabled Software
Baldonado, J. M., Bonomo, F., & Braberman, V. A. (2025) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw64639.2025.10962470 · Cited by 2
“Particularly, phenomena such as hallucination [8] impact what can be guaranteed about such applications.”
Klüwer, N., Nalis, I., & Neidhardt, J. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3721205 · Cited by 2
Principles for the responsible application of Generative AI
Clarke, R. (2025) · Computer law & security review · doi:10.1016/j.clsr.2025.106131 · Cited by 10
Ward, R., Dalal, D., Buitelaar, P., & Breslin, J. G. (2025) · ACM Symposium on Applied Computing · doi:10.1145/3672608.3707748 · Cited by 2
Causation analysis of crane-related accident reports by utilizing ChatGPT and complex networks
Wang, Y., Chen, J., Xiao, B., Mueller, S. T., & Guo, J. (2025) · Journal of Building Design and Environment · doi:10.70401/jbde.2025.0009 · Cited by 3
Olivier, T. J., Ma, Z., Patel, A. A., Shi, W., Murtuza, M., Hatchard, N., Pan, X. N., & Annaswamy, T. M. (2025) · Interventional Pain Medicine · doi:10.1016/j.inpm.2025.100592 · Cited by 4
“These models can sometimes generate inaccurate or non-specific responses [19 – 21], may lack the nuance required for complex clinical decision-making [19,22], and are influenced by biases present in their training data [23,24].”
"I apologize for my actions": Emergent Properties and Technical Challenges of Generative Agents
Diamond, N., & Banerjee, S. (2025) · doi:10.1109/alife-ciscompanion65078.2025.11031053 · Cited by 1
“This behaviour is likely highly dependent on the structure, training, and alignment of the utilised language model (Ji et al. 2023; Ye et al. 2023).”
Palma, D. D., Biancofiore, G. M., Anelli, V. W., Narducci, F., & Noia, T. D. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3708319.3733680 · Cited by 4
Trustworthiness Evaluation of Large Language Models
Chen, P., & Liu, S. (2025) · Springer eBooks · doi:10.1007/978-3-031-76770-8_12 · Cited by 2
Beyond the Turing Test: A Bergsonian Exploration of Proto-Consciousness in Large Language Models
Thomas, A. (2025) · Journal of Artificial Intelligence and Consciousness · doi:10.1142/s2705078524500097 · Cited by 1
Shi, Z., Chung, D., Du, Y., Zhang, J., Raina, S., & Matarić, M. J. (2025) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3713043.3728841 · Cited by 9
Liu, W., Deng, Z., Niu, Z., Wang, J., Wang, H., & Li, R. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00758 · Cited by 6
RARR Unraveled: Component-Level Insights into Hallucination Detection and Mitigation
Ross, J. J., Khramtsova, E., van der Vegt, A. H., Koopman, B., & Zuccon, G. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730337 · Cited by 5
Katwe, P. K., Balabantaray, R. C., & Vittala, K. P. (2024) · Annual Meeting of the Forum for Information Retrieval Evaluation (FIRE) · doi:10.1145/3734947.3734958 · Cited by 2
Enhanced Semantic Retrieval with Structured Prompt and Dimensionality Reduction for Big Data
Kim, D., Park, M., Lee, J. S., Lee, I., Jin, J., & Sung, Y. (2025) · Mathematics · doi:10.3390/math13152469 · Cited by 4
Sposato, M. (2025) · Leadership & Organization Development Journal · doi:10.1108/lodj-02-2025-0104 · Cited by 2
Linearly-interpretable concept embedding models for text analysis
Santis, F. D., Bich, P., Ciravegna, G., Barbiero, P., Giordano, D., & Cerquitelli, T. (2025) · Machine Learning · doi:10.1007/s10994-025-06839-5 · Cited by 5
“Recent studies have highlighted the unreliability of LLMs, as they often occur hallucinations [19], and when prompted for explanations, their responses frequently do not reflect the actual decision-making process [4, 5, 20].”
Challenges of Generative AI on Human–AI Interaction and Collaboration
Liang, S., & Cai, Y. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009587877.012 · Cited by 2
Budding, T. (2025) · Public Money & Management · doi:10.1080/09540962.2025.2560477 · Cited by 2
Zappavigna, M. (2025) · Discourse Studies · doi:10.1177/14614456251366432 · Cited by 4
“1 While early work focused on technical performance and factual accuracy (Ji et al., 2023), more recent studies explore how chatbots co-construct interpersonal meaning with users.”
MESH - Understanding Videos Like Human: Measuring Hallucinations in Large Video Models
Yang, G., Chen, Z., Wong, M. H., Lei, H., Chen, Y., Li, Z., Zhou, K., & Cheng, J. S.-C. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755626 · Cited by 6
Modular RAG-based Legal Question Answering System Reflecting the Hierarchical Structure of Civil Law
Shin, E., So, H., Yoon, Y., Jung, C., Kim, D., & Jung, H. (2025) · The Journal of Korean Institute of Information Technology · doi:10.14801/jkiit.2025.23.11.27 · Cited by 1
The Potential Applications and Implications of Large Language Models in the Medical Field
Sim, M. S., & Hong, S. W. (2025) · Gyemyeong uidae haksulji · doi:10.46308/kmj.2025.00178 · Cited by 1
“Such phenomena have been widely reported as critical barriers to the adoption of LLMs in medicine [18].”
Vision-Language Models for Automated Property Feature Extraction in Tax Assessment: A Case Study
Ribeiro, G. R., Saraiva, P. A. M., Rosa, L. H. A., Marques, E. L., Pereira, G. L. B., Campos, P. M. L., Pascoal, L. M. L., & de Oliveira, S. S. T. (2025) · doi:10.5753/eniac.2025.13998 · Cited by 1
“A higher temperature could cause the model to ”hallucinate” a feature that is not present or to deviate from the provided list of options, thereby resulting in an incorrect classification [Ji et al. 2023].”
Parameterized Argumentation-based Reasoning Tasks for Benchmarking Generative Language Models
Steging, C., Renooij, S., & Verheij, B. (2025) · International Conference on Artificial Intelligence and Law · doi:10.1145/3769126.3769230 · Cited by 5
Goldstein, O., Haj, N. M., & Zidan, W. (2026) · AI in Education · doi:10.3390/aieduc2030029 · Cited by 1
“Regarding the use of AI tools specifically, the researchers acknowledge known challenges such as limited transparency, potential bias, hallucinations, and uneven data attention (Bender et al., 2021; Ji et al., 2023; Kasneci et al., 2023).”
Towards In-Context Non-Expert Evaluation of Reflection Generation for Counselling Conversations
Wu, Z., Balloccu, S., Helaoui, R., Recupero, D. R., & Riboni, D. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.9 · Cited by 4
“Ishii et al. (2022) define a response to be “intrinsic hallucination” if it contradicts the input(e.g., Dziri et al., 2019; Li et al., 2020a) and “ex-trinsic hallucination” if it cannot be verified based on the input (e.g., Mielke et al., 2022; Roller et al., 2021).”
BackgroundControllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model
Daheim, N., Thulke, D., Dugast, C., & Ney, H. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.98 · Cited by 6
Coşkun, B., Ocakoğlu, G., Yetemen, M., et al. (2023) · Urology · doi:10.1016/j.urology.2023.05.042 · Cited by 1
Beyond the Bias: Unveiling the Quality of Implicit Causality Prompt Continuations in Language Models
Sieker, J., Bott, O., Solstad, T., & Zarrieß, S. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.15 · Cited by 4
“the decision rule used to determine the output sequence of a model, is one of the most important factors that affects the quality and various linguistic properties of the generated text, as several papers have demonstrated (e.g. Holtzman et al., 2019; Zarrieß et al., 2021; Meister et al., 2022; Ji et al., 2023).”
BackgroundMethodChatGPT and Large Language Models (LLMs) in Healthcare: Opportunities and RisksPreprint
Ali, H., Qadir, J., Shah, Z., Alam, T., & Househ, M. (2023) · doi:10.36227/techrxiv.22579852.v1 · Cited by 2
GPT-3.5 Hallucinates Nonexistent Citations: Evidence from EconomicsPreprint
Buchanan, J. A., & Shapoval, O. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4467968 · Cited by 2
ChatGPT and Large Language Models (LLMs) in Healthcare: Opportunities and RisksPreprint
Ali, H., Qadir, J., Shah, Z., Alam, T., & Househ, M. (2023) · doi:10.36227/techrxiv.22579852.v3 · Cited by 1
Hallucinating Machines: Exploring the ethical implications of generative language models
Hill, M. (2023) · Open Access Victoria University of Wellington | Te Herenga Waka (Figshare) · doi:10.26686/wgtn.24180456 · Cited by 7
Stress-Induced Failure of Embodied CognitionPreprint
Wallace, R., & Fricchione, G. L. (2023) · Research Square · doi:10.21203/rs.3.rs-3527414/v1 · Cited by 1
RansomCillin: Leveraging NTFS Spare Space to Recover from Ransomware AttacksPreprint
Takeuchi, K., Fujima, H., Kumamoto, T., & Yoshida, Y. (2023) · doi:10.36227/techrxiv.24596754 · Cited by 1
Welfare Implications of Democratization in Content Creation: Generative AI and BeyondPreprint
Zou, T., Shi, Z., & Wu, Y. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4662282 · Cited by 1
Metze, K., Lorand-Metze, I. G. H., Morandin, R. C., & Florindo, J. B. (2024) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2023.10.026 · Cited by 1
Can ChatGPT Generate Stock Tickers to Buy and Sell for Day Trading?Preprint
Cho, S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4759311 · Cited by 1
Metadata
- Authors
- Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P.
- Affiliations
- Hong Kong University of Science and Technology (HK)
- ORCID
- Venue
- ACM Computing Surveys
- Published
- 2023-03-03
- Volume
- 55
- Issue
- 12
- Pages
- 1-38
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4309674289
- Semantic Scholar
- 3def68bd0f856886d34272840a7f81588f2bc082
- Cited by
- 5,025
- References
- 111
- Citations by year
- 2026: 2296 · 2025: 1441 · 2024: 817 · 2023: 453
- FWCI
- 224.36
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Computing Surveys (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- hallucinations, natural language generation, hallucination mitigation, abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine translation, large language models, transformer models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.




