The DOI matches its Crossref record.
Fixing Reference Hallucinations of LLMs
Open accessAbstract
In October and November 2024, using popular LLMs like OpenAI ChatGPT (4 and below), Azure OpenAI and its Copilot instantiations, Google Gemini and GenAI LLM tuned for scientific papers like Zendy, asking a question and references produces with every LLM fake references, well constructed, but with different titles or authors than the web or journal reference actually associated to the citation, or sometimes totally invented. Prompting to ensure that the reference exists and is correct may help for some, but in general it does not. Others have reported similar issues when using these LLM/GenAI services to produce legal briefs, and other legal documents.This paper suggest simple ways to address this, instead of trying to just improve the LLMs and hope hallucinations will be reduced; they won’t, no matter what, they are inherent to LLM. It is very surprising and mindboggling that LLM providers have not been implementing these kind of solutions: just check if the references exist, are correctly cited, and relevant to the paper/context. We also expand the approach with our MultiAI approach to improve on the previous approach, or address other hallucinations; actually eliminating in our tests.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“these models do confabulate references (Maes, 2024).”
From Adding a simple log to in-text citations to strengthen information literacy and argumentation while blocking casual misuse of generative AI (2026) · cites this paper as Maes, 2024
The cited source says what the claim attributes to it.
In the citing paper
“Bibliography items may not point to a record that exists.”
From Adding a simple log to in-text citations to strengthen information literacy and argumentation while blocking casual misuse of generative AI (2026) · cites this paper as Maes, 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 21
- Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al., (2023), "A survey of large language models", arXiv:2303.18223v15. · arXiv
- Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen, (2024), "The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models", arXiv:2401.03205v1. · arXiv
- Ayush Agrawal, Mirac Suzgun, Lester Mackey, Adam Tauman Kala, (2023-2024), "Do Language Models Know When They're Hallucinating References?", arXiv:2305.18248v3. · arXiv
- Abe Bohan Hou, William Jurayj, Nils Holzenberger, Andrew Blair-Stanek, Benjamin Van Durme, (2024), "Gaps or Hallucinations? Gazing into Machine-Generated Legal Analysis for Fine-grained Text Evaluations", arXiv:2409.09947v2. · arXiv
- A A Zhukov, I E Batov, (2023), "Regimes of electronic transport in doped InAs nanowire", arXiv:2305.13983v1. · arXiv
- Ayush Agrawal, Mirac Suzgun, Lester Mackey, Adam Tauman Kalai, (2023-2024), "Do Language Models Know When They're Hallucinating References?", arXiv:2305.18248v3. · arXiv
- Sourav Banerjee, Ayushi Agarwal, Saloni Singla, (2024), "LLMs Will Always Hallucinate, and We Need to Live With This", arXiv:2409.05746v1. · arXiv
- Xuhui Jiang, Yuxing Tian, Fengrui Hua, Chengjin Xu, Yuanzhuo Wang, Jian Guo, (2024), "A Survey on Large Language Model Hallucination via a Creativity Perspective", arXiv:2402.06647v1. · arXiv
- Muru Zhang, Ofir Press, William Merrill, Alisa Liu, Noah A. Smith, (2023), "How Language Model Hallucinations Can Snowball", arXiv:2305.13534v1. · arXiv
- Amos Azaria, Tom Mitchell, (2023), "The Internal State of an LLM Knows When It's Lying", arXiv:2304.13734v2. · arXiv
- Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, Haofen Wang, (2023-2024), "Retrieval-Augmented Generation for Large Language Models: A Survey", arXiv:2312.10997v5. · arXiv
- Noah Gift, Alfredo Deza, (2021), "Practical MLOps: Operationalizing Machine Learning Models", O'Reilly Media.
- Raschka, Sebastian, (2024), "Machine Learning and AI Beyond the Basics", No Starch Press.
- Jeremy Watt, Reza Borhani, Aggelos Katsaggelos, (2020), "Machine Learning Refined: Foundations, Algorithms, and Applications", Cambridge University Press.
- Stephane H Maes, (2023), "A GUIDE TO Building Your Own Service Desk Virtual Agent and simulating human conversations", https://www.researchgate.net/publication/370100099_Building_Your_Own_Service_Desk_Virtual_Agent_and_si mulating_human_conversations_A_Guide_to_Building_Your_Own_Service_Desk_Virtual_Agent_and_Simulating_ Human_Conversations, April 18, 2023.
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, (2022-2023), "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models", arXiv:2201.11903v6. · arXiv
- Lijie Hu, Liang Liu, Shu Yang, Xin Chen, Zhen Tan, Muhammad Asif Ali, Mengdi Li, Di Wang, (2024), "Understanding Reasoning in Chain-of-Thought from the Hopfieldian View", arXiv:2410.03595v1. · arXiv
- Zhen Huang, Haoyang Zou, Xuefeng Li, Yixiu Liu, Yuxiang Zheng, Ethan Chern, Shijie Xia, Yiwei Qin, Weizhe Yuan, Pengfei Liu, (2024), "O1 Replication Journey --Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson?", arXiv:2411.16489v1. · arXiv
- Stephane H. Maes, (2000), "Elements of conversational computing-A paradigm shift", International Conference on Spoken Language.
- A Tiwari, RA Hosn, SH Maes, (2003), "Conversational multi-modal browser: an integrated multi-modal browser and dialog manager", 2003 Symposium on Applications and the Internet. Proceedings, 348-351.
- Stephane H. Maes (1999), "Conversational biometrics", The European Conference on Speech Communication and Technology.
Cited by
Sorted by citations
Bibliographic Metadata Extraction Using Deterministic Retrieval and Specialized AI Agents
Rasheva-Yordanova, K., Hadzhiyska, Y., Ivanov, I., Kostadinova, I., Lozev, L., & Lozeva, A. (2026) · Information · doi:10.3390/info17090841
Tamas, P., & Kamminga, L. (2026) · Journal of Information Literacy · doi:10.11645/20.1.847
Metadata
- Authors
- Maes, S.
- Corresponding author
- Maes, Stephane
- ORCID
- Maes, Stephane
- Publisher
- Center for Open Science
- Published
- 2024-12-26
- OpenAlex
- W4405801703
- Cited by
- 3
- References
- 0
- Citations by year
- 2026: 2 · 2025: 1
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- OSF Preprints (OSF Preprints) (submittedVersion, cc-by)
- Zenodo (CERN European Organization for Nuclear Research) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- large language models, citation verification, legal document generation, prompt engineering, AI hallucination mitigation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
