Evaluating the Integrity of LLM-Generated Citations: Prevalence and Risks of Fabricated References in Scientific Literature
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Large Language Models have become important in our lives, and academia is not agnostic to this trend, offering tools like text rephrasing and summarisation. However, this integration raises significant concerns regarding the integrity of science. In this paper, we investigate hallucinations of LLMs when generating scientific references. Using nine LLMs, we generated a dataset of 74,196 BIBTEX references to quantify and analyse fabricated references, focusing on distinguishing between intrinsic and extrinsic hallucinations. Also, we extracted and analysed 127,063 references from 3541 published papers in 2023 to assess the prevalence of fake bibliographic data. Our manual verification process identified eight instances of fabricated references. While the overall rate is statistically low, the mere existence of fabricated content in the peer-reviewed literature is a critical integrity issue, demonstrating a vulnerability in current academic validation systems. The significance of our finding is not the statistical prevalence but rather the necessity for rigorous, human-validated processes to prevent the injection of spurious citations regardless of their source.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 4,688
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013
Artificial hallucinations in ChatGPT: implications in scientific writing
Alkaissi, H., & McFarlane, S. I. (2023) · Cureus · doi:10.7759/cureus.35179 · Cited by 1,540 · cited 3×
“When applied to scientific bibliography, some recent studies showed that none of the automatically generated scientific references exists when prompted to generate a scientific text with references that support that text; i.e., all the references were fabricated [12–14].”
On Faithfulness and Factuality in Abstractive Summarization
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.173 · Cited by 1,185 · cited 1×
“These are typically categorised into two types: intrinsic and extrinsic [11].”
On Faithfulness and Factuality in Abstractive Summarization
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020) · arXiv · doi:10.48550/arxiv.2005.00661 · Cited by 1,185 · cited 1×
“These are typically categorised into two types: intrinsic and extrinsic [11].”
Perkins, M. (2023) · Journal of University Teaching and Learning Practice · doi:10.53761/1.20.02.07 · Cited by 839
Fabrication and errors in the bibliographic citations generated by ChatGPT
Walters, W. H., & Wilder, E. I. (2023) · Scientific Reports · doi:10.1038/s41598-023-41032-5 · Cited by 499 · cited 2×
“When applied to scientific bibliography, some recent studies showed that none of the automatically generated scientific references exists when prompted to generate a scientific text with references that support that text; i.e., all the references were fabricated [12–14].”
Athaluri, S. A., Manthena, S. V., Kesapragada, V. S. R. K. M., Yarlagadda, V., Dave, T., & Duddumpudi, R. T. S. (2023) · Cureus · doi:10.7759/cureus.37432 · Cited by 403
High Rates of Fabricated and Inaccurate References in ChatGPT-Generated Medical Content
Bhattacharyya, M., Miller, V. M., Bhattacharyya, D., & Miller, L. E. (2023) · Cureus · doi:10.7759/cureus.39238 · Cited by 294
Ethics of large language models in medicine and medical research
Li, H., Moon, J. T., Purkayastha, S., Celi, L. A., Trivedi, H., & Gichoya, J. W. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00083-3 · Cited by 286 · cited 1×
“Nowadays, Internet users use LLMs for rephrasing text and adapting it to different contexts [1], automatically translating text into multiple languages [2], and summarising long and complex texts to be easily understandable [3], among many other examples.”
Trust in scientists and their role in society across 68 countries
Cologna, V., Mede, N. G., Berger, S., Besley, J., Brick, C., Joubert, M., Maibach, E. W., Mihelj, S., Oreskes, N., Schäfer, M. S., et al. (2025) · Nature Human Behaviour · doi:10.1038/s41562-024-02090-5 · Cited by 251 · cited 2×
“Trust in science is crucial, and undermining it could cause people to stop trusting research; even more distressing is the fact that researchers themselves may stop trusting each other as well [8].”
Using ChatGPT in academic writing is (not) a form of plagiarism: What does the literature say?
Jarrah, A. M., Wardat, Y., & Fidalgo, P. (2023) · Online Journal of Communication and Media Technologies · doi:10.30935/ojcmt/13572 · Cited by 220
Aljamaan, F., Temsah, M.-H., Altamimi, I., Al-Eyadhy, A., Jamal, A., Alhasan, K., Mesallam, T. A., Farahat, M., & Malki, K. H. (2024) · JMIR Medical Informatics · doi:10.2196/54345 · Cited by 211
The Use of Artificial Intelligence in Writing Scientific Review Articles
Kacena, M. A., Plotkin, L. I., & Fehrenbacher, J. C. (2024) · Current Osteoporosis Reports · doi:10.1007/s11914-023-00852-0 · Cited by 193 · cited 1×
“With the incipient irruption of NLP and more concretely with automatic text generation, there is some controversy in the literature regarding the use of tools like ChatGPT or Gemini in scientific writing [4,5,12,25–28].”
Artificial intelligence in scientific writing: a friend or a foe?
Altmäe, S., Sola-Leyva, A., & Salumets, A. (2023) · Reproductive BioMedicine Online · doi:10.1016/j.rbmo.2023.04.009 · Cited by 174 · cited 2×
“While LLMs might undoubtedly be a powerful tool when correctly used in academia [4–6], if misused, the scientific integrity and trust of the society in research might be affected negatively [7,8].”
Web crawling ethics revisited: Cost, privacy, and denial of service
Thelwall, M. A., & Stuart, E. (2006) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20388 · Cited by 110 · cited 1×
“To ensure ethical compliance and minimise impact on platform performance, our semi-automatic process strictly adheres to a constrained request rate (e.g., approximately one request per second) [22–24].”
History, development, and principles of large language models: an introductory survey
Wang, Z., Chu, Z., Doan, T. V., Ni, S., Yang, M., & Zhang, W. (2025) · AI and Ethics · doi:10.1007/s43681-024-00583-7 · Cited by 109 · cited 1×
“In the 1990s, the existing NLP models were based on neural networks [15].”
Kendall, G., & Teixeira da Silva, J. A. (2024) · Learned Publishing · doi:10.1002/leap.1578 · Cited by 100 · cited 1×
“With the incipient irruption of NLP and more concretely with automatic text generation, there is some controversy in the literature regarding the use of tools like ChatGPT or Gemini in scientific writing [4,5,12,25–28].”
Mugaanyi, J., Cai, L., Cheng, S., Lu, C., & Huang, J. (2024) · Journal of Medical Internet Research · doi:10.2196/52935 · Cited by 74 · cited 2×
“For example, some authors automatically generated 102 references and concluded that over 75% of the generated DOI exist [30].”
Balancing innovation and integrity: the role of AI in research and scientific writing
BaHammam, A. (2023) · Nature and Science of Sleep · doi:10.2147/nss.s455765 · Cited by 50 · cited 1×
“With the incipient irruption of NLP and more concretely with automatic text generation, there is some controversy in the literature regarding the use of tools like ChatGPT or Gemini in scientific writing [4,5,12,25–28].”
Analysing the impact of ChatGPT in research
Picazo-Sanchez, P., & Ortiz-Martin, L. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05298-0 · Cited by 46 · cited 3×
“Only after the non-existence was confirmed bibliographically did we randomly select a few paragraphs from the paper and checked them using some of the https://doi.org/10.3390/data11050122 most well-known AI content detectors available, which is similar to previous research in the field [9].”
InfluentialSelf-citationAdapting LLMs for Efficient Context Processing through Soft Prompt Compression
Wang, C., Yang, Y., Li, R., Sun, D., Cai, R., Zhang, Y., & Fu, C. (2024) · Proceedings of the International Conference on Modeling, Natural Language Processing and Machine Learning · doi:10.1145/3677779.3677794 · Cited by 45 · cited 1×
“Nowadays, Internet users use LLMs for rephrasing text and adapting it to different contexts [1], automatically translating text into multiple languages [2], and summarising long and complex texts to be easily understandable [3], among many other examples.”
Prompt Consistency for Zero-Shot Task Generalization
Zhou, C., He, J., Ma, X., Berg-Kirkpatrick, T., & Neubig, G. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.192 · Cited by 41
Brewer, R., Westlake, B. G., Hart, T., & Arauza, O. (2021) · Springer eBooks · doi:10.1007/978-3-030-74837-1_22 · Cited by 37
How accurate are the references generated by ChatGPT in internal medicine?
Sebo, P. (2024) · Internal and Emergency Medicine · doi:10.1007/s11739-023-03484-5 · Cited by 11 · cited 1×
“While bibliographic hallucinations have drawn the attention of researchers in the last few years, most previous work has involved manually checking these fabricated references on a small scale, e.g., 5 [12], 10 [29], 102 [30], 115 [31], 178 [26], and 636 [14] references.”
Bik, E. M. (2024) · Nature Reviews Cancer · doi:10.1038/s41568-024-00702-w · Cited by 11 · cited 1×
“While LLMs might undoubtedly be a powerful tool when correctly used in academia [4–6], if misused, the scientific integrity and trust of the society in research might be affected negatively [7,8].”
Stringhini, G., & Blackburn, J. (2026) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw72489.2026.00037 · Cited by 2
Research works printed but not linked: 4
- Kocmi, T., and Federmann, C. (2023, January 12–15). Large Language Models Are State-of-the-Art Evaluators of Translation Quality. Proceedings of the 24th Annual Conference of the European Association for Machine Translation, Tampere, Finland.
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the 2017 Conference of Advances in Neural Information Processing Systems, Long Beach, CA, USA.
- Huang, W., Liu, Y., Qin, H., Li, Y., Zhang, S., Liu, X., Magno, M., and QI, X. (2024, January 21–27). BiLLM: Pushing the Limit of Post-Training Quantization for LLMs. Proceedings of the International Conference on Machine Learning, Vienna, Austria.
- Elsevier (2026, March 26). The Use of Generative AI and AI-Assisted Technologies in Writing for Elsevier. Available online: https://www.elsevier.com/about/policies-and-standards/the-use-of-generative-ai-and-ai-assisted-technologies-in-writing-for-elsevier.
Cited by
ONE-CLICK AWAY LEGAL PRACTICE: A Perspective on Modern Legal Practice with AI in the Philippines
Acas, R. T., & Alejo-Abitago, J. (2026) · International Journal of Digital Law and Governance · doi:10.1515/ijdlg-2026-0006 · Cited by 1
“Although fabricated citations become rare in published research, they are considered a major breach of integrity, as current validation systems do not consider them to be a problem (Picazo-Sanchez and Ortiz-Martin 2026).”
Metadata
- Authors
- Picazo-Sanchez, P., & Ortiz-Martin, L.
- Affiliations
- School of Information Technology, Halmstad University, 301 18 Halmstad, Sweden
- Venue
- Data
- Publisher
- MDPI AG
- Published
- 2026-05-20
- Volume
- 11
- Issue
- 5
- Pages
- 122
- ISSN
- 2306-5729 (ISSN-L)
- OpenAlex
- W7161820716
- Semantic Scholar
- 04157dd41c69aafdb48d54cbf8151247673bbd8a
- Cited by
- 1
- References
- 20
- Citations by year
- 2026: 1
- FWCI
- 5.77
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Data (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- LLM hallucination, citation integrity, large language models, peer review, scientific integrity, citation verification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.