The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
A Preliminary Investigation of Fake Peer-Reviewed Citations and References Generated by ChatGPT
Abstract
An analysis of academic citations and references generated by the ChatGPT artificial intelligence (AI) chatbot reveals the citations and references are in fact, fake. They are clearly generated by a predictive process rather than known facts. This suggests that early optimism regarding this technology for assisting in research could be misplaced, and that student misuse of the chatbot can be detected by the identification of fake citations and references. Despite these problems, the technology could have application in the writing of course materials for lower level undergraduate courses that do not necessarily require references. Subject matter expertise is required, however, to identify and remove incorrect information. The need to identify incorrect information provided by an AI chatbot is a skill that students will also increasingly need.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Fabricated citations work against each of those goals, especially since (a) ChatGPT's fabricated citations tend to look legitimate at first glance13,28 and (b) ChatGPT often provides incorrect responses when asked “Is this citation correct” or “Do you fabricate citations?”12,13,18,24,29.”
From Fabrication and errors in the bibliographic citations generated by ChatGPT (2023) · cites this paper as [28]
In the citing paper
“ChatGPT's fabricated citations tend to look legitimate at first glance13,28”
From Fabrication and errors in the bibliographic citations generated by ChatGPT (2023) · cites this paper as [28]
In the citing paper
“Bhattacharyya et al.27 found that 87% of the citations to real (non-fabricated) works had 1 or more of 7 errors—incorrect PubMedID number, author name, article title, date, journal title, volume number, or page numbers—and Day28 reported very similar results.”
From Fabrication and errors in the bibliographic citations generated by ChatGPT (2023) · cites this paper as [28]
In the citing paper
“6 studies have systematically investigated the extent of the problem13,15,26–29.”
From Fabrication and errors in the bibliographic citations generated by ChatGPT (2023) · cites this paper as [28]
In the citing paper
“As Day (2023) argues, the ability to identify hallucinated sources is increasingly becoming a core academic skill, as students must now contend not just with evaluating peer-reviewed research but also verifying AI-generated information.”
From Training Students to Identify and Correct Fabricated References in ChatGPT-Generated Literature Reviews (2025) · cites this paper as Day (2023)
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115
Kung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
ChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842 · cited 2×
“It has quickly become a cultural sensation (Thorp 2023) since it trial launched in November 2022, with, “ far-reaching consequences for science and society ” (van Dis et al. 2023), and with “ remarkable performance ” on medical licensing exams (Kung et al. 2023).”
BackgroundChatGPT: Bullshit spewer or the end of traditional assessments in higher education?
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.9 · Cited by 1,806 · cited 1×
“Concerns about the impact of the technology on academic research, and on student academic integrity also quickly emerged, however (Rudolph, Tan, and Tan 2023).”
BackgroundChatGPT is fun, but not an author
Thorp, H. H. (2023) · Science · doi:10.1126/science.adg7879 · Cited by 1,437
Wen, Q., Li, J. H., Ding, J., & Wang, J. (2022) · Land Use Policy · doi:10.1016/j.landusepol.2022.106506 · Cited by 73
Yin, C., Meng, F., Yang, X., Yang, F., Fu, P., Yao, G., & Chen, R. (2022) · Land Use Policy · doi:10.1016/j.landusepol.2022.106114 · Cited by 51
Migration, Schooling Choice, and Student Outcomes in China
Wang, X., Bai, Y. P., Zhang, L., & Rozelle, S. (2017) · Population and Development Review · doi:10.1111/padr.12101 · Cited by 34
Geography Achievement and Future Geographers
Solem, M. (2022) · The Professional Geographer · doi:10.1080/00330124.2022.2081227 · Cited by 27
Geographies of Imaginaries and Environmental Governance
Chhetri, N., Ghimire, R., & Eisenhauer, D. C. (2022) · The Professional Geographer · doi:10.1080/00330124.2022.2087698 · Cited by 16
Cited by
Abulibdeh, A., Zaidan, E., & Abulibdeh, R. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2023.140527 · Cited by 641
Fabrication and errors in the bibliographic citations generated by ChatGPT
Walters, W. H., & Wilder, E. I. (2023) · Scientific Reports · doi:10.1038/s41598-023-41032-5 · Cited by 499
“Bhattacharyya et al. 27 found that 87% of the citations to real (non-fabricated) works had 1 or more of 7 errors—incorrect PubMedID number, author name, article title, date, journal title, volume number, or page numbers—and Day 28 reported very similar results.”
BackgroundResultChelli, M., Descamps, J., Lavoué, V., Trojani, C., Azar, M., Deckert, M., Raynier, J.-L., Clowez, G., Boileau, P., & Ruetsch-Chelli, C. (2024) · Journal of Medical Internet Research · doi:10.2196/53164 · Cited by 602
AI literacy and its implications for prompt engineering strategies
Knoth, N., Tolzin, A., Janson, A., & Leimeister, J. M. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100225 · Cited by 368
“Similar to how digital literacy empowered individuals to use digital information and communication technologies, developing AI literacy becomes increasingly important for interacting with the omnipresent AI systems in our personal and professional spheres (Ga ˇ sevi ´ c, Siemens, & Sadiq, 2023; Ng et al., 2021; Vuorik…”
BackgroundInfluentialLiu, M., Zhang, L. J., & Biebricher, C. (2024) · Computers & Education · doi:10.1016/j.compedu.2023.104977 · Cited by 282
Metadata
- Authors
- Day, T.
- Affiliations
- Okanagan College (CA)
- Corresponding author
- Day, Terence
- ORCID
- Day, Terence
- Venue
- The Professional Geographer
- Publisher
- Informa UK Limited
- Published
- 2023-04-12
- Volume
- 75
- Issue
- 6
- Pages
- 1024-1027
- ISSN
- 0033-0124 (ISSN-L)
- 1467-9272
- OpenAlex
- W4365143850
- Semantic Scholar
- 324bee1adcaafc336f0255980db8544fe883449f
- Cited by
- 160
- References
- 13
- Citations by year
- 2026: 29 · 2025: 41 · 2024: 51 · 2023: 38
- FWCI
- 40.32
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Professional Geographer (publishedVersion)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Keywords
- ChatGPT, AI-generated citations, citation detection, academic integrity, AI-assisted research, undergraduate education, information literacy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Preliminary Investigation of Fake Peer-Reviewed Citations and References Generated by ChatGPT. https://app.cochat.ai/papers/10.1080/00330124.2023.2190373
- The paper
- Day, T. (2023). A Preliminary Investigation of Fake Peer-Reviewed Citations and References Generated by ChatGPT. The Professional Geographer. https://doi.org/10.1080/00330124.2023.2190373
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.