The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
ChatGPT References Unveiled: Distinguishing the Reliable from the Fake
Abstract
This study explored the potential of ChatGPT in the search for reliable references. Results showed that the reliability of ChatGPT's references vary across different source types. For journal articles, they were all fabricated and not compliant with APA 7 format. Webpages did not match actual existing sites. However, for books, the generated references correctly corresponded to real, existing ones. In the case of journal websites, the links within the responses were functional. Although this exploration gained interesting insights, researchers are strongly recommended not to depend on ChatGPT for references due to accuracy issues and must undertake manual, scholarly searches.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Artificial hallucinations in ChatGPT: implications in scientific writing
Alkaissi, H., & McFarlane, S. I. (2023) · Cureus · doi:10.7759/cureus.35179 · Cited by 1,540 · cited 1×
“As ChatGPT demonstrates proficiency in crafting credible scientific narratives, the content it generates is a mixture of factual and entirely contrived information (Alkaissi & McFarlane, 2023).”
BackgroundPrompt Engineering with ChatGPT: A Guide for Academic Writers
Giray, L. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03272-4 · Cited by 776 · cited 3×
“Certain individuals adeptly navigate its prompts, most especially the ones who are versatile in prompt engineering to attain their specific objectives (Giray, 2023a; Lo, 2023).”
BackgroundSelf-citationArtificial intelligence in scientific writing: a friend or a foe?
Altmäe, S., Sola-Leyva, A., & Salumets, A. (2023) · Reproductive BioMedicine Online · doi:10.1016/j.rbmo.2023.04.009 · Cited by 174 · cited 1×
“ChatGPT has garnered acknowledgment as a valuable tool for diverse research endeavors, as evidenced by Altmäe, Sola-Leyva, and Salumets (2023).”
MethodA Preliminary Investigation of Fake Peer-Reviewed Citations and References Generated by ChatGPT
Day, T. (2023) · The Professional Geographer · doi:10.1080/00330124.2023.2190373 · Cited by 160 · cited 2×
“Interestingly, while ChatGPT is known for generating fake references (Day, 2023), all of the reference books generated by ChatGPT here are real existing books.”
BackgroundChatGPT and Bard exhibit spontaneous citation fabrication during psychiatry literature search
McGowan, A., Gui, Y., Dobbs, M., Shuster, S., Cotter, M., Selloni, A., Goodman, M., Srivastava, A., Cecchi, G. A., & Corcoran, C. M. (2023) · Psychiatry Research · doi:10.1016/j.psychres.2023.115334 · Cited by 140 · cited 1×
“Regarding its utility in generating references, a study conducted by McGowan et al. (2023) revealed that out of 35 citations produced, only two could be verified as accurate, with 12 closely resembling authentic manuscripts but exhibiting discrepancies, and 21 appearing as amalgama-tions of various…”
BackgroundThe Art and Science of Prompt Engineering: A New Literacy in the Information Age
Lo, L. S. (2023) · Internet Reference Services Quarterly · doi:10.1080/10875301.2023.2227621 · Cited by 136 · cited 1×
“Certain individuals adeptly navigate its prompts, most especially the ones who are versatile in prompt engineering to attain their specific objectives (Giray, 2023a; Lo, 2023).”
BackgroundChatGPT and Its Possible Impact on Library Reference Services
Chen, X. (2023) · Internet Reference Services Quarterly · doi:10.1080/10875301.2023.2181262 · Cited by 96 · cited 1×
“However, just like what Chen (2023) pointed out, researchers should not ignore the valuable lessons ChatGPT offers, or miss out on its potential benefits and, at the same time, they must not ignore the possible problems it can bring.”
BackgroundHua, H.-U., Kaakour, A.-H., Rachitskaya, A., Srivastava, S., Sharma, S., & Mammo, D. A. (2023) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2023.3119 · Cited by 76 · cited 1×
“While Hua et al. (2023) discovered a significant occurrence of fictitious reference generation when utilizing ChatGPT, this study strongly emphasizes the need for caution when employing such tools for research due to the potential for producing unreliable output.”
MethodChatGPT and Librarians for Reference Consultations
Adetayo, A. J. (2023) · Internet Reference Services Quarterly · doi:10.1080/10875301.2023.2203681 · Cited by 72 · cited 1×
“Despite these shortcomings, students expressed interest in seeking reference assistance from ChatGPT alongside librarians (Adetayo, 2023).”
BackgroundGiray, L. (2023) · Journal of the Practice of Cardiovascular Sciences · doi:10.4103/jpcs.jpcs_45_23 · Cited by 19 · cited 3×
“Certain individuals adeptly navigate its prompts, most especially the ones who are versatile in prompt engineering to attain their specific objectives (Giray, 2023a; Lo, 2023).”
BackgroundSelf-citation
Research works printed but not linked: 1
- 10.1177/0846537123117112 · doi:10.1177/0846537123117112
Cited by
Giray, L., Sevnarayan, K., & Ranjbaran, F. (2025) · Internet Reference Services Quarterly · doi:10.1080/10875301.2024.2437174 · Cited by 78
“AI can significantly improve the efficiency and quality of academic writing by assisting in outlining, drafting, and editing, thereby easing the communication burden and empowering writers (Giray, 2023).”
Self-citationStrzelecki, A. (2024) · Learned Publishing · doi:10.1002/leap.1650 · Cited by 34
“The literature to date has paid attention to bibliographic citations and the fact that ChatGPT does not provide real information (Giray, 2024).”
Acut, D. P., Malabago, N. K., Malicoban, E. V., Galamiton, N. S., & Garcia, M. B. (2024) · Internet Reference Services Quarterly · doi:10.1080/10875301.2024.2426793 · Cited by 28
“This aligns with previous research by Alyasiri et al. (2024) and Giray (2023a), which highlighted that AI models tend to struggle with abstract or less structured prompts.”
InfluentialSpennemann, D. H. R. (2025) · Publications · doi:10.3390/publications13010012 · Cited by 23
“…or ‘confabulate’ (Alkaissi & McFarlane, 2023; Bang et al., 2023; Millidge, 2023), generating both genuine and spurious output (Athaluri et al., 2023; Giray, 2024; Spennemann, 2023a , Given the potential of generative Ai models to become an eminently suitable tool for public interaction, it is…”
Cabezas-Clavijo, Á., & Sidorenko-Bautista, P. (2026) · Journal of Data and Information Science · doi:10.1515/jdis-2025-0326 · Cited by 22
Metadata
- Authors
- Giray, L.
- Affiliations
- Colegio de Muntinlupa (PH)
- Corresponding author
- Giray, Louie
- ORCID
- Giray, Louie
- Venue
- Internet Reference Services Quarterly
- Publisher
- Informa UK Limited
- Published
- 2023-10-05
- Volume
- 28
- Issue
- 1
- Pages
- 9-18
- ISSN
- 1087-5301 (ISSN-L)
- 1540-4749
- OpenAlex
- W4387363411
- Cited by
- 26
- References
- 10
- Citations by year
- 2026: 7 · 2025: 13 · 2024: 6
- FWCI
- 20.41
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Internet Reference Services Quarterly (publishedVersion)
- Topics
- Social Sciences › Library and Information Sciences › Library Science and Information Literacy
- Social Sciences › Library and Information Sciences › Library Science and Information
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Keywords
- ChatGPT, reference generation, citation reliability, source verification, journal articles, web pages, books
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ChatGPT References Unveiled: Distinguishing the Reliable from the Fake. https://app.cochat.ai/papers/10.1080/10875301.2023.2265369
- The paper
- Giray, L. (2024). ChatGPT References Unveiled: Distinguishing the Reliable from the Fake. Internet Reference Services Quarterly. https://doi.org/10.1080/10875301.2023.2265369
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.