The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
ChatGPT and the potential growing of ghost bibliographic references
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A recent paper suggests that while this is unlikely to be a problem with the final versions of published articles, fabricated citations are more likely to appear in the papers posted to preprint servers and institutional repositories35.”
From Fabrication and errors in the bibliographic citations generated by ChatGPT (2023) · cites this paper as [35]
In the citing paper
“In contrast to previous anecdotal reports or general evaluations, 20,21 there remains a lack of systematic research quantifying the accuracy and correction potential of hallucinated references in musculoskeletal-focused scientific content.”
From Fabricated or accurate? Ethical concerns and citation hallucination in aI-generated scientific writing on musculoskeletal topics (2025) · cites this paper as [21]
Citations
Cites
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., et al. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102642 · Cited by 4,584
Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24750 · Cited by 919 · cited 1×
“However, ChatGPT still has numerous errors and limitations, so responsible use of the tool is recommended to make the most of it (Lund et al., 2023).”
ChatGPT and the Rise of Large Language Models: The New AI-Driven Infodemic Threat in Public Health
De Angelis, L., Baglivo, F., Arzilli, G., Privitera, G. P., Ferragina, P., Tozzi, A. E., & Rizzo, C. (2023) · doi:10.2139/ssrn.4352931 · Cited by 708
ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health
De Angelis, L., Baglivo, F., Arzilli, G., Privitera, G. P., Ferragina, P., Tozzi, A. E., & Rizzo, C. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1166120 · Cited by 708 · cited 1×
“When the chatbot is asked for bibliographic references on a specific subject, it provides plausible results, with real or potentially real titles published in leading journals in the discipline and by prominent authors in it, but they are ghost references (Angelis et al., 2023; Day, 2023).”
BackgroundOn the use of AI-based tools like ChatGPT to support management research
Burger, B., Kanbach, D. K., Kraus, S., Breier, M., & Corvello, V. (2023) · European Journal of Innovation Management · doi:10.1108/ejim-02-2023-0156 · Cited by 310 · cited 1×
“Its potential advantages in different aspects of scientific research have been pointed out in various works (Burger et al., 2023; Dwivedi et al., 2023).”
BackgroundThe Google scholar experiment: How to index false papers and manipulate bibliometric indicators
Delgado López‐Cózar, E., Robinson‐García, N., & Torres‐Salinas, D. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23056 · Cited by 272
A Preliminary Investigation of Fake Peer-Reviewed Citations and References Generated by ChatGPT
Day, T. (2023) · The Professional Geographer · doi:10.1080/00330124.2023.2190373 · Cited by 160 · cited 2×
“This would add an extra burden to legitimate journals, as they should verify the authenticity of references before publication (Day, 2023).”
BackgroundLabbé, C., & Labbé, D. (2013) · Scientometrics · doi:10.1007/s11192-012-0781-y · Cited by 132 · cited 1×
“The manipulation of citing and falsely cited documents in Google Scholar has been discussed previously in the literature (see Labbé., 2010; Labbé & Labbé, 2013; Delgado López ‐ Cózar, Robinson ‐ García & Torres ‐ Salinas, 2014).”
MethodIke Antkare one of the great stars in the scientific firmament
Antkare, I. (2010) · Cited by 83
Cited by
Fabrication and errors in the bibliographic citations generated by ChatGPT
Walters, W. H., & Wilder, E. I. (2023) · Scientific Reports · doi:10.1038/s41598-023-41032-5 · Cited by 499
“A recent paper suggests that while this is unlikely to be a problem with the final versions of published articles, fabricated citations are more likely to appear in the papers posted to preprint servers and institutional repositories 35 .”
BackgroundChatGPT: A brief narrative review
Gupta, B., Mufti, T., Sohail, S. S., & Madsen, D. Ø. (2023) · Cogent Business & Management · doi:10.1080/23311975.2023.2275851 · Cited by 107
“For example, several authors have noted that ChatGPT sometimes makes up facts and “hallucinates” (Alkaissi & McFarlane, 2023; Azamfirei et al., 2023), for example, by generating “ghost” bibliographic references that sound plausible but in reality are fake (Orduña-Malea & Cabezas-Clavijo, 2023).”
BackgroundArtificial intelligence in the tourism sector: Its sustainability and innovation potential
Dávid, L. D., & Dadkhah, M. (2023) · Equilibrium Quarterly Journal of Economics and Economic Policy · doi:10.24136/eq.2023.019 · Cited by 29
ChatGPT and the future of impact assessment
Khan, M., Chaudhry, M. N., Ahsan, M., & Ahmad, R. (2024) · Environmental Science & Policy · doi:10.1016/j.envsci.2024.103779 · Cited by 36
Goyanes, M., Lopezosa, C., & Piñeiro-Naval, V. (2025) · Scientometrics · doi:10.1007/s11192-025-05377-0 · Cited by 29
“Misuse of these technologies can distort research outcomes, undermining the very foundations of scientific inquiry (Naik, 2022; Orduña-Malea & Cabezas-Clavijo, 2023).”
Metadata
- Authors
- Orduña-Malea, E., & Cabezas-Clavijo, Á.
- Affiliations
- Universidad Internacional de La Rioja (ES)
- Venue
- Scientometrics
- Published
- 2023-07-31
- Volume
- 128
- Issue
- 9
- Pages
- 5351-5355
- ISSN
- 0138-9130 (ISSN-L)
- 1588-2861
- OpenAlex
- W4385417324
- Cited by
- 31
- References
- 8
- Citations by year
- 2026: 9 · 2025: 10 · 2024: 5 · 2023: 7
- FWCI
- 7.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Scientometrics (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Re-Unir (International University of La Rioja) (submittedVersion)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- ChatGPT, large language models, AI-generated references, predatory journals, preprint servers, publication ethics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ChatGPT and the potential growing of ghost bibliographic references. https://app.cochat.ai/papers/10.1007/s11192-023-04804-4
- The paper
- Orduña-Malea, E., & Cabezas-Clavijo, Á. (2023). ChatGPT and the potential growing of ghost bibliographic references. Scientometrics. https://doi.org/10.1007/s11192-023-04804-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.