Cited by 69Artificial Intelligence in Healthcare and Educationscientometrics and bibliometrics researchAcademic Writing and Publishing

A bibliometric analysis of the rise of ChatGPT in medical research

Open access
Barrington, Nikki M., Gupta, Nithin, Musmar, Basel, Doyle, David, Panico, Nicholas, Godbole, Nikhil, Reardon, Taylor, D’Amico, Randy S.·2023·Medical Sciences

Abstract

The rapid emergence of publicly accessible artificial intelligence platforms such as large language models (LLMs) has led to an equally rapid increase in articles exploring their potential benefits and risks. We performed a bibliometric analysis of ChatGPT literature in medicine and science to better understand publication trends and knowledge gaps. Following title, abstract, and keyword searches of PubMed, Embase, Scopus, and Web of Science databases for ChatGPT articles published in the medical field, articles were screened for inclusion and exclusion criteria. Data were extracted from included articles, with citation counts obtained from PubMed and journal metrics obtained from Clarivate Journal Citation Reports. After screening, 267 articles were included in the study, most of which were editorials or correspondence with an average of 7.5 +/- 18.4 citations per publication. Published articles on ChatGPT were authored largely in the United States, India, and China. The topics discussed included use and accuracy of ChatGPT in research, medical education, and patient counseling. Among non-surgical specialties, radiology published the most ChatGPT-related articles, while plastic surgery published the most articles among surgical specialties. The average citation number among the top 20 most-cited articles was 60.1 +/- 35.3. Among journals with the most ChatGPT-related publications, there were on average 10 +/- 3.7 publications. Our results suggest that managing the inevitable ethical and safety issues that arise with the implementation of LLMs will require further research exploring the capabilities and accuracy of ChatGPT, to generate policies guiding the adoption of artificial intelligence in medicine and science.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/medsci11030061

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The issue of transparency and interpretability is even more pronounced with some powerful generative models, where technical details, model weights, and datasets are proprietary and undisclosed.

    “This issue of transparency/interpretability is even more pronounced with some of the powerful generative models, where technical details, model weights, and datasets are proprietary and undisclosed.6”

    From Advancing methodological development of artificial intelligence in patient-centered comparative clinical effectiveness research: Patient-Centered Outcomes Research Institute’s unique contribution to research done differently (2025) · cites this paper as [6]

    The Discussion addresses transparency about where and how AI was used in a publication, not the interpretability of generative models. Although its Limitations section notes that LLaMA is open source, our copy does not say that other models’ technical details, weights, or datasets are proprietary and undisclosed. Searches for those terms and for model opacity found no supporting passage. (LLM, conf=0.73)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. [7]Bahdanau D. Cho K. Bengio Y. Neural Machine Translation by Jointly Learning to Align and Translate arXiv 2016 1409.04732014 Available online: https://arxiv.org/abs/1409.0473 (accessed on 1 August 2023) · arXiv
  2. [31]First Time Journal Citation Reports Inclusion List 2023. Clarivate Available online: https://clarivate.com/first-time-journal-citation-reports-inclusion-list-2023/ (accessed on 1 August 2023)
  3. [34]Country/Territory Tables | Nature Index Available online: https://www.nature.com/nature-index/country-outputs/generate/all/global (accessed on 4 September 2023)

Cited by

Sorted by citations

Authors
Barrington, N. M., Gupta, N., Musmar, B., Doyle, D., Panico, N., Godbole, N., Reardon, T., & D’Amico, R. S.
Affiliations
  • Rosalind Franklin University of Medicine and Science (US)
  • Campbell University (US)
  • An-Najah National University (PS)
  • Central Michigan University (US)
  • Lake Erie College of Osteopathic Medicine (US)
  • Tulane University (US)
  • Henry Ford Health (US)
  • Lenox Hill Hospital (US)
Corresponding author
Barrington, Nikki M.
Venue
Medical Sciences
Publisher
MDPI AG
Published
2023-09-17
Volume
11
Issue
3
Pages
61
ISSN
2076-3271 (ISSN-L)
OpenAlex
W4386837623
Cited by
69
References
37
Citations by year
2026: 8 · 2025: 29 · 2024: 30 · 2023: 2
FWCI
0.81
Type
journal article
Language
en
Open access
Locations
  • Medical Sciences (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
Keywords
ChatGPT, large language models, artificial intelligence in healthcare, medical research, publication trends, citation analysis, medical education, patient counseling, radiology, surgical specialties, AI adoption policy
MeSH terms
Artificial Intelligence, Humans, Radiology, Bibliometrics, Benchmarking, Biomedical Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review