The DOI matches its Crossref record; 95 of the 95 references it cites were found in a registry or an index.
The Utility of ChatGPT as an Example of Large Language Models in Healthcare Education, Research and Practice: Systematic Review on the Future Perspectives and Potential Limitations
Open accessPreprint — published as 10.3390/healthcare11060887 · Paper page
Abstract
An artificial intelligence (AI)-based conversational large language model (LLM) was launched in November 2022 namely, “ChatGPT”. Despite the wide array of potential applications of LLMs in healthcare education, research and practice, several valid concerns were raised. The current systematic review aimed to investigate the possible utility of ChatGPT and to highlight its limitations in healthcare education, research and practice. Using the PRIMSA guidelines, a systematic search was conducted to retrieve English records in PubMed/MEDLINE and Google Scholar under the term “ChatGPT”. Eligibility criteria included the published research or preprints of any type that discussed ChatGPT in the context of healthcare education, research and practice. A total of 280 records were identified, and following full screening, a total of 60 records were eligible for inclusion. Benefits/applications of ChatGPT were cited in 51/60 (85.0%) records with the most common being the utility in scientific writing followed by benefits in healthcare research (efficient analysis of massive datasets, code generation and rapid concise literature reviews besides utility in drug discovery and development). Benefits in healthcare practice included cost saving, documentation, personalized medicine and improved health literacy. Concerns/possible risks of ChatGPT use were expressed in 58/60 (96.7%) records with the most common being the ethical issues including the risk of bias, plagiarism, copyright issues, transparency issues, legal issues, lack of originality, incorrect responses, limited knowledge, and inaccurate citations. Despite the promising applications of ChatGPT which can result in paradigm shifts in healthcare education, research and practice, the embrace of this application should be done with extreme caution. Specific applications of ChatGPT in health education include the promising utility in personalized learning tools and shift towards more focus on critical thinking and problem-based learning. In healthcare practice, ChatGPT can be valuable for streamlining the workflow and refining personalized medicine. Saving time for the focus on experimental design and enhancing research equity and versatility are the benefits in scientific research. Regarding authorship in scientific articles, as it currently stands, ChatGPT does not qualify to be listed as an author unless the ICMJE/COPE guidelines are revised and amended. An initiative involving all stakeholders involved in healthcare education, research and practice is urgently needed to set a code of ethics and conduct on the responsible practices involving ChatGPT among other LLMs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In this exploratory study, we observed a high level of ease of use among the small group of participants who reported using ChatGPT, likely due to its user-friendly nature and free accessibility [17,71,90].”
From Assessing Health Students' Attitudes and Usage of ChatGPT in Jordan: Validation Study (2023) · cites this paper as [71]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“With regard to transparency, two sources suggest that LLM-supported diagnoses hamper the process of providing adequate justification due to their opacity36,74.”
From The ethics of ChatGPT in medicine and healthcare: a systematic review on large language models (LLMs) (2024) · cites this paper as [74]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Unintended negative consequences, such as inappropriate or inaccurate content, pose significant risks in health care settings, necessitating careful consideration before its adoption in health care education [9,69-71].”
From Assessing Health Students' Attitudes and Usage of ChatGPT in Jordan: Validation Study (2023) · cites this paper as [71]
The cited source says what the claim attributes to it.
In the citing paper
“LLMs have many potentially beneficial applications in healthcare practice and research, including diagnostic (e.g., prediction of disease risk and outcomes) and procedural (e.g., streamlining of clinical workflows, documentation, cost-effectiveness) tasks59.”
From Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging (2024) · cites this paper as [59]
The cited source says what the claim attributes to it.
In the citing paper
“Consequently, there is a noticeable demand for ethical guidelines, as evidenced within the literature surrounding healthcare applications of LLMs46,60,64,70,71,74,75,78.”
From The ethics of ChatGPT in medicine and healthcare: a systematic review on large language models (LLMs) (2024) · cites this paper as [74]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · cited 1×
“The current systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRIMSA) guidelines [20].”
MethodLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 2×
“, San Francisco, CA, USA), ChatGPT etymology is related to being a chatbot (a program able to understand and generate responses using a text-based interface) and being based on the Generative Pre-trained Transformer (GPT) architecture [6,7].”
BackgroundKaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955 · cited 1×
“This was followed by the development of machine learning (ML) algorithms that allow decision making or predictions based on the patterns in large datasets [4].”
BackgroundKung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · cited 2×
“The need to rethink and revise the current assessment tools in healthcare education comes in light of ChatGPT’s ability to pass reputable exams (e.g., USMLE), and possibility of ChatGPT misuse that would result in increased academic dishonesty [28,34,64,66,68,72].”
BackgroundRajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212 · cited 1×
“The utility of AI in healthcare appears promising with possible applications in personalized medicine, drug discovery, and analysis of large datasets has been outlined previously besides the potential benefits in improving diagnosis and clinical decisions [17,18].”
BackgroundChatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087 · cited 1×
“The need to rethink and revise the current assessment tools in healthcare education comes in light of ChatGPT’s ability to pass reputable exams (e.g., USMLE), and possibility of ChatGPT misuse that would result in increased academic dishonesty [28,34,64,66,68,72].”
BackgroundChatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023) · JMIR Medical Education · doi:10.2196/45312 · Cited by 2,078
A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955
McCarthy, J., Minsky, M., Rochester, N., et al. (2006) · The AI Magazine · doi:10.1609/aimag.v27i4.1904 · Cited by 2,029 · cited 1×
“The history of AI as a scientific discipline can be traced back to the mid-XX century at the Dartmouth Summer Research Project on AI [3].”
Background
Research works printed but not linked: 4
- Domingos, P. The master algorithm: how the quest for the ultimate learning machine will remake our world, First paperback edition ed.; Basic Books, a member of the Perseus Books Group: New York, 2018; p. 329 pages.
- 10.13140/RG.2.2.31280.23043/1 · DOI
- 10.52866/IJCSM.2023.01.01.0018 · doi:10.52866/ijcsm.2023.01.01.0018 · Paper page
- 10.1258/jrsm.99.4.178 · doi:10.1258/jrsm.99.4.178 · Paper page
Cited by
What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature
Lo, C. K. (2023) · Education Sciences · doi:10.3390/educsci13040410 · Cited by 2,024
“Immediate action must be taken to mitigate the impact of ChatGPT on education.”
BackgroundA SWOT analysis of ChatGPT: Implications for educational practice and research
Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2195846 · Cited by 1,384
The imperative for regulatory oversight of large language models (or generative AI) in healthcare
Meskó, B., & Topol, E. J. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00873-0 · Cited by 1,047
“Systematic reviews highlighted other potential bene fi ts too such as improved scienti fi c writing, enhancing research equity, streamlining the healthcare work fl ow, cost saving, and improved personalized learning in medical education 3,4 .”
Practical and ethical challenges of large language models in education: A systematic scoping review
Yan, L., Sha, L., Zhao, L., Li, Y., Martinez‐Maldonado, R., Chen, G., Li, X., Jin, Y., & Gašević, D. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13370 · Cited by 1,435
“%) focused on using LLMs to automate classification tasks, there could be more future studies that aimed to tackle the automation of prediction and generation tasks with the more capable LLMs (Sallam, 2023).”
BackgroundMethodInfluentialLarge language models in healthcare and medical domain: A reviewPreprint
Nazi, Z. A., & Peng, W. (2023) · Informatics · doi:10.48550/arxiv.2401.06775 · Cited by 551
“It is therefore crucial for medical professionals to exercise caution and thoroughly review and validate the recommendations provided by large language models before integrating them into their clinical decision-making processes [127].”
Background
Metadata
- Authors
- Sallam, M.
- Affiliations
- University of Jordan (JO)
- Corresponding author
- Sallam, Malik
- ORCID
- Sallam, Malik
- Venue
- Healthcare
- Publisher
- openRxiv
- Published
- 2023-02-21
- Volume
- 11
- Issue
- 6
- Pages
- 887-887
- ISSN
- 2227-9032 (ISSN-L)
- PMC
- PMC10048148
- OpenAlex
- W4327946446
- Semantic Scholar
- 3bbb397ea3b09a10de198f167fba045da680b4e8
- Cited by
- 3,123
- References
- 78
- Citations by year
- 2026: 558 · 2025: 839 · 2024: 1052 · 2023: 669
- FWCI
- 36.63
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Healthcare (publishedVersion, cc-by)
- medRxiv (acceptedVersion, cc-by-nc-nd)
- Preprints.org (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Social Sciences › Education › Artificial Intelligence in Education
- Keywords
- ChatGPT, large language models, artificial intelligence in healthcare, healthcare education, health care research, clinical practice, scientific writing, research equity, drug discovery, personalized medicine, health literacy, plagiarism, hallucinations, authorship criteria, AI ethics, infodemic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Utility of ChatGPT as an Example of Large Language Models in Healthcare Education, Research and Practice: Systematic Review on the Future Perspectives and Potential Limitations. https://app.cochat.ai/papers/10.1101/2023.02.19.23286155
- The paper
- Sallam, M. (2023). The Utility of ChatGPT as an Example of Large Language Models in Healthcare Education, Research and Practice: Systematic Review on the Future Perspectives and Potential Limitations. Healthcare. https://doi.org/10.1101/2023.02.19.23286155
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.