Considering the possibilities and pitfalls of Generative Pre-trained Transformer 3 (GPT-3) in healthcare delivery
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Natural language computer applications are becoming increasingly sophisticated and, with the recent release of Generative Pre-trained Transformer 3, they could be deployed in healthcare-related contexts that have historically comprised human-to-human interaction. However, for GPT-3 and similar applications to be considered for use in health-related contexts, possibilities and pitfalls need thoughtful exploration. In this article, we briefly introduce some opportunities and cautions that would accompany advanced Natural Language Processing applications deployed in eHealth.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These models are trained on large amounts of text, for example, obtained from the internet, and can answer questions, provide summaries or translations and create stories or poems (Fig. 1a)3,4.”
From The future landscape of large language models in medicine (2023) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, these models also raise concerns about misinformation, privacy, biases in the training data, and potential for misuse3,7–10.”
From The future landscape of large language models in medicine (2023) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, ChatGPT has the potential to provide logically coherent but incorrect responses or inaccurate information due to its inability to consciously assess the accuracy of its output text [75,77].”
From Ethical Considerations of Using ChatGPT in Health Care (2023) · cites this paper as [77]
Experimental — accuracy not yet measuredIn the citing paper
“Due to the inherent characteristics of LLMs, LLM-driven chatbots may be error-prone [48] or digress from their tasks [103].”
From Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data (2024) · cites this paper as [48]
In the citing paper
“This example suggests the importance of giving precautions to users that the chatbots' messages do not guarantee medical or professional soundness [48].”
From Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data (2024) · cites this paper as [48]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 6×
“…at OpenAI think it performs well on translation, question answering, and cloze tasks (e.g., a fi ll-in-the-blank test to demonstrate comprehension of text by providing the missing words in a sentence) 1 , it does not always predict a correct string of words that are believable as a conversation.”
BackgroundMethodInfluentialI.—COMPUTING MACHINERY AND INTELLIGENCE
TURING, A. M. (1950) · Mind · doi:10.1093/mind/lix.236.433 · Cited by 10,290
Computing Machinery and Intelligence (1950)
Turing, A. (2004) · The Essential Turing · doi:10.1093/oso/9780198250791.003.0017 · Cited by 6,118
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
GPT-3: Its Nature, Scope, Limits, and Consequences
Floridi, L., & Chiriatti, M. (2020) · Minds and Machines · doi:10.1007/s11023-020-09548-1 · Cited by 2,519
Natural language processing: an introduction
Nadkarni, P. M., Ohno-Machado, L., & Chapman, W. W. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000464 · Cited by 1,975
Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties
Sinsky, C., Colligan, L., Li, L., Prgomet, M., Reynolds, S., Goeders, L., Westbrook, J., Tutty, M., & Blike, G. (2016) · Annals of Internal Medicine · doi:10.7326/m16-0961 · Cited by 1,494
Translational AI and deep learning in diagnostic pathology
Serag, A., Ion-Margineanu, A., Qureshi, H., McMillan, R., Saint Martin, M.-J., Diamond, J., O'Reilly, P., & Hamilton, P. (2019) · Frontiers in Medicine · doi:10.3389/fmed.2019.00185 · Cited by 274
Anthropomorphism in Human–Robot Co-evolution
Damiano, L., & Dumouchel, P. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00468 · Cited by 252
Can GPT-3 pass a writer’s turing test?
Elkins, K., & Chun, J. (2020) · Journal of Cultural Analytics · doi:10.22148/001c.17212 · Cited by 202 · cited 2×
“One of the major limitations of GPT-3 is that it repeats itself semantically, loses coherence over long conversations, and contradicts itself 1,14 .”
Background
Cited by
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
“While the medical domain is challenging, specific proposals for LLMs have already included examples as varied as augmenting non-critical clinical assessments to summarisation of complex medical communications [3, 41, 75].”
BackgroundLarge Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
“While the medical domain is challenging, specific proposals for LLMs have already included examples as varied as augmenting non-critical clinical assessments to summarisation of complex medical communications [3, 41, 75].”
BackgroundOn the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00819-6 · Cited by 1,099
“ChatGPT is built on Generative Pre-trained Transformer-3 (GPT-3), which is one of the largest of these types of models, trained with 175 billion parameters 15 .”
The future landscape of large language models in medicine
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J.-N., Laleh, N. G., Löffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., et al. (2023) · Communications Medicine · doi:10.1038/s43856-023-00370-1 · Cited by 1,045
Metadata
- Authors
- Korngiebel, D. M., & Mooney, S. D.
- Affiliations
- Hastings Center (US)
- University of Washington (US)
- Corresponding author
- Korngiebel, Diane M.
- Venue
- npj Digital Medicine
- Published
- 2021-06-03
- Volume
- 4
- Issue
- 1
- Pages
- 93-93
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC8175735
- OpenAlex
- W3164718925
- Cited by
- 259
- References
- 18
- Citations by year
- 2026: 22 · 2025: 45 · 2024: 83 · 2023: 89 · 2022: 19 · 2021: 1
- FWCI
- 13.82
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- U.S. Department of Health & Human Services | National Institutes of Health (R21HG011277)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- GPT-3, large language models, natural language processing, health care delivery, e-health, human-to-human interaction, opportunities and risks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Considering the possibilities and pitfalls of Generative Pre-trained Transformer 3 (GPT-3) in healthcare delivery. https://app.cochat.ai/papers/10.1038/s41746-021-00464-x
- The paper
- Korngiebel, D. M., & Mooney, S. D. (2021). Considering the possibilities and pitfalls of Generative Pre-trained Transformer 3 (GPT-3) in healthcare delivery. npj Digital Medicine. https://doi.org/10.1038/s41746-021-00464-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.