CoChat
Sign Up
Cited by 213Artificial Intelligence in Healthcare and EducationCOVID-19 Clinical Research StudiesMachine Learning in Healthcare

Opportunities and Risks of ChatGPT in Medicine, Science, and Academic Publishing: A Modern Promethean Dilemma

Open access
Homolak, Jan·2023·Croatian Medical Journal
DOI 10.3325/cmj.2023.64.1PMC PMC10028563
Open this paper in CoChat — free accountOpen PDF

Abstract

Aim To evaluate the predictive properties of several com - mon prognostic scores regarding survival outcomes in hospitalized COVID-19 patients. Methods We retrospectively reviewed the medical records of 4014 consecutive COVID-19 patients hospitalized in our tertiary level institution from March 2020 to March 2021. Prognostic properties of the WHO COVID-19 severity clas - sification, COVID-GRAM, Veterans Health Administration COVID-19 (VACO) Index, 4C Mortality Score, and CURB65 score regarding 30-day mortality, in-hospital mortality, presence of severe or critical disease on admission, need for an intensive care unit treatment, and mechanical venti - lation during hospitalization were evaluated. Results All of the investigated prognostic scores signifi - cantly distinguished between groups of patients with differ - ent 30-day mortality. The CURB-65 and 4C Mortality Score had the best prognostic properties for prediction of 30-day mortality (area under the curve [AUC] 0.761 for both) and in-hospital mortality (AUC 0.757 and 0.762, respectively). The 4C Mortality Score and COVID-GRAM best predicted the presence of severe or critical disease (AUC 0.785 and 0.717, respectively). In the multivariate analysis evaluating 30-day mortality, all scores mutually independently provid - ed additional prognostic information, except the VACO In - dex, whose prognostic properties were redundant. Conclusion Complex prognostic scores based on many parameters and comorbid conditions did not have better prognostic properties regarding survival outcomes than a simple CURB-65 prognostic score. CURB-65 also provides the largest number of prognostic categories (five), allowing more precise risk stratification than other prognostic scores.

AI snapshot

Written from the paper’s full text.

TL;DR

This commentary argues that ChatGPT and similar conversational AI will not replace physicians but can substantially augment their work, while posing serious risks to scientific integrity and academic publishing. It calls for mindful, debated adoption rather than hasty implementation or outright bans.

Key findings

  • ChatGPT is a 175-billion-parameter model that allegedly gained more than 1 million users in its first few days and 100 million within 2 months, making it the fastest-growing consumer application in history.
  • On knowledge-based medical tests, ChatGPT scored ~66% on Basic Life Support and ~72% on Advanced Cardiovascular Life Support, and performed at or near the passing threshold on the United States Medical Licensing Exam; however, the author notes algorithms remain "notoriously bad at context and nuance."
  • Citing Frey and Osborne, the probability of automating physicians' and surgeons' jobs is 0.42%, versus 91% for health information technicians — supporting the claim that AI's value lies in redistributing workload, not replacing clinicians.
  • In one cited experiment, 50 ChatGPT-generated research abstracts passed a plagiarism checker, an AI-output detector, and human reviewers; the author warns the chatbot also fabricates plausible-sounding references, and StackOverflow banned its generated text because the average rate of correct answers is too low.

Methods

  • Design: a narrative commentary/opinion piece (no primary data collection or systematic methodology), structured around three questions — whether ChatGPT can replace physicians, whether it can act as a "chatbot-scientist," and what should happen next.
  • Evidence base: synthesis of cited literature, news reports, publisher policies, and social media posts, including studies of ChatGPT's exam performance, abstract generation, and review-article writing.
  • Main analysis: qualitative argumentation about opportunities (reduced paperwork, faster literature search, shifting research focus from presentation to content) and risks (bias amplification, fabricated sources, accountability and plagiarism gaps, lack of legal frameworks for AI-generated work).

Limitations

  • Not stated in the paper: the text does not include a limitations section, as is typical of a commentary.
  • Evident from the text: it is an opinion piece without primary data or systematic review, so its claims rest on secondary sources of varying rigor — including Twitter posts (refs 24, 25), a blog post, and a forum policy announcement.
  • Evident from the text: several reference access dates are internally inconsistent (e.g., "Accessed: February 24, 2022" for sources published in 2023), and the author's assertions about future AI capability ("can only be improved," "might soon be able to") are speculative rather than evidenced.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3325/cmj.2023.64.1

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 25 cited sources checked · 0 supported · 0 flagged

  • Premisecitation

    Algorithms perform relatively well on knowledge-based tests despite the lack of domain-specific training.

    “Algorithms perform relatively well on knowledge-based tests despite the lack of domain-specific training”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [5] Fijačko N Gosak L Štiglic G Picard CT John Douma M Can ChatGPT pass the life support exams without entering the American Heart Association Course? Resuscitation 2023 169 97 8 10.1016/j.resuscitation.2023.109732 36775020

      Not checked in the free run.

    • cites [6] Gilson A Safranek C Huang T Socrates V Chi L How does ChatGPT perform on the United States Medical Licensing Examination? The Implications of large language models for medical education and knowledge assessment. JMIR Med Educ 2023 9 e45312 10.2196/45312 36753318 PMC9947764

      Not checked in the free run.

    • cites [7] Kung TH Cheatham M Medenilla A Sillos C Leon LD Elepaño C Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLoS Digit Health. 2023 2 e0000198 10.1371/journal.pdig.0000198 36812645 PMC9931230

      Not checked in the free run.

  • Interpretationreasoning

    The tremendous potential of AI in healthcare does not lie in replacing physicians, but rather in increasing physicians' efficacy by redistributing workload and optimizing performance.

    “the tremendous potential of AI in healthcare does not lie in the possibility of replacing physicians, but rather in the capacity to increase physicians’ efficacy by redistributing workload and optimizing performance”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Context and nuance are critical for safe and effective patient care.
  • Premisecitation

    ChatGPT performed at or near the passing threshold on the United States Medical Licensing Exam.

    “performed at or near the passing threshold on the United States Medical Licensing Exam (6,7)”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [6] Gilson A Safranek C Huang T Socrates V Chi L How does ChatGPT perform on the United States Medical Licensing Examination? The Implications of large language models for medical education and knowledge assessment. JMIR Med Educ 2023 9 e45312 10.2196/45312 36753318 PMC9947764

      Not checked in the free run.

    • cites [7] Kung TH Cheatham M Medenilla A Sillos C Leon LD Elepaño C Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLoS Digit Health. 2023 2 e0000198 10.1371/journal.pdig.0000198 36812645 PMC9931230

      Not checked in the free run.

  • Interpretationreasoning

    A failure to acknowledge the limitations of conversational AI might pose an additional strain on the publishing system already flooded with meaningless data and low-quality manuscripts.

    “A failure to acknowledge the limitations of conversational AI might pose an additional strain on the publishing system already flooded with meaningless data and low-quality manuscripts”
    Quote in the paperNot a substantial claim
  • Premisecitation

    AI algorithms are notoriously bad at context and nuance.

    “they are notoriously bad at context and nuance (8)”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [8] Mbakwe AB Lourentzou I Celi LA Mechanic OJ Dagan A ChatGPT passing USMLE shines a spotlight on the flaws of medical education. PLoS Digit Health. 2023 2 e0000205 10.1371/journal.pdig.0000205 36812618 PMC9931307

      Not checked in the free run.

  • May combine two claims. 12
  • Classified as probably not a substantial claim. 2

  • May combine two claims.

    Premisecitation

    ChatGPT achieved approximately 66% on the Basic Life Support test and approximately 72% on the Advanced Cardiovascular Life Support test.

    “ChatGPT achieved ~ 66% and ~ 72% on Basic Life Support and Advanced Cardiovascular Life Support tests, respectively (5)”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [5] Fijačko N Gosak L Štiglic G Picard CT John Douma M Can ChatGPT pass the life support exams without entering the American Heart Association Course? Resuscitation 2023 169 97 8 10.1016/j.resuscitation.2023.109732 36775020

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    AI has tremendous potential to revolutionize health care and make it more efficient by improving diagnostics, detecting medical errors, and reducing the burden of paperwork.

    “AI has a tremendous potential to revolutionize health care and make it more efficient by improving diagnostics, detecting medical errors, and reducing the burden of paperwork (3,4)”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [3] Ahn JS Ebrahimian S McDermott S Lee L Naccarato JF Di Capua FY Association of artificial intelligence-aided chest radiograph interpretation with reader performance and efficiency. JAMA Netw Open ••• 5 e2229289 10.1001/jamanetworkopen.2022.29289 36044215 PMC9434361

      Not checked in the free run.

    • cites [4] Patel SB Lam K ChatGPT: the future of discharge summaries? Lancet Digit Health 2023 5 e145 6 36754724

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Frey and Osborne estimate that the probability of automating physicians' and surgeons' jobs is 0.42%, while the probability of automating administrative health care jobs is relatively high (e.g., 91% for health information technicians).

    “Frey and Osborne estimate that, while the probability of administrative health care jobs automation is relatively high (eg, 91% for health information technicians), the probability of automating the jobs of physicians and surgeons is 0.42% (9)”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [9] Frey CB Osborne MA The future of employment: How susceptible are jobs to computerisation? Technol Forecast Soc Change 2017 114 254 80 10.1016/j.techfore.2016.08.019

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The ability of ChatGPT to write text of surprising quality might deceive reviewers and readers, leading to an accumulation of dangerous misinformation.

    “the ability of ChatGPT to write a text of surprising quality might deceive reviewers and readers, with the final result being an accumulation of dangerous misinformation”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation
    § Opportunities and risks of ChatGPT in medicine, science, and academic publishing: a modern Promethean dilemma

    The quality of ChatGPT-generated content can only be improved with additional training and optimization.

    “the quality of the chatbot-generated content can only be improved with additional training and optimization”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [2] ChatGPT: Optimizing Language Models for Dialogue. Available from: https://openai.com/blog/chatgpt/. Accessed: February 24, 2022.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    StackOverflow banned the use of ChatGPT-generated text because the average rate of getting correct answers from ChatGPT is too low and posting such answers is substantially harmful.

    “StackOverflow, a popular forum for computer programming-related discussions, banned the use of ChatGPT-generated text “because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking and looking for correct answers“ (22)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] StackOverflow. 2022. Temporary policy: ChatGPT is banned. Available from: https://meta.stackoverflow.com/questions/421831/temporary-policy-chatgpt-is-banned. Accessed: February 25, 2023.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    ChatGPT can provide biased outputs and perpetuate sexist stereotypes.

    “ChatGPT can provide biased outputs and perpetuate sexist stereotypes (13)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] Chat GPT Friend or foe? Lancet Digit Health 2023 5 e1 36754723

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers, which is a challenging issue to fix.

    “the creators of ChatGPT openly disclosed that the fact that “ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers” a “challenging issue to fix“ (2)”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [2] ChatGPT: Optimizing Language Models for Dialogue. Available from: https://openai.com/blog/chatgpt/. Accessed: February 24, 2022.

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    In medicine, AI might dramatically improve efficacy by alleviating paperwork, and optimized chatbots might speed up and improve literature search.

    “In medicine, AI might dramatically improve efficacy just by alleviating a fragment of the suffocating paperwork (27), and optimized chatbots (eg, Stanford's BioMedLM) (28) might speed up and improve literature search”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [27] Siegler JE Patel NN Dine CJ Prioritizing paperwork over patient care: why can’t we do both? J Grad Med Educ 2015 7 16 8 10.4300/JGME-D-14-00494.1 26217415 PMC4507919

      Not checked in the free run.

    • cites [28] Bolton E, Hall D, Yasunaga M, Lee T, Manning C, Liang P. PubMedGPT 2.7B.158. Available from: https://crfm.stanford.edu/2022/12/15/pubmedgpt.html. Accessed: February 25, 2023.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    AI models are susceptible to availability, selection, and confirmation bias and are unreluctant to amplify it.

    “the models are not only susceptible to availability, selection, and confirmation bias but are also unreluctant to amplify it (12)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12] Rich AS Gureckis TM Lessons for artificial intelligence from the study of natural stupidity. Nat Mach Intell 2019 1 174 80 10.1038/s42256-019-0038-z

      Not checked in the free run.

    • Rests on an assumption: Current algorithms are often trained on biased data sets.

  • May combine two claims.

    Premisecitation

    ChatGPT is not a useful tool for writing reliable scientific texts without strong human intervention; it lacks the knowledge and expertise necessary to accurately convey complex scientific concepts.

    “Blanco-Gonzalez et al assessed the ability of ChatGPT to assist human authors in writing review articles and concluded that “…ChatGPT is not a useful tool for writing reliable scientific texts without strong human intervention. It lacks the knowledge and expertise necessary to accurately and adequately convey complex scientific concepts and information.” (23)”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [23] Blanco-Gonzalez A, Cabezon A, Seco-Gonzalez A, Conde-Torres D, Antelo-Riveiro P, Pineiro A, et al. The role of AI in drug discovery: challenges, opportunities, and strategies. 2022. arXiv:2212.08104. PMC10302890 37375838

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Conversational AIs are just language models trained to sound convincing, without the ability to interpret and understand content.

    “conversational AIs are just language models trained to sound convincing, but without the ability to interpret and understand the content”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    ChatGPT has already written essays, scholarly manuscripts, and computer code, summarized scientific literature, and performed statistical analyses.

    “ChatGPT already wrote essays, scholarly manuscripts, and computer code, summarized scientific literature, and performed statistical analyses (18,19)”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [18] van Dis EAM Bollen J Zuidema W van Rooij R Bockting CL ChatGPT: five priorities for research. Nature 2023 614 224 6 10.1038/d41586-023-00288-7 36737653

      Not checked in the free run.

    • cites [19] Liebrenz M Schleifer R Buadze A Bhugra D Smith A Generating scholarly content with ChatGPT: Ethical challenges for medical publishing. Lancet Digit Health 2023 5 e105 6 10.1016/S2589-7500(23)00019-5 36754725

      Not checked in the free run.

13 of 13

  • The paper relies on this without support.

    The implementation of conversational AI in medicine, science, and academic publishing is inevitable.

    “As the inevitable implementation of this disruptive technology will have far-reaching consequences for medicine, science, and academic publishing, we need to discuss both the opportunities and risks of its use.”

  • The paper relies on this without support.

    Conversational AIs are here to stay and will improve with additional training and optimization.

    “The powerful disruptive technology of conversational AIs is here to stay, and we can only expect them to improve with additional training and optimization.”

  • The paper relies on this without support.

    A chatbot cannot be held accountable for its work, and there is no legal framework to determine who owns the rights to AI-generated work.

    “A chatbot cannot be held accountable for its work, and there is no legal framework to determine who owns the rights to the AI-generated work – the author of the manuscript, the author of the AI, or the (unknown) authors who contributed training data?”

  • The paper relies on this without support.

    ChatGPT often fails to disclose the source of information.

    “since ChatGPT often fails to disclose the source of information, who is to blame for plagiarism if the chatbot decides to plagiarize?”

  • The paper relies on this without support.

    Current algorithms are often trained on biased data sets.

    “current algorithms are often trained on biased data sets”

  • The paper relies on this without support.

    Context and nuance are critical for safe and effective patient care.

    “two things critical for safe and effective patient care, which requires the implementation of medical knowledge, concepts, and principles in real-world settings”
6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • AI systems for clinical trial operations can parse biomarker requirements, prior therapies, and performance status from EHRs, identify potentially eligible patients, and flag candidates for screening.

    “Systems parse biomarker requirements, prior therapies, and performance status from EHRs, identify potentially eligible patients, and flag candidates for screening [56,57,58,59,60,61,62].”

    From Artificial Intelligence in Clinical Oncology: From Productivity Enhancement to Creative Discovery (2025) · cites this paper as [62]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Reference accessed before its stated publication year

    Minor

    Reference 1 is dated 2023, but its access date is February 24, 2022-a year before the stated publication year.

1 of 1

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Homolak, J.
Affiliations
University of Zagreb (HR)
Corresponding author
Homolak, Jan
ORCID
Homolak, Jan
Venue
Croatian Medical Journal
Publisher
Croatian Medical Journals
Published
2023-02
Volume
64
Issue
1
Pages
1-3
ISSN
  • 0353-9504 (ISSN-L)
  • 1332-8166
DOI
10.3325/cmj.2023.64.1
PMC
PMC10028563
OpenAlex
W4323035111
Semantic Scholar
49455e5f5ee94b43c13b039c9353b972ccf05286
Cited by
208
References
23
Citations by year
2026: 17 · 2025: 37 · 2024: 75 · 2023: 79
FWCI
8.28
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Croatian Medical Journal (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • University of Zagreb University Computing Centre (SRCE) (submittedVersion, other-oa)
Funders
Klinička Bolnica Dubrava
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Infectious Diseases › COVID-19 Clinical Research Studies
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
ChatGPT, large language models, artificial intelligence in healthcare, scholarly publishing, research integrity, CURB-65, 4C Mortality Score, 30-day mortality, risk stratification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review