CoChat
Sign Up
Cited by 213Artificial Intelligence in Healthcare and EducationCOVID-19 Clinical Research StudiesMachine Learning in Healthcare

The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index; none of the 25 sources its claims cite is checked for support yet.

Opportunities and risks of ChatGPT in medicine, science, and academic publishing: a modern Promethean dilemma

Open access
Homolak, Jan·2023·Croatian Medical Journal
DOI 10.3325/cmj.2023.64.1PMC PMC10028563
Open this paper in CoChat — free accountOpen PDF

Abstract

Aim To evaluate the predictive properties of several com - mon prognostic scores regarding survival outcomes in hospitalized COVID-19 patients. Methods We retrospectively reviewed the medical records of 4014 consecutive COVID-19 patients hospitalized in our tertiary level institution from March 2020 to March 2021. Prognostic properties of the WHO COVID-19 severity clas - sification, COVID-GRAM, Veterans Health Administration COVID-19 (VACO) Index, 4C Mortality Score, and CURB65 score regarding 30-day mortality, in-hospital mortality, presence of severe or critical disease on admission, need for an intensive care unit treatment, and mechanical venti - lation during hospitalization were evaluated. Results All of the investigated prognostic scores signifi - cantly distinguished between groups of patients with differ - ent 30-day mortality. The CURB-65 and 4C Mortality Score had the best prognostic properties for prediction of 30-day mortality (area under the curve [AUC] 0.761 for both) and in-hospital mortality (AUC 0.757 and 0.762, respectively). The 4C Mortality Score and COVID-GRAM best predicted the presence of severe or critical disease (AUC 0.785 and 0.717, respectively). In the multivariate analysis evaluating 30-day mortality, all scores mutually independently provid - ed additional prognostic information, except the VACO In - dex, whose prognostic properties were redundant. Conclusion Complex prognostic scores based on many parameters and comorbid conditions did not have better prognostic properties regarding survival outcomes than a simple CURB-65 prognostic score. CURB-65 also provides the largest number of prognostic categories (five), allowing more precise risk stratification than other prognostic scores.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This commentary argues that ChatGPT and similar conversational AI will not replace physicians but can substantially augment their work, while posing serious risks to scientific integrity and academic publishing. It calls for mindful, debated adoption rather than hasty implementation or outright bans.

Key findings

  • ChatGPT is a 175-billion-parameter model that allegedly gained more than 1 million users in its first few days and 100 million within 2 months, making it the fastest-growing consumer application in history.
  • On knowledge-based medical tests, ChatGPT scored ~66% on Basic Life Support and ~72% on Advanced Cardiovascular Life Support, and performed at or near the passing threshold on the United States Medical Licensing Exam; however, the author notes algorithms remain "notoriously bad at context and nuance."
  • Citing Frey and Osborne, the probability of automating physicians' and surgeons' jobs is 0.42%, versus 91% for health information technicians — supporting the claim that AI's value lies in redistributing workload, not replacing clinicians.
  • In one cited experiment, 50 ChatGPT-generated research abstracts passed a plagiarism checker, an AI-output detector, and human reviewers; the author warns the chatbot also fabricates plausible-sounding references, and StackOverflow banned its generated text because the average rate of correct answers is too low.

Methods

  • Design: a narrative commentary/opinion piece (no primary data collection or systematic methodology), structured around three questions — whether ChatGPT can replace physicians, whether it can act as a "chatbot-scientist," and what should happen next.
  • Evidence base: synthesis of cited literature, news reports, publisher policies, and social media posts, including studies of ChatGPT's exam performance, abstract generation, and review-article writing.
  • Main analysis: qualitative argumentation about opportunities (reduced paperwork, faster literature search, shifting research focus from presentation to content) and risks (bias amplification, fabricated sources, accountability and plagiarism gaps, lack of legal frameworks for AI-generated work).

Limitations

  • Not stated in the paper: the text does not include a limitations section, as is typical of a commentary.
  • Evident from the text: it is an opinion piece without primary data or systematic review, so its claims rest on secondary sources of varying rigor — including Twitter posts (refs 24, 25), a blog post, and a forum policy announcement.
  • Evident from the text: several reference access dates are internally inconsistent (e.g., "Accessed: February 24, 2022" for sources published in 2023), and the author's assertions about future AI capability ("can only be improved," "might soon be able to") are speculative rather than evidenced.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3325/cmj.2023.64.1

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 of 25 cited sources checked · 0 supported · 0 flagged

Show all 9 claims

  • It may state two separate claims as one. 12
  • Probably not a substantial claim. 2
13 of 13

  • The paper relies on this without support.

    The implementation of conversational AI in medicine, science, and academic publishing is inevitable.

    “As the inevitable implementation of this disruptive technology will have far-reaching consequences for medicine, science, and academic publishing, we need to discuss both the opportunities and risks of its use.”

  • The paper relies on this without support.

    Conversational AIs are here to stay and will improve with additional training and optimization.

    “The powerful disruptive technology of conversational AIs is here to stay, and we can only expect them to improve with additional training and optimization.”

  • The paper relies on this without support.

    A chatbot cannot be held accountable for its work, and there is no legal framework to determine who owns the rights to AI-generated work.

    “A chatbot cannot be held accountable for its work, and there is no legal framework to determine who owns the rights to the AI-generated work – the author of the manuscript, the author of the AI, or the (unknown) authors who contributed training data?”

  • The paper relies on this without support.

    ChatGPT often fails to disclose the source of information.

    “since ChatGPT often fails to disclose the source of information, who is to blame for plagiarism if the chatbot decides to plagiarize?”

  • The paper relies on this without support.

    Current algorithms are often trained on biased data sets.

    “current algorithms are often trained on biased data sets”

  • The paper relies on this without support.

    Context and nuance are critical for safe and effective patient care.

    “two things critical for safe and effective patient care, which requires the implementation of medical knowledge, concepts, and principles in real-world settings”
6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Reference accessed before its stated publication year

    Minor

    Reference 1 is dated 2023, but its access date is February 24, 2022-a year before the stated publication year.

1 of 1

Citations

Cites

Flagged first, then by citations

Show all 23 references

Cited by

Sorted by citations

Items 201–213 of 213

Related

See all

Metadata

Authors
Homolak, J.
Affiliations
University of Zagreb (HR)
Corresponding author
Homolak, Jan
ORCID
Homolak, Jan
Venue
Croatian Medical Journal
Publisher
Croatian Medical Journals
Published
2023-02
Volume
64
Issue
1
Pages
1-3
ISSN
  • 0353-9504 (ISSN-L)
  • 1332-8166
DOI
10.3325/cmj.2023.64.1
PMC
PMC10028563
OpenAlex
W4323035111
Semantic Scholar
49455e5f5ee94b43c13b039c9353b972ccf05286
Cited by
208
References
23
Citations by year
2026: 17 · 2025: 37 · 2024: 75 · 2023: 79
FWCI
8.28
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Croatian Medical Journal (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • University of Zagreb University Computing Centre (SRCE) (submittedVersion, other-oa)
Funders
Klinička Bolnica Dubrava
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Infectious Diseases › COVID-19 Clinical Research Studies
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
ChatGPT, large language models, artificial intelligence in healthcare, scholarly publishing, research integrity, CURB-65, 4C Mortality Score, 30-day mortality, risk stratification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review