CoChat
Sign Up
Cited by 1Artificial Intelligence in Healthcare and EducationClinical Reasoning and Diagnostic SkillsMachine Learning in Healthcare

Structural error asymmetry and harm-weighted analysis of ChatGPT versus ICU Physicians in acid–base interpretation: a prospective observational study

Open access
Gulen, Derful, Gözden, Hilmi Erdem, Ekin, Serpil, Ceylan, Ilkay·2026·Scientific Reports
DOI 10.1038/s41598-026-44576-4PMC PMC13179372
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) have demonstrated potential in clinical reasoning tasks; however, their performance in real-world intensive care unit (ICU) acid–base interpretation remains insufficiently characterized, particularly in complex and mixed disorders. Most existing evaluations rely primarily on aggregate accuracy metrics without examining structural error patterns or the clinical severity of misclassification. In high-acuity ICU settings, under-recognition of physiological complexity may carry disproportionate safety implications. In this prospective observational study, arterial blood gas (ABG) data from 50 consecutive ICU patients were interpreted independently by ICU physicians and ChatGPT using a standardized prompt. Interpretations were harmonized into six predefined diagnostic categories and compared with a final reference diagnosis established by a blinded expert panel. Agreement was assessed using Cohen’s kappa and diagnostic accuracy metrics. Additional analyses included complexity-stratified evaluation, mixed-disorder sensitivity, multi-label component-level (metabolic and respiratory) detection, false reassurance risk assessment, harm-weighted misclassification modeling, bootstrap confidence intervals, and post-hoc power analysis. Overall categorical accuracy was 82% for ICU physicians and 72% for ChatGPT. Agreement was substantial in both groups (κ = 0.73 vs. 0.63). Paired comparison did not demonstrate a statistically significant difference in overall classification ( p = 0.267), and post-hoc power analysis indicated limited ability to detect modest effect sizes (power = 0.22). However, stratified analyses revealed clinically meaningful structural differences. Sensitivity for mixed acid–base disorders was 0.96 for ICU physicians and 0.63 for ChatGPT. ChatGPT uniquely classified 16.7% of mixed cases as normal (false reassurance), whereas ICU physicians produced no false-normal classifications. Component-level analysis demonstrated lower respiratory component sensitivity for ChatGPT (0.88 vs. 1.00), contributing to under-recognition of physiological complexity. Harm-weighted misclassification modeling showed significantly higher clinical severity of errors for ChatGPT (mean difference 0.12; 95% CI 0.032–0.220; p = 0.026). While aggregate diagnostic agreement appeared broadly comparable, complexity-stratified and harm-weighted analyses demonstrated asymmetric error patterns with potential safety implications. These findings do not establish clinical equivalence and suggest that evaluation of AI diagnostic tools in critical care should extend beyond overall accuracy to incorporate safety-oriented and harm-weighted assessment frameworks.

AI snapshot

Written from the paper’s full text.

TL;DR

In a prospective single-center ICU cohort of 50 patients, ChatGPT and ICU physicians had broadly comparable overall acid–base diagnostic accuracy (72% vs. 82%), but stratified analyses showed asymmetric error patterns. ChatGPT missed mixed disorders more often and produced false-normal classifications, with significantly higher harm-weighted error scores.

Key findings

  • Overall categorical accuracy was 82% (41/50) for ICU physicians and 72% (36/50) for ChatGPT; agreement was substantial (κ = 0.73 vs. 0.63), and the paired difference was not statistically significant (p = 0.267; post-hoc power = 0.22).
  • Mixed acid–base disorder sensitivity was 0.96 (23/24) for ICU physicians vs. 0.63 (15/24) for ChatGPT; ChatGPT classified 16.7% (4/24) of mixed cases as normal, while ICU physicians produced 0/24 false-normal classifications.
  • Component-level sensitivity was lower for ChatGPT for respiratory components (0.88 vs. 1.00) and slightly lower for metabolic components (0.89 vs. 0.97).
  • Harm-weighted misclassification scores were higher for ChatGPT (mean 0.184 vs. 0.064; mean difference 0.12; 95% CI 0.032–0.220; p = 0.026).

Methods

  • Single-center prospective observational study in an adult ICU; 50 consecutive eligible patients enrolled August–September 2025, with the first qualifying ABG sample per patient included.
  • Each case was interpreted independently by ICU physicians and ChatGPT (GPT-4o via web interface with a standardized prompt), then compared with a final reference diagnosis from a blinded expert panel; diagnoses were harmonized into six predefined categories.
  • Analyses included Cohen’s kappa, McNemar’s test, complexity stratification, mixed-disorder sensitivity, multi-label metabolic/respiratory component detection, false reassurance assessment, harm-weighted misclassification modeling with Wilcoxon signed-rank testing and bootstrap confidence intervals, and post-hoc power analysis.

Limitations

  • The paper states that n = 50 limits precision of subgroup estimates, ChatGPT outputs lacked calibrated uncertainty/probabilistic confidence estimates, and the single-center design may limit generalizability.
  • The paper states that granular internal GPT model version identifiers and backend update logs were unavailable, and that the expert-panel reference standard may introduce subjectivity compared with a fully protocolized algorithmic standard.
  • The paper states that it evaluated single time-point case-based assessments and did not incorporate longitudinal physiological trends, treatment response, or escalation behavior; it also notes no adjustment for multiple comparisons because secondary analyses were exploratory.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41598-026-44576-4

Cites

14 of 14 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

5 of 11 cited sources checked · 0 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    Performance declines with increasing complexity.

    “This confirms prior observations that performance declines with increasing complexity3,9.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [3] Berend, Kenrick et al. “Physiological approach to assessment of acid-base disturbances. The New England Journal of Medicine, 371(15), 1434-45. https://doi.org/10.1056/NEJMra1003327. (2014) source
    • cites [9] Hirosawa, T. et al. ChatGPT-Generated Differential Diagnosis Lists for Complex Case-Derived Clinical Vignettes: Diagnostic Accuracy Evaluation. JMIR Med Inform. 11, e48808. https://doi.org/10.2196/48808 (2023). source

      In “Comparison With Prior Work” (page not specified), the paper reports lower diagnostic accuracy for ChatGPT-3.5 on its complex vignettes than in a prior study of more common presentations, and largely attributes the difference to case complexity. That supports the observation for ChatGPT-3.5, but not an unqualified decline in performance across models or settings. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
  • Findingown data

    Overall categorical accuracy was 82% for ICU physicians and 72% for ChatGPT.

    “Overall categorical accuracy was 82% for ICU physicians and 72% for ChatGPT.”
    Quote in the paperSubstantial claim
  • Findingown data

    Harm-weighted misclassification modeling showed significantly higher clinical severity of errors for ChatGPT (mean difference 0.12; 95% CI 0.032-0.220; p = 0.026).

    “Harm-weighted misclassification modeling showed significantly higher clinical severity of errors for ChatGPT (mean difference 0.12; 95% CI 0.032–0.220; p = 0.026).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Misclassification of a mixed acid–base disorder as normal is clinically high-severity due to potential implications for delayed escalation in critical care settings.
    • The authors state this as an assumption: The severity hierarchy used for harm weights—highest for false reassurance in mixed disorders, then mixed as single disorders, and lowest for intra-class substitutions—reflects clinical harm.
  • Findingown data

    ICU physicians produced no false-normal classifications of mixed acid-base disorder cases.

    “whereas ICU physicians produced no false-normal classifications.”
    Quote in the paperSubstantial claim
  • Findingown data

    Paired comparison did not demonstrate a statistically significant difference in overall classification between ICU physicians and ChatGPT (p = 0.267).

    “Paired comparison did not demonstrate a statistically significant difference in overall classification (p = 0.267)”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 20
  • May combine two claims. 20

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    ChatGPT can achieve high accuracy in classifying basic parameters such as pH and oxygenation status.

    “Studies have demonstrated that ChatGPT can achieve high accuracy in classifying basic parameters such as pH and oxygenation status8.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [8] Rao, A. et al. Assessing the Utility of ChatGPT Throughout the Entire Clinical Workflow: Development and Usability Study. Journal of medical Internet research, 25, e48659. (2023). source

      The Results (Overall Performance; page unavailable) report 71.8% accuracy across clinical vignettes, supporting a general claim about ChatGPT’s clinical-task accuracy. The paper does not report accuracy for classifying pH or oxygenation status specifically (searched: “pH,” “oxygenation,” and their combination). (LLM, conf=0.78)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    ChatGPT demonstrates strong performance on straightforward cases but reduced reliability in mixed acid-base disorders typical of critically ill ICU populations.

    “ChatGPT demonstrates strong performance on straightforward cases but reduced reliability in mixed acid–base disorders typical of critically ill ICU populations9.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [9] Hirosawa, T. et al. ChatGPT-Generated Differential Diagnosis Lists for Complex Case-Derived Clinical Vignettes: Diagnostic Accuracy Evaluation. JMIR Med Inform. 11, e48808. https://doi.org/10.2196/48808 (2023). source

      In “Comparison With Prior Work” (page not provided), the paper reports lower ChatGPT-3.5 diagnostic accuracy for complex general-internal-medicine vignettes than for more common clinical presentations. It does not examine mixed acid–base disorders or critically ill ICU patients, so the claim’s specific application to those cases is unsupported. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    AI performance frequently approaches but does not surpass specialist-level interpretation.

    “This aligns with broader meta-analytic observations that AI performance frequently approaches but does not surpass specialist-level interpretation5,10-12.”
    Quote in the paperNot a substantial claim2 of 4 sources checked
    • cites [5] Liu X, Faes L, Kale AU, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health. 2019 1(6), e271-e297. https://doi.org/10.1016/S2589-7500(19)30123-2 (2019). source
    • cites [10] Ouyang, David et al. “Electrocardiographic deep learning for predicting post-procedural mortality: a model development and validation study. The Lancet. Digital health 6(1), e70-e78. https://doi.org/10.1016/S2589-7500(23)00220-0 (2024). source

      The Results compare the ECG algorithm with a clinical risk score, and the Discussion says it outperformed human-interpretable ECG features. Neither section reports a meta-analysis or compares AI performance with specialist-level interpretation. Searches for specialist and expert interpretation found no passage establishing the claimed pattern. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
    • cites [11] Patel, S. B. & Lam, K. “ChatGPT: the future of discharge summaries?. The Lancet. Digital health. 5(3). e107-e108. https://doi.org/10.1016/S2589-7500(23)00021-3 (2023). source
    • cites [12] Nagendran, M. et al. Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. BMJ 368, m689 (2020). source

      The Results (Non-randomised studies; page not provided) report that 25 studies described AI as comparable to clinicians, supporting the idea that AI often approaches clinician performance. But 23 described it as superior, and the review cautions that these comparisons have substantial limitations. It does not establish that AI generally fails to surpass specialist-level interpretation. (LLM, conf=0.85)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data

    Agreement was substantial in both groups, with Cohen's kappa of 0.73 for ICU physicians and 0.63 for ChatGPT.

    “Agreement was substantial in both groups (κ = 0.73 vs. 0.63).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Final acid-base diagnoses were distributed as mixed acid-base disorders 48% (24/50), metabolic alkalosis 16% (8/50), respiratory alkalosis 12% (6/50), normal status 10% (5/50), respiratory acidosis 8% (4/50), and metabolic acidosis 6% (3/50).

    “After category harmonization into six standardized diagnostic groups, mixed acid–base disorders represented 48% of cases (24/50), metabolic alkalosis 16% (8/50), respiratory alkalosis 12% (6/50), normal status 10% (5/50), respiratory acidosis 8% (4/50), and metabolic acidosis 6% (3/50).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    ABG interpretation can be challenging and subject to human error, especially in time-pressured critical care scenarios.

    “However, ABG interpretation can be challenging and subject to human error, especially in time-pressured critical care scenarios2.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [2] Adrogué, H. J. & Madias, N. E. Secondary responses to altered acid-base status: the rules of engagement. Journal of the American Society of Nephrology: JASN, 21(6), 920-3. https://doi.org/10.1681/ASN.2009121211 (2010). source

  • May combine two claims.

    Interpretationown data

    Complexity-stratified and harm-weighted analyses demonstrated asymmetric error patterns between ChatGPT and ICU physicians with potential safety implications, despite broadly comparable aggregate diagnostic agreement.

    “While aggregate diagnostic agreement appeared broadly comparable, complexity-stratified and harm-weighted analyses demonstrated asymmetric error patterns with potential safety implications.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: In high-acuity ICU environments, the consequences of under-recognizing physiological complexity—particularly mixed disorders—may exceed those of simple categorical disagreement.
    • Rests on an assumption: Misclassification of a mixed acid–base disorder as normal is clinically high-severity due to potential implications for delayed escalation in critical care settings.
    • The authors state this as an assumption: The severity hierarchy used for harm weights—highest for false reassurance in mixed disorders, then mixed as single disorders, and lowest for intra-class substitutions—reflects clinical harm.

  • Classified as probably not a substantial claim.

    Premisecitation

    Variability in clinician expertise may lead to misinterpretation or delayed recognition of complex acid-base disturbances.

    “Variability in clinician expertise may lead to misinterpretation or delayed recognition of complex acid-base disturbances3.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3] Berend, Kenrick et al. “Physiological approach to assessment of acid-base disturbances. The New England Journal of Medicine, 371(15), 1434-45. https://doi.org/10.1056/NEJMra1003327. (2014) source

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The final reference diagnosis was established by a blinded expert panel.

    “compared with a final reference diagnosis established by a blinded expert panel.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Complexity-stratified analyses were pre-specified to evaluate differential performance in physiologically complex cases.

    “Complexity-stratified analyses were pre-specified to evaluate differential performance in physiologically complex cases.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    Diagnostic harmonization was performed prior to statistical analysis and blinded to reference standard outcomes.

    “This harmonization step was performed prior to statistical analysis and blinded to reference standard outcomes.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    ChatGPT outputs did not provide calibrated uncertainty or probabilistic confidence estimates, precluding formal calibration analysis.

    “Second, ChatGPT outputs did not provide calibrated uncertainty or probabilistic confidence estimates, precluding formal calibration analysis.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    The sample size was determined by the number of consecutive eligible ICU admissions during the predefined study period rather than by a priori power calculation; no formal sample size calculation was performed before enrollment.

    “The sample size was determined by the number of consecutive eligible ICU admissions during the predefined study period rather than by a priori power calculation.”
    • Also stated in: § Study population and patient inclusion criteria · plain
      “Given the exploratory and feasibility-oriented nature of the study, no formal sample size calculation was performed before enrollment.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The results do not suggest universal inferiority; rather, they indicate differential strengths depending on case complexity.

    “At the same time, the results do not suggest universal inferiority; rather, they indicate differential strengths depending on case complexity.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Single-center design may limit generalizability.

    “Third, single-center design may limit generalizability.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The final reference diagnosis was established by an expert panel using case-based vignette review; reference standard adjudication may still introduce subjectivity compared with a fully protocolized algorithmic standard.

    “Fourth, the final reference diagnosis was established by an expert panel using case-based vignette review; although panelists were not involved in bedside care of the included patients and did not have access to longitudinal clinical trajectories or outcomes, reference standard adjudication may still introduce subjectivity compared with a fully protocolized algorithmic standard.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Only the first qualifying ABG sample per patient was included to avoid intra-individual correlation.

    “Only the first qualifying ABG sample per patient was included to avoid intra-individual correlation.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    All clinical inputs were provided to ChatGPT in English using a fixed prompt and standardized structure, and the prompt and variable set were identical for all cases and were not modified during the study period.

    “All clinical inputs were provided to ChatGPT in English using a fixed prompt and standardized structure. The prompt and variable set were identical for all cases and were not modified during the study period.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationown data

    The high proportion of mixed disorders reflects the physiological complexity typical of critically ill ICU populations and provides a clinically relevant test environment for diagnostic decomposition analysis.

    “The high proportion of mixed disorders reflects the physiological complexity typical of critically ill ICU populations and provides a clinically relevant test environment for diagnostic decomposition analysis.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductnone given

    All cases were derived from real ICU patients and prospectively collected during routine clinical care; no simulated or hypothetical cases were used.

    “All cases were derived from real ICU patients and prospectively collected during routine clinical care; no simulated or hypothetical cases were used.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    ChatGPT access used GPT-4o via the standard web interface without API-based customization, and no parameter tuning, system-level configuration, or temperature modification was performed.

    “During the study period (August–September 2025), GPT-4o, a GPT-4–class large language model, was utilized. Model access occurred through the standard web interface without API-based customization. Exact internal version identifiers were not accessible to the investigators. No parameter tuning, system-level configuration, or temperature modification was performed.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The study utilized a GPT-4-based model accessible between August and September 2025; granular internal version identifiers and backend update logs were not available to investigators.

    “The study utilized a GPT-4–based model accessible between August and September 2025; however, granular internal version identifiers and backend update logs were not available to investigators.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    Severity-of-illness scores (APACHE II and SOFA) were recorded for descriptive and exploratory analyses but were not provided to ChatGPT.

    “Severity-of-illness scores (APACHE II and SOFA) were recorded for descriptive and exploratory analyses but were not provided to ChatGPT.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    All interpretations were performed independently, and evaluators were blinded to each other's assessments.

    “All interpretations were performed independently, and evaluators were blinded to each other’s assessments.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Reduced sensitivity for mixed acid-base disorders and the presence of false reassurance classifications highlight structural limitations in physiological decomposition under high-complexity conditions.

    “Specifically, reduced sensitivity for mixed acid–base disorders and the presence of false reassurance classifications highlight structural limitations in physiological decomposition under high-complexity conditions.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    No adjustment for multiple comparisons was applied, as secondary analyses were exploratory and hypothesis-generating.

    “No adjustment for multiple comparisons was applied, as secondary analyses were exploratory and hypothesis-generating.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The discrepancy between ICU physicians and ChatGPT was not limited to reduced sensitivity but included a distinct false reassurance pattern.

    “Importantly, the discrepancy was not limited to reduced sensitivity but included a distinct false reassurance pattern: ChatGPT classified 16.7% of mixed disorders as normal, whereas ICU physicians produced no false-normal classifications.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductcitation

    This study evaluated single time-point case-based assessments and did not incorporate longitudinal physiological trends, treatment response, or escalation behavior.

    “This study evaluated single time-point case-based assessments and did not incorporate longitudinal physiological trends, treatment response, or escalation behavior7.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [7] Komorowski, M. et al. Komorowski M, et al. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med. 24, 1716–1720 (2018). source
28 of 28

  • The paper relies on this without support.

    Most prior comparisons have relied primarily on aggregate accuracy or agreement metrics, without examining structural error patterns, diagnostic decomposition, or clinical severity of misclassification.

    “Importantly, most prior comparisons have relied primarily on aggregate accuracy or agreement metrics, without examining structural error patterns, diagnostic decomposition, or the clinical severity of misclassification.”
    “Most existing evaluations rely primarily on aggregate accuracy metrics without examining structural error patterns or the clinical severity of misclassification.”

  • The paper relies on this without support.

    In high-acuity ICU environments, the consequences of under-recognizing physiological complexity—particularly mixed disorders—may exceed those of simple categorical disagreement.

    “In high-acuity ICU environments, the consequences of under-recognizing physiological complexity—particularly mixed disorders—may exceed those of simple categorical disagreement.”
    “In high-acuity ICU settings, under-recognition of physiological complexity may carry disproportionate safety implications.”

  • The paper relies on this without support.

    Misclassification of a mixed acid–base disorder as normal is clinically high-severity due to potential implications for delayed escalation in critical care settings.

    “This misclassification was considered clinically high-severity due to potential implications for delayed escalation in critical care settings.”

  • The authors state this as an assumption.

    The severity hierarchy used for harm weights—highest for false reassurance in mixed disorders, then mixed as single disorders, and lowest for intra-class substitutions—reflects clinical harm.

    “Predefined harm weights were assigned to each misclassification type based on a severity hierarchy, with highest weight assigned to false reassurance in mixed disorders, followed by misclassification of mixed as single (primary) disorders, and lowest weight assigned to intra-class substitutions.”

  • The paper relies on this without support.

    Case complexity cannot be prospectively identified at the bedside.

    “Because case complexity cannot be prospectively identified at the bedside and because current general-purpose large language models do not provide calibrated uncertainty signaling or escalation mechanisms, their outputs should not be considered interchangeable with physician-level interpretation in ICU settings.”

  • The paper relies on this without support.

    Current general-purpose large language models do not provide escalation mechanisms.

    “Because case complexity cannot be prospectively identified at the bedside and because current general-purpose large language models do not provide calibrated uncertainty signaling or escalation mechanisms, their outputs should not be considered interchangeable with physician-level interpretation in ICU settings.”

  • The paper relies on this without support.

    Early recognition of evolving deterioration and worst-case scenario assessment are essential to patient safety in critical care.

    “In critical care environments, where early recognition of evolving deterioration and worst-case scenario assessment are essential to patient safety, such asymmetries in error patterns warrant careful consideration.”
7 of 7

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Gulen, D., Gözden, H. E., Ekin, S., & Ceylan, I.
Affiliations
Sağlık Bilimleri Üniversitesi (TR)
ORCID
Ekin, Serpil
Venue
Scientific Reports
Publisher
Springer Science and Business Media LLC
Published
2026-03-27
Volume
16
Issue
1
ISSN
2045-2322 (ISSN-L)
DOI
10.1038/s41598-026-44576-4
PMC
PMC13179372
OpenAlex
W7141203043
Semantic Scholar
46fe0715844db25806fb0b1ada23cc9862b530e9
Cited by
1
References
14
Citations by year
2026: 1
FWCI
2.15
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • PubMed Central (submittedVersion, cc-by)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
ChatGPT, large language models, arterial blood gases, intensive care unit, mixed acid-base disorders, false reassurance, clinical reasoning
MeSH terms
Large Language Models, Acid-Base Equilibrium, Blood Gas Analysis, Female, Humans, Intensive Care Units, Middle Aged, Physicians, Prospective Studies

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review