CoChat
Sign Up
Cited by 2Artificial Intelligence in Healthcare and EducationMachine Learning in HealthcareClinical Reasoning and Diagnostic Skills

Structured reasoning failures compromise LLM interpretation of clinical oncology notes

Open access
Kenaston, Matthew W., Ayub, Umair, Parmar, Mihir, Kumar, Priya, Anjum, Muhammad Umair, Naqvi, Syed Arsalan Ahmed, Chaudhuri, Aadel A., Zakharia, Yousef +8 more·2026·npj Digital Medicine
DOI 10.1038/s41746-026-02951-5
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) show strong performance on clinical benchmarks, yet their reasoning reliability in real-world oncology care remains unclear. We evaluated LLM reasoning on authentic oncology notes using a novel hierarchical error taxonomy across two retrospective cohorts spanning breast, pancreatic, and prostate cancer. GPT-4 produced reasoning errors in 23.1% of note interpretations, the majority reflecting cognitive bias patterns. Errors were more frequent in recommendation tasks and were strongly associated with guideline-discordant recommendations and lower clinician-rated clinical impact scores. Confirmation bias, anchoring bias, and omission errors were most strongly linked to potentially harmful outputs. Compared to GPT-4, GPT-5.1 demonstrated reduced error rates and improved clinical performance but retained structured reasoning failure patterns. Automated LLM-based evaluators detected error presence but failed to reliably classify subtypes, and a preliminary self-mitigation strategy yielded only modest improvement. Endpoint accuracy alone may mask clinically meaningful reasoning failures. Therefore, evaluating and monitoring reasoning fidelity should be a prerequisite for safe deployment of LLMs in oncology decision support.

AI snapshot

Written from the paper’s full text.

TL;DR

This study evaluated LLM reasoning on authentic oncology notes using a hierarchical error taxonomy across breast, pancreatic, and prostate cancer cohorts. GPT-4 showed frequent reasoning errors linked to guideline-discordant and lower-impact recommendations, while GPT-5.1 reduced but did not eliminate structured reasoning failures.

Key findings

  • In the CORAL development set, GPT-4-32k had an overall error rate of 27.4% (24.9–29.9) across 600 chain-of-thought responses from 40 oncology notes, with higher frequency in breast cancer (32.1%, 28.5–35.9) than pancreatic cancer (22.7%, 19.5–26.2). In prostate cancer validation, error rates were 17.0% (14.1–20.2) for evaluation, 22.4% (19.2–25.9) for management, and 22.3% (18.8–26.2) for presentation.
  • Among erroneous annotations, reasoning failures dominated (86.4%, 82.2–89.7); the most common Tier 2 subtypes were False Association (40.6%, 35.4–46.0) and Missing Step (45.5%, 40.2–50.8). The discussion reports model-elicited reasoning accounted for 85.4% of GPT-4 errors and 73.2% of GPT-5 errors.
  • Recommendations containing reasoning errors received markedly lower clinical-impact scores (p < 0.0001) and were more likely to deviate from NCCN guidelines. Confirmation bias, anchoring bias, and stark omission were consistently linked to the lowest impact scores (Likert = 1).
  • GPT-5.1 reduced error rates and improved NCCN guideline concordance overall (p = 0.00019), with guideline references in answers at 4.3% vs. 0% and in reasoning chains at 15.0% vs. 2.1% compared with GPT-4. Confirmation bias fell from 23.4% to 4.2%, while anchoring bias rose from 12.5% to 20.3%. Automated evaluators detected error presence but poorly classified subtypes; self-mitigation reduced mean error rate from 15.5% (12.7–18.7) to 13.5% (10.9–16.6), resolving 19 errors and introducing 6 (McNemar p = 0.0164).

Methods

  • Staged retrospective evaluation using the CORAL dataset of breast and pancreatic cancer notes (40 expert-labeled notes plus 100 deidentified notes per cancer type) and a Mayo Clinic prostate cancer cohort of 24 cases from 2021–2024 spanning localized disease, biochemical recurrence, mCSPC, and mCRPC.
  • GPT-4-32k and GPT-5.1 generated zero-shot chain-of-thought responses for extraction, analysis, and recommendation tasks stratified by presentation, evaluation, and management phases, with temperature 0. Annotators applied a three-tier error taxonomy; recommendations were rated on a 5-point clinical impact scale and assessed for NCCN v2.2024 guideline concordance.
  • Analyses included Cohen’s kappa, Wilson 95% confidence intervals, multivariable logistic regression, semantic similarity (normalized Levenshtein distance and TF-IDF cosine similarity), automated evaluators (Gemini-2.0-Flash, GPT-4o, Claude-3.5-Sonnet), and a preliminary self-mitigation prompt, conducted in R v4.3.1.

Limitations

  • Stated: this was a retrospective, note-based simulation conducted upstream of actual clinical care in small cohorts; the prostate validation cohort was underpowered for stage-stratified analyses, and the CORAL development set was not intended for precise prevalence estimation.
  • Stated: the clinician reasoning comparison was limited to a single oncologist and a small set of responses, underpowered for detecting statistically significant differences and potentially not capturing inter-clinician variability; findings focused on GPT-4 and GPT-5 may not fully generalize across other model families.
  • Stated/evident: zero-shot prompting likely overestimates error prevalence relative to structured clinical deployments; overlapping annotators may introduce bias; automated evaluation and self-mitigation results are exploratory with small subsets and no systematic evaluator configurations, and rationale-level analyses may reflect verbalized reasoning rather than internal LLM decision-making.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41746-026-02951-5

Cites

0 citation errors · 1 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 22 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    GPT-5.1 retained structured reasoning failure patterns.

    “Compared to GPT-4, GPT-5.1 demonstrated reduced error rates and improved clinical performance but retained structured reasoning failure patterns.”
    Quote in the paperSubstantial claim
  • Findingown data

    Direct comparison of paired responses between GPT-4 and GPT-5 revealed substantial semantic divergence between models across tasks.

    “Direct comparison of paired responses between GPT-4 and GPT-5 revealed substantial semantic divergence between models across tasks (Figure 5A-C).”
    Quote in the paperSubstantial claim
  • Findingown data

    Recommendations containing reasoning errors received markedly lower clinical-impact scores (p < 0.0001).

    “Recommendations containing reasoning errors received markedly lower clinical-impact scores (p < 0.0001) (Figure 4B).”
    Quote in the paperSubstantial claim
  • Findingown data

    Automated LLM-based evaluators failed to reliably classify error subtypes.

    “Automated LLM-based evaluators detected error presence but failed to reliably classify subtypes, and a preliminary self-mitigation strategy yielded only modest improvement.”
    Quote in the paperSubstantial claim
  • Findingown data

    Among erroneous annotations, reasoning failures dominated (86.4%, 82.2-89.7).

    “Among erroneous annotations, reasoning failures dominated (86.4%, 82.2-89.7),while comprehension errors were conversely rarer (Figure 2C).”
    Quote in the paperSubstantial claim
  • May combine two claims. 55
  • Classified as probably not a substantial claim. 29

  • May combine two claims.

    Findingown data

    Assumed risk was greatest for management-phase prompts and metastatic castration-resistant disease.

    “Assumed risk was greatest for managementphase prompts and metastatic castration-resistant disease.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    GPT-4 exhibited error rates of 17.0% (14.1-20.2) in evaluation, 22.4% (19.2-25.9) in management, and 22.3% (18.8-26.2) in presentation in prostate cancer.

    “GPT-4 exhibited error rates of 17.0% (14.1-20.2) in evaluation, 22.4% (19.2-25.9) in management, and 22.3% (18.8-26.2) in presentation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Error frequency did not vary by chart note or its file size (p = 0.736).

    “Error frequency did not vary by chart note or its file size (p = 0.736).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Redundancy increased in GPT-5 (13.1% vs. 5.0%), consistent with its longer, more verbose output style.

    “Redundancy also increased in GPT-5 (13.1% vs. 5.0%), consistent with its longer, more verbose output style in final answer outputs (Table 1).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    All clinician errors were reasoning failures, including three false associations and one missing reasoning step.

    “All clinician errors were reasoning failures, including three false associations and one missing reasoning step (Supplementary Figure 4D).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    At Tier 3, confirmation bias was the most prevalent subtype in GPT-4 (23.4%) and was markedly reduced in GPT-5 (4.2%).

    “At Tier 3, confirmation bias-the most prevalent subtype in GPT-4 (23.4%)-was markedly reduced in GPT-5 (4.2%), while anchoring bias increased (20.3% vs. 12.5%).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Among all errors detected within this dataset, Tier 1 comprehension errors were rare and carried limited clinical consequence.

    “Among all errors detected within this dataset, Tier 1 comprehension errors were rare and carried limited clinical consequence (Figure 4E).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In Tier 2, the most common error subtypes were False Association (40.6%, 35.4-46.0) and Missing Step (45.5%, 40.2-50.8).

    “In Tier 2, the most common error subtypes were False Association (40.6%, 35.4-46.0)and Missing Step (45.5%, 40.2-50.8),as both arise from Tier 1 Reasoning errors.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Tier 3 errors were more diverse, with Confirmation Bias, Factual Error, use of Chart Context, and Stark Omission representing the most frequent subtypes.

    “Tier 3 errors were more diverse, with Confirmation Bias, Factual Error, use of Chart Context, and Stark Omission representing the most frequent subtypes.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Queries regarding CSPC had higher odds of error versus localized disease, though this was relatively minimal.

    “Evaluation-phase contexts carried lower error risk than presentation (OR 0.52 [0.31-0.88]),and queries regarding CSPC had higher odds of error versus localized disease, though this was relatively minimal.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Certain reasoning errors were associated with lower clinician-rated clinical impact scores and greater guideline discordance.

    “Certain reasoning errors were associated with lower clinician-rated clinical impact scores and greater guideline discordance.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In recommendation tasks, all 33 clinician recommendations were NCCN guideline-concordant, whereas GPT-5 produced four discordant or alternative recommendations (Fisher p = 0.114).

    “In recommendation tasks, all 33 clinician recommendations were NCCN guideline-concordant, whereas GPT-5 produced four discordant or alternative recommendations (Fisher p = 0.114; Supplementary Figure 4F).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Recommendation tasks did not differ significantly from extraction in error risk, albeit more likely to result in reasoning errors for clinical management.

    “Analysis tasks were also less error-prone than extraction, while recommendation tasks did not differ significantly, albeit more likely to result in reasoning errors for clinical management (Figure 3B).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    All clinician errors occurred in cases where GPT-5 also erred, suggesting shared case difficulty.

    “Notably, all clinician errors occurred in cases where GPT-5 also erred, suggesting shared case difficulty.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Confirmation bias was highest for analysis tasks, while premature closure and anchoring bias were highest for recommendation tasks.

    “Confirmation bias was highest for analysis tasks, while premature closure and anchoring bias were highest for recommendation tasks.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Stage-stratified error rates showed a similar trend, with GPT-5 improving marginally across most disease stages and significantly in CSPC.

    “Stage-stratified error rates showed a similar trend, with GPT-5 improving marginally across most disease stages and significantly in CSPC (Figure 5F).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    67.2% of clinician reasoning chains were classified as a significant semantic shift (cosine similarity ≤0.20) whereas 32.8% were rewrite-only.

    “67.2% of clinician reasoning chains were classified as a significant semantic shift (cosine similarity ≤0.20) whereas 32.8% were rewrite-only (Supplementary Figure 4A-B).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Clinician responses contained fewer errors than GPT-5 (6.6% [4/61] vs. 16.4% [10/61]), though this difference was not statistically significant (McNemar p = 0.077).

    “Clinician responses contained fewer errors than GPT-5 (6.6% [4/61] vs. 16.4% [10/61]), though this difference was not statistically significant (McNemar p = 0.077; Supplementary Figure 4C).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Error rates differed slightly by cancer stage, though all within overlapping confidence intervals.

    “Error rates differed slightly by cancer stage (Figure 3C), though all within overlapping confidence intervals.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The tier-based approach performed best for Gemini (P = 0.35) and the error-based approach achieved higher true mistake accuracy in Claude (P = 0.44 vs. P=0.26).

    “Accuracy on true mistakes was consistently poor (tier-based: P=0.25-0.35,error-based: P=0.28-0.44; Figure 6B), with the tierbased approach performing best for Gemini (P = 0.35) and the error-based approach achieving higher true mistake accuracy in Claude (P = 0.44 vs. P=0.26).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The McNemar test p-value was 0.0164, though the absolute effect size remained modest and may not be clinically meaningful in this limited exploratory subset.

    “( McNemar p = 0.0164), though the absolute effect size remained modest and may not be clinically meaningful in this limited exploratory subset.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    When applied to GPT-5 outputs, the taxonomy demonstrated high reproducibility while revealing a lower overall frequency of reasoning failures and a shift in error composition, with marked reductions in confirmation bias and factual errors.

    “When applied to GPT-5 outputs, the taxonomy demonstrated high reproducibility while revealing a lower overall frequency of reasoning failures and a shift in error composition, with marked reductions in confirmation bias and factual errors.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Similar trends were observed in extraction and analysis tasks.

    “Similar trends were observed in extraction and analysis tasks.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Mitigation resolved more baseline errors than it introduced; among erroneous responses, 19 (33% of errors) were resolved, whereas 6 new errors were introduced.

    “Of note, mitigation resolved more baseline errors than it introduced.Among erroneous responses, 19 (33% of errors) were resolved, whereas 6 new errors were introduced (Figure 7C).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Our reproducible hierarchical taxonomy reveals model-elicited reasoning accounted for 85.4% of errors in GPT-4 and 73.2% in GPT-5.

    “Our reproducible hierarchical taxonomy reveals model-elicited reasoning accounted for 85.4% of errors in GPT-4 and 73.2% in GPT-5.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Claude showed higher false positive rates under the error-based approach, suggesting a tendency toward over-classification.

    “Claude showed the opposite, with higher false positive rates under the error-based approach, suggesting a tendency toward over-classification (Figure 6C).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    Anchoring bias was most prevalent in recommendation tasks, reflecting overweighting of initial clinical cues.

    “Anchoring bias was most prevalent in recommendation tasks, reflecting overweighting of initial clinical cues.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    GPT-5 recommendations marginally improved upon this, and more often referenced specific NCCN pathways.

    “GPT-5 recommendations marginally improved upon this, and more often referenced specific NCCN pathways.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Error rates appeared higher in advanced disease states, though these stage-stratified comparisons are exploratory and underpowered for definitive inference.

    “Error rates appeared higher in advanced disease states, though these stage-stratified comparisons are exploratory and underpowered for definitive inference.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Confirmation bias was the only subtype that persisted following GPT-5 self-mitigation and appeared in the clinician subset.

    “However, confirmation bias-the only subtype that persisted following GPT-5 selfmitigation and appeared in the clinician subset-occurred when an existing management framework (e.g., active surveillance or prior staging assumptions) suppressed retrieval of escalation-relevant evidence in our case-series (Supplementary Figure 3).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    When responses sharing the same baseline error were compared in paired analyses, a large portion were resolved following mitigation, but errors related to redundancy and factual inaccuracies persisted more frequently.

    “When responses sharing the same baseline error were compared in paired analyses, a large portion were resolved following mitigation, but errors related to redundancy and factual inaccuracies persisted more frequently (Figure 7D, right).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For error subtypes, improvement was consistent but most pronounced in missing-step errors, particularly premature closure.

    “For error subtypes, improvement was consistent but most pronounced in missing-step errors, particularly premature closure (Figure 7D, left).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    GPT-4 elicited reasoning failures correlated with guideline-discordant and potentially harmful recommendations, albeit GPT-5 demonstrated a significant error reduction under identical conditions.

    “GPT-4 elicited reasoning failures correlated with guideline-discordant and potentially harmful recommendations, albeit GPT-5 demonstrated a significant error reduction under identical conditions.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The taxonomy was validated on multiple common solid tumor types with high inter-rater reliability.

    “multiple common solid tumor types with high inter-rater reliability”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Although not statistically significant, the consistent direction of findings suggests greater reasoning fidelity and a safer clinical baseline in oncologist responses within this small subset.

    “Although not statistically significant, the consistent direction of these findings suggests greater reasoning fidelity and a safer clinical baseline in oncologist responses within this small subset.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationown data

    Real-world clinical tasks elicit different cognitive pressures, with decision-making under uncertainty particularly prone to shortcut reasoning.

    “These patterns mirror how real-world clinical tasks elicit different cognitive pressures, with decision-making under uncertainty particularly prone to shortcut reasoning.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    Inter-reviewer agreement was strong for both clinical impact and NCCN guideline concordance metrics.

    “Inter-reviewer agreement was strong for both metrics (Figure 4A).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    Inter-reviewer agreement remained high across all tiers (κ = 0.85-0.89), confirming reproducibility of the framework.

    “Inter-reviewer agreement remained high across all tiers (κ = 0.85-0.89),confirming reproducibility of the framework (Figure 2A, Supplementary Figure 2).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    While a newer architecture reduced the prevalence of several bias subtypes, it exhibited a distinct error profile rather than uniformly diminishing all error types.

    “These findings indicate that while a newer architecture reduced the prevalence of several bias subtypes, it exhibited a distinct error profile rather than uniformly diminishing all error types.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    GPT-4 produced consistent, clinically interpretable reasoning failures even during extraction tasks, supporting the taxonomy's validity.

    “Collectively, these findings demonstrate that even during extraction tasks, GPT-4 produced consistent, clinically interpretable reasoning failures, supporting the taxonomy's validity for identifying error patterns.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning

    The clinician reasoning comparison was limited to a single oncologist and a small set of responses, which was underpowered for detecting statistically significant differences and may not capture the full range of inter-clinician variability.

    “The clinician reasoning comparison was limited to a single oncologist and a small set of responses, which was underpowered for detecting statistically significant differences and may not capture the full range of inter-clinician variability.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisecitation

    Not all reasoning failures carry equal clinical risk, and automated systems alone are often insufficient for clinical deployment.

    “Nonetheless, this remains relevant since not all reasoning failures carry equal clinical risk 17,62 , and automated systems alone are often insufficient for clinical deployment.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [17]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [62]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The study follows TRIPOD-LLM reporting standards.

    “The study follows TRIPOD-LLM reporting standards 67 .”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [67]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationcitation

    Such vulnerabilities may be amplified in rarer malignancies, as LLMs appear to struggle with heterogeneous and rapidly evolving clinical pathways in rarer diseases.

    “Such vulnerabilities may be amplified in rarer malignancies, as LLMs appear to struggle with heterogeneous and rapidly evolving clinical pathways in rarer diseases [40][41][42] .”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40][41][42]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Discrepant labels were adjudicated via consensus review to reach a gold standard.

    “Discrepant labels were adjudicated via consensus review to reach a gold standard.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Absolute error rates and subgroup differences may vary across broader clinical contexts.

    “Thus, absolute error rates and subgroup differences may vary across broader clinical contexts.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Cohen's kappa was calculated across tiers to assess inter-rater reliability.

    “Cohen's kappa was calculated across tiers to assess inter-rater reliability.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductnone given

    The prostate validation cohort was underpowered for stage-stratified analyses, and the CORAL development set was not intended for precise prevalence estimation.

    “The prostate validation cohort was underpowered for stage-stratified analyses, and the CORAL development set was not intended for precise prevalence estimation.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Because we did not systematically explore evaluator configurations like in-context calibration or evaluator ensembling, we only describe initial feasibility and results should be interpreted as exploratory, as our small subset limits precision of effect estimates and increases sensitivity to case-specific variability.

    “Because we did not systematically explore evaluator configurations like in-context calibration or evaluator ensembling [58][59][60][61] , we only describe initial feasibility and results should be interpreted as exploratory, as our small subset limits precision of effect estimates and increases sensitivity to case-specific variability.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [58][59][60][61]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    Modest improvements from self-mitigation could reflect how such mitigation targets context-specific error patterns observed in elicited reasoning chains, which may not alter the LLM's internal reasoning for more persistent errors.

    “Modest improvements could reflect how such mitigation targets context-specific error patterns observed in elicited reasoning chains, which may not alter the LLM's internal reasoning for more persistent errors.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    This was a retrospective, note-based simulation conducted upstream of actual clinical care in small cohorts.

    “For study design, this was a retrospective, note-based simulation conducted upstream of actual clinical care in small cohorts.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    By mapping computational failures onto established cognitive bias frameworks, the taxonomy bridges computer science error analysis.

    “By mapping computational failures onto established cognitive bias frameworks, our taxonomy bridges computer science error analysis”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Each clinician-generated reasoning chain was graded by an independent reviewer using the same three-tier error taxonomy, five-point clinical impact scale, and NCCN concordance classification applied to LLM elicited reasoning chains.

    “Each clinician-generated reasoning chain was converted to stepwise format and graded by an independent reviewer using the same three-tier error taxonomy, five-point clinical impact scale, and NCCN concordance classification applied to LLM elicited reasoning chains.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Even with GPT-5's improvements, notable reasoning failures persist and remain clinically meaningful.

    “Even with GPT-5's improvements, notable reasoning failures persist and remain clinically meaningful.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    Zero-shot prompting to approximate clinician-style queries likely overestimates error prevalence relative to structured clinical deployments.

    “Regarding generation strategy, we applied zero-shot prompting to approximate clinician-style queries, which likely overestimates error prevalence relative to structured clinical deployments.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    A board-certified oncologist specializing in prostate cancer independently generated stepwise reasoning chains for a subset of 61 question-note pairs spanning four prostate cancer patients across all disease stages.

    “A board-certified oncologist, specializing in prostate cancer, independently generated stepwise reasoning chains for a subset of 61 question-note pairs spanning four prostate cancer patients across all disease stages (localized, biochemical recurrence, mCSPC, mCRPC).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The clinician responded to identical prompts using only the source oncology notes, blinded from LLM responses and from the assessment and plan.

    “They responded to identical prompts using only the source oncology notes, blinded from LLM responses and from the assessment and plan.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Although inter-rater reliability was high, overlapping annotators across study tasks may introduce bias.

    “Finally, although inter-rater reliability was high, overlapping annotators across study tasks may introduce bias.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given

    Contemporary implementations increasingly incorporate retrieval-augmented generation (RAG), structured outputs, and oversight mechanisms that may reduce factual errors, though such approaches do not inherently resolve flawed reasoning patterns.

    “Contemporary implementations increasingly incorporate retrieval-augmented generation (RAG), structured outputs, and oversight mechanisms that may reduce factual errors, though such approaches do not inherently resolve flawed reasoning patterns.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Findings may not fully generalize across other model families because we focused on GPT-4 and GPT-5.

    “In terms of model evaluation, we focused on GPT-4 and GPT-5 to enable in-depth characterization of widely used architectures, but findings may not fully generalize across other model families.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Explainable AI literature has long emphasized how post hoc explanations may reflect alignment objectives or stylistic conventions rather than underlying mechanisms.

    “Explainable AI literature has long emphasized how post hoc explanations may reflect alignment objectives or stylistic conventions rather than underlying mechanisms 48,49 .”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [48]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [49]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    EHR notes are nonlinear documents where contextual cues, provisional impressions, and objective data coexist in complex relationships.

    “EHR notes are nonlinear documents where contextual cues, provisional impressions, and objective data coexist in complex relationships 44 .”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [44]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Two independent annotators reviewed all outputs following the same schema as in development.

    “Two independent annotators reviewed all outputs following the same schema as in development.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data

    The natural attenuation of several bias subtypes in GPT-5 suggests some safeguards may emerge from architectural scaling and training alone.

    “Of course, the natural attenuation of several bias subtypes in GPT-5 suggests some safeguards may emerge from architectural scaling and training alone.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    This study was approved by the Mayo Clinic Institutional Review Board with a waiver of informed consent given its retrospective nature.

    “This study was approved by the Mayo Clinic Institutional Review Board with a waiver of informed consent given its retrospective nature.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Assessment and plan sections were removed to blind inference.

    “Assessment and plan sections were removed to blind inference.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisecitation

    Reasoning through EHR notes requires causal inference, temporal sequencing, and some skepticism-capabilities not captured by endpoint accuracy.

    “Reasoning through these requires causal inference, temporal sequencing, and some skepticism-capabilities not captured by endpoint accuracy 45,46 .”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [45]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [46]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisecitation

    Recent work shows models may rely on latent decision factors not faithfully represented in their chain-of-thought, especially in harder tasks.

    “Recent work shows models may rely on latent decision factors not faithfully represented in their chain-of-thought, especially in harder tasks 47 .”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [47]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    The CORAL database includes deidentified progress notes from breast and pancreatic cancer patients at the University of California, San Francisco.

    “The CORAL database, accessed via the controlled-access repository PhysioNet 68 , includes deidentified progress notes from BCa and PDAC patients at the University of California, San Francisco (UCSF) 6 .”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [68]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [6]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    The taxonomy was developed de novo but informed by established literature on cognitive errors in clinical reasoning, natural language processing error classifications, and prior failure mode analyses.

    “The taxonomy was developed de novo but informed by established literature on cognitive errors in clinical reasoning [69][70][71][72] , natural language processing error classifications 29,73 , and prior failure mode analyses.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [69][70][71][72]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [29]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [73]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationcitation

    For both models, task-dependent error distribution implies different contexts elicit distinct failure modes: extraction triggers pattern-matching errors, while recommendation tasks weighing uncertainty expose premature closure and insufficient deliberation.

    “For both models we observed task-dependent error distribution, implying different contexts elicit distinct failure modes: extraction triggers pattern-matching errors, while recommendation tasks weighing uncertainty expose premature closure and insufficient deliberation [37][38][39] .”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [37][38][39]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    The taxonomy provides a clinically conscious approach to classifying LLM reasoning errors.

    “provides a clinically conscious approach to classifying LLM reasoning errors”
    Quote in the paperNot a substantial claim
72 of 72

  • The paper relies on this without support.

    The reasoning reliability of LLMs in real-world oncology care remains unclear.

    “Large language models (LLMs) show strong performance on clinical benchmarks, yet their reasoning reliability in real-world oncology care remains unclear.”

  • The paper relies on this without support.

    Clinician-rated clinical impact scores and NCCN guideline concordance are valid proxies for clinically meaningful risk.

    “Clinical impact was rated on a 5-point Likert scale (1 = potentially harmful, 5 = clinically optimal; Table 2), and we also evaluated for NCCN guideline concordance.”

  • The paper relies on this without support.

    The 2024 NCCN guidelines represent appropriate clinical care for prostate cancer.

    “Such tasks were specifically aligned with 2024 NCCN guidelines 30 .”

  • The paper relies on this without support.

    Chain-of-thought reasoning traces elicited from LLMs provide a clinically interpretable basis for identifying reasoning failures.

    “To create an error taxonomy for clinical reasoning breakdowns, we analyzed chain-ofthought (CoT) responses generated by GPT-4-32k from extraction-level prompts.”
4 of 4

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 2

  1. Using large language model to guide patients to create efficient and comprehensive clinical care message | Journal of the American Medical Informatics Association | Oxford Academic. https://academic.oup.com/jamia/article/31/8/1665/7699038.
  2. Jacovi, A. & Goldberg, Y. Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness? in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (eds.

Cited by

Sorted by citations

Authors
Kenaston, M. W., Ayub, U., Parmar, M., Kumar, P., Anjum, M. U., Naqvi, S. A. A., Chaudhuri, A. A., Zakharia, Y., Heath, E. I., Bekaii-Saab, T. S., Tao, C., Van Allen, E. M., Zhou, B., Choi, Y., Baral, C., & Riaz, I. B.
Affiliations
  • Mayo Clinic (US)
  • Arizona State University (US)
Corresponding author
Riaz, Irbaz Bin
ORCID
Venue
npj Digital Medicine
Publisher
Springer Science and Business Media LLC
Published
2026-06-30
ISSN
2398-6352 (ISSN-L)
DOI
10.1038/s41746-026-02951-5
OpenAlex
W7166705415
Cited by
2
References
0
Citations by year
2026: 2
FWCI
4.30
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
Funders
American Society of Clinical Oncology
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Keywords
large language models, GPT-4, cognitive biases, confirmation bias, anchoring effect, omission errors, automatic error detection, clinical decision support

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review