CoChat
Sign Up
Cited by 1Artificial Intelligence in Healthcare and EducationAI in cancer detectionArtificial Intelligence in Healthcare

Feasibility and Concordance of a Large Language Model (ChatGPT-5) as a Clinical Decision Support Tool in Gynecologic Oncology Tumor Boards: A Blinded, Multi-Observer Study

Open access
Asoglu, Hatice, Kose, Sevgul, Kayim, Oguzhan, Abaci, Ali, Khatib, Ghanim, Vardar, Mehmet Ali, Kilic Bagir, Emine, Gumurdulu, Derya +5 more·2026·Journal of Clinical Medicine
DOI 10.3390/jcm15124451PMC PMC13301218
Open this paper in CoChat — free account

Abstract

Background: The reliability of large language models (LLMs) in complex oncologic decision-making remains inadequately validated. This study evaluated the concordance of ChatGPT-5 with multidisciplinary tumor board (MDT) decisions in gynecologic oncology, assessing accuracy, reproducibility, and domains of discordance. Methods: We analyzed 242 gynecologic cancer cases (endometrial n = 102, ovarian n = 85, cervical n = 40, rare n = 15) discussed at the Çukurova University Gynecologic Oncology MDT (2024–2025). Standardized clinical summaries were input into ChatGPT-5 using a structured prompt template. Each case was queried three times within a single calendar day using independent conversations. Recommendations were evaluated by two blinded medical oncologists using a 5-point Likert scale. A composite performance score (CPS) was calculated as (mean Likert/5) × 100. Concordance was analyzed using Cohen’s kappa (κ). Results: Inter-rater reliability was substantial to almost perfect for both MDT (κ = 0.761) and AI (κ = 0.814) evaluations (both p < 0.001). MDT–AI concordance was fair (Rater 1: κ = 0.258; Rater 2: κ = 0.334). CPS were significantly higher for MDT versus AI (Rater 1: 93.8% ± 5.2 vs. 89.4% ± 6.7; Rater 2: 93.4% ± 5.5 vs. 89.7% ± 6.4; both p < 0.001). Full consistency across three queries was achieved in only 37.2% of cases (90/242). AI performance was significantly inferior in advanced-stage disease (p = 0.008), genetic testing (p = 0.006), fertility-sparing (p = 0.018), and novel therapeutics (p = 0.003). Conclusions: ChatGPT-5 demonstrates potential as a clinical decision support tool but lacks sufficient reliability for independent use. Key limitations include inconsistency in 62.8% of cases, suboptimal performance in advanced-stage disease, and deficiencies in precision oncology domains. These findings suggest that human expertise remains indispensable for the individualized management of complex gynecologic malignancies.

AI snapshot

Written from the paper’s full text.

TL;DR

In 242 gynecologic cancer cases, ChatGPT-5 showed only fair concordance with multidisciplinary tumor board (MDT) decisions (κ = 0.258–0.334) and full same-day reproducibility in just 37.2%. It underperformed in advanced-stage disease, genetic testing, fertility-sparing management, and novel therapeutics, indicating it is not reliable for independent clinical use.

Key findings

  • Inter-rater reliability was substantial for MDT (κ = 0.761) and almost perfect for AI (κ = 0.814), but MDT–AI concordance was fair (Rater 1: κ = 0.258; Rater 2: κ = 0.334; both p < 0.001).
  • Composite performance scores were significantly higher for MDT than AI (Rater 1: 93.8% ± 5.2 vs. 89.4% ± 6.7; Rater 2: 93.4% ± 5.5 vs. 89.7% ± 6.4; both p < 0.001). AI scored highest in endometrial cancer (91.2% ± 5.8) and lowest in rare tumors (84.2% ± 8.4; Kruskal–Wallis p = 0.014).
  • Full consistency across three same-day queries occurred in only 37.2% of cases (90/242); mean reproducibility score was 4.06 ± 0.87. Among inconsistent cases, variation most often involved systemic therapy selection (33.6%), surgical management (23.0%), and genetic testing recommendations (15.8%).
  • AI performance was significantly worse in advanced-stage disease (CPS 87.2% ± 7.3 vs. 91.6% ± 5.4; p = 0.008), genetic testing (86.8% ± 7.8 vs. 90.6% ± 5.9; p = 0.006), fertility-sparing management (87.4% ± 7.2 vs. 90.2% ± 6.2; p = 0.018), and novel therapeutics (85.6% ± 8.1 vs. 90.8% ± 5.6; p = 0.003).

Methods

  • Retrospective, cross-sectional, comparative single-center study at Çukurova University (January 2024–December 2025) including 242 consecutive gynecologic cancer cases: endometrial n = 102, ovarian n = 85, cervical n = 40, rare n = 15. Cases with incomplete staging or discussed solely for palliative management were excluded.
  • Standardized clinical summaries were entered into ChatGPT-5 using a structured prompt template; each case was queried three times within one calendar day using independent conversations. MDT consensus recommendations served as the reference standard. Two blinded medical oncologists independently rated MDT and AI recommendations on a 5-point Likert scale; composite performance score (CPS) = (mean Likert/5) × 100.
  • Concordance and inter-rater reliability used Cohen’s kappa; CPS comparisons used Wilcoxon signed-rank tests; subgroup comparisons used Mann–Whitney U and chi-squared/Fisher’s exact tests; two-sided p < 0.05; SPSS version 25.0.

Limitations

  • Stated by the authors: single-institution retrospective design limits generalizability; both blinded raters were medical oncologists, imposing a specialty-specific lens; single-LLM design; same-day-only reproducibility assessment; unstandardized prompt variation; limited power for the rare tumor subgroup (n = 15); concordance used as a surrogate without patient outcome data; CPS is a pragmatic, non-validated metric.
  • Stated by the authors: ChatGPT-5’s training data cutoff (June 2024) may have limited incorporation of newer guideline updates and emerging therapies.
  • Evident in the text: no multi-model comparison, no prospective outcome validation, and no assessment of reproducibility over longer intervals (weeks to months); the authors note these as future research priorities.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/jcm15124451

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

32 of 45 cited sources checked · 27 supported · 0 flagged · accuracy not yet measured

  • Interpretationcitation

    Our findings align closely with Pergialiotis et al.

    “Our findings align closely with the most directly comparable study: Pergialiotis et al.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [15] Pergialiotis, V., Thomakos, N., Lygizos, V., Fanaki, M., Varthaliti, A., Vlachos, D.E., and Haidopoulos, D. (2026). Discrepancies Between MDT Recommendations and AI-Generated Decisions in Gynecologic Oncology: A Retrospective Comparative Cohort Study. Cancers, 18. source
  • Interpretationreasoning

    The convergence of our ChatGPT-5 results with the earlier GPT-4o data of Karabuğa et al. indicates that advancing model generation does not, by itself, substantially narrow the LLM-expert gap.

    “the convergence of our ChatGPT-5 results with the earlier GPT-4o data of Karabuğa et al. indicates that advancing model generation does not, by itself, substantially narrow the LLM–expert gap.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [25] Karabuğa, B., Karaçin, C., Büyükkör, M., Bayram, D., Aydemir, E., Kaya, O.B., Yılmaz, M.E., Çamöz, E.S., and Ergün, Y. (2025). The Role of Artificial Intelligence (ChatGPT-4o) in Supporting Tumor Board Decisions. J. Clin. Med., 14. source
  • Premisecitation

    Current consensus guidelines for ovarian, endometrial, and cervical cancer increasingly incorporate molecular classification, biomarker-driven therapy, and fertility-sparing algorithms requiring contextualized interpretation.

    “Current consensus guidelines for ovarian [33], endometrial [34], and cervical cancer [35] increasingly incorporate molecular classification, biomarker-driven therapy, and fertility-sparing algorithms requiring contextualized interpretation”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites [33] Colombo, 2019, ESMO-ESGO Consensus Conference Recommendations on Ovarian Cancer: Pathology and Molecular Biology, Early and Advanced Stages, Borderline Tumours and Recurrent Disease, Ann. Oncol. source
    • cites [34] Concin, 2021, ESGO/ESTRO/ESP Guidelines for the Management of Patients with Endometrial Carcinoma, Int. J. Gynecol. Cancer source
    • cites [35] Cibula, 2018, The European Society of Gynaecological Oncology/European Society for Radiotherapy and Oncology/European Society of Pathology Guidelines for the Management of Patients with Cervical Cancer, Int. J. Gynecol. Cancer source

      The Fertility-Sparing Treatment recommendations (indexed copy; page not supplied) give eligibility criteria and management steps that depend on tumor size, histology, and nodal status. The General Recommendations also call for multidisciplinary, individualized treatment planning. The cited cervical-cancer guideline does not establish the claim that it incorporates molecular classification or biomarker-driven therapy, or that these features are increasingly incorporated over time. (LLM, conf=0.87)

      Experimental — accuracy not yet measured
  • Premisecitation

    Contextualization and model selection materially affect oncologic output quality.

    “contextualization and model selection materially affect oncologic output quality [26]”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [26] Brown, 2025, Precision Oncology in Non-Small Cell Lung Cancer: A Comparative Study of Contextualized ChatGPT Models, Cureus source
  • Interpretationown data

    Our data corroborate Griewing et al.'s finding that genetic testing is the domain of lowest LLM-MDT concordance.

    “a finding that our data corroborate.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [32] Griewing, 2024, Evolution of Publicly Available Large Language Models for Complex Decision-Making in Breast Cancer Care, Arch. Gynecol. Obstet. source
  • May combine two claims. 27
  • Classified as probably not a substantial claim. 28

  • May combine two claims.

    Premisecitation

    LLMs can express unwarranted confidence in domains beyond their reliable competence, complicating unsupervised use in specialized oncologic practice.

    “A further concern is that these models can express unwarranted confidence in domains beyond their reliable competence, complicating their unsupervised use in specialized oncologic practice [5].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [5] Pavlik, E.J., Woodward, J.L., Lawton, F., Swiecki-Sikora, A.L., Ramaiah, D.D., and Rives, T.A. (2025). Artificial Intelligence in Relation to Accurate Information and Tasks in Gynecologic Oncology and Clinical Medicine—Dunning–Kruger Effects and Ultracrepidarianism. Diagnostics, 15. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Gumilar et al. found variable accuracy across three LLMs on gynecologic cancer vignettes.

    “Gumilar et al. found variable accuracy across three LLMs on gynecologic cancer vignettes [14].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [14] Gumilar, 2024, Assessment of Large Language Models (LLMs) in Decision-Making Support for Gynecologic Oncology, Comput. Struct. Biotechnol. J. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Reporting followed, where applicable, the REFINE consensus checklist for foundation and large language models in medical research.

    “reporting followed, where applicable, the REFINE consensus checklist for foundation and large language models in medical research [17].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [17] Mese, I., Akinci D’Antonoli, T., Bluethgen, C., Bressem, K., Cuocolo, R., Chaudhari, A., Tejani, A.S., Isaac, A., Ponsiglione, A., and Meddeb, A. (2026). Reporting Checklist for Foundation and Large Language Models in Medical Research (REFINE): An International Consensus Guideline. Diagn. Interv. Radiol., online ahead of print. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Meyer et al. evaluated GPT-4 in 60 gynecologic and senologic cases.

    “Meyer et al. evaluated GPT-4 in 60 gynecologic and senologic cases [12]”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [12] Meyer, 2025, Exploring the Potential of AI-Powered Applications for Clinical Decision-Making in Gynecologic Oncology, Int. J. Gynecol. Obstet. source

  • Classified as probably not a substantial claim.

    Premisecitation

    Generative AI platforms have demonstrated growing capacity for treatment recommendation generation, mutation interpretation, and clinical trial matching in oncology.

    “Against this benchmark, generative AI platforms have demonstrated growing capacity for treatment recommendation generation, mutation interpretation, and clinical trial matching in oncology [9].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [9] Hamamoto, 2026, Implementing Generative Artificial Intelligence in Precision Oncology: Safety, Governance, and Significance, J. Hematol. Oncol. source

      The sections on genetic-mutation interpretation and clinical-trial eligibility describe developing AI applications and report evaluated trial-matching performance (page not provided). The paper also describes support for treatment selection and tumor-board deliberations, but the passages do not establish that generative AI platforms have demonstrated the capacity to generate treatment recommendations as a distinct capability. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    A recent systematic review concluded that, although LLMs show promise in gynecologic oncology, critical evidence gaps persist regarding complex case-based decision-making and reproducibility.

    “A recent systematic review concluded that, although LLMs show promise in gynecologic oncology, critical evidence gaps persist regarding complex case-based decision-making and reproducibility [11].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [11] Rosati, 2026, Natural Language Processing as Consultation Service Platform or Clinical Decision Support System in Gynecologic Oncology: A Systematic Review, Int. J. Gynecol. Cancer source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    LLMs are approaching or exceeding passing thresholds on medical licensing examinations.

    “Their performance on standardized assessments has advanced rapidly, with LLMs approaching or exceeding passing thresholds on medical licensing examinations [2]”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [2] Kung, T.H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., and Maningo, J. (2023). Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models. PLoS Digit. Health, 2. source

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    LLMs have demonstrated remarkable capacity to encode clinical knowledge and generate contextually appropriate medical responses.

    “These models, trained on vast biomedical corpora, have demonstrated remarkable capacity to encode clinical knowledge and generate contextually appropriate medical responses [1].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [1] Singhal, 2023, Large Language Models Encode Clinical Knowledge, Nature source

  • May combine two claims.

    Premisecitation

    Pergialiotis et al. evaluated ChatGPT-5 against MDT decisions in 599 gynecologic cancer cases, reporting high overall concordance but recurrent discrepancies in advanced and recurrent disease, with staging disagreements predominating.

    “Most recently, Pergialiotis et al. evaluated ChatGPT-5 against MDT decisions in 599 gynecologic cancer cases, reporting high overall concordance but recurrent discrepancies in advanced and recurrent disease, with staging disagreements predominating [15].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [15] Pergialiotis, V., Thomakos, N., Lygizos, V., Fanaki, M., Varthaliti, A., Vlachos, D.E., and Haidopoulos, D. (2026). Discrepancies Between MDT Recommendations and AI-Generated Decisions in Gynecologic Oncology: A Retrospective Comparative Cohort Study. Cancers, 18. source

      The Results report 599 patients and 77.0% agreement on FIGO staging; the Discussion (section 4.2) identifies advanced and recurrent cases as challenging and staging disagreement as the most prominent issue. But the Conclusions characterize overall agreement with MDT decisions as moderate, not high, so the claim overstates that aspect. (LLM, conf=0.87)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Chatbot-generated replies to patient questions have, in some settings, been rated comparably to or more favorably than physician replies for quality and empathy.

    “chatbot-generated replies to patient questions have, in some settings, been rated comparably to or more favorably than physician replies for quality and empathy [3].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [3] Ayers, 2023, Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum, JAMA Intern. Med. source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Psilopatis et al. demonstrated feasibility but noted discrepancies in complex cases.

    “Psilopatis et al. demonstrated feasibility but noted discrepancies in complex cases [13]”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [13] Psilopatis, 2025, The Role of Artificial Intelligence in Gynecologic Oncology Decision-Making: A Feasibility Study, Gynecol. Obstet. Investig. source

  • May combine two claims.

    Interpretationcitation

    The complexity of current ASCO guidelines for germline and somatic testing in epithelial ovarian cancer and the rapidly evolving landscape of biomarker-driven therapy likely exceed what training data alone can capture.

    “the complexity of current ASCO guidelines for germline and somatic testing in epithelial ovarian cancer [31] and the rapidly evolving landscape of biomarker-driven therapy likely exceed what training data alone can capture.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [31] Konstantinopoulos, 2020, Germline and Somatic Tumor Testing in Epithelial Ovarian Cancer: ASCO Guideline, J. Clin. Oncol. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Structured chatbot evaluations in thyroid nodule and papillary thyroid cancer management report a pattern of promise tempered by inconsistency in nuanced cases.

    “structured chatbot evaluations in other complex domains, such as thyroid nodule and papillary thyroid cancer management, report a similar pattern of promise tempered by inconsistency in nuanced cases [27].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [27] Effraimidis, 2026, Promise and Pitfalls of AI Chatbots in Complex Decision-Making for Thyroid Nodules and Papillary Thyroid Cancer, Eur. Thyroid J. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Büyükceran et al. achieved 93.9% concordance in 33 patients using highly structured inputs.

    “Büyükceran et al. achieved 93.9% concordance in 33 patients using highly structured inputs [21]”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [21] Seyfettin, 2025, High Concordance between GPT-4o and Multidisciplinary Tumor Board Decisions in Breast Cancer: A Retrospective Decision Support Analysis, J. Med. Syst. source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Dogan et al. observed 76.4% agreement in mixed solid tumors, with most discrepancies arising in cases requiring individualized judgment.

    “In mixed solid tumors, Dogan et al. observed 76.4% agreement, with most discrepancies arising in cases requiring individualized judgment [24].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [24] Dogan, I., Bartin, M.K., Sonmez, E., Seyran, E., Bozkurt, H.A., Yuksek, M., Serbes, E.D., Zalova, G., and Celik, S. (2025). Chat GPT Performance in Multi-Disciplinary Boards—Should AI Be a Member of Cancer Boards?. Healthcare, 13. source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Dehdab et al. found deficiencies in treatment sequencing and chemotherapy selection in sarcoma despite reasonable overall performance.

    “in sarcoma, Dehdab et al. found deficiencies in treatment sequencing and chemotherapy selection despite reasonable overall performance [23].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [23] Dehdab, 2026, When AI Joins the Table: Evaluating Large Language Model Performance in Soft Tissue Sarcoma Tumor Board Decisions, J. Cancer Res. Clin. Oncol. source

  • May combine two claims.

    Premisecitation

    Retrieval-augmented generation, domain-specific fine-tuning, and deployment within structured clinical-decision-support platforms have been associated with improved guideline concordance in oncology settings.

    “Retrieval-augmented generation grounded in current NCCN and ESGO guideline corpora, domain-specific fine-tuning on curated gynecologic oncology cases, and deployment within structured clinical-decision-support platforms have each been associated with improved guideline concordance in oncology settings [28,36]”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [28] Hack, 2026, Evaluation of Large Language Models as Decision Support Tools for Head and Neck Cancer Management: A Blinded Multidisciplinary Simulation Study, Oral Oncol. source
    • cites [36] Lajmi, 2026, Simulation-Based Evaluation of a Large Language Model-Enabled Clinical Decision Support Platform in Oncology, JCO Clin. Cancer Inform. source

      The Introduction describes an oncology clinical-decision-support platform with a Guidelines tool, but the Methods and Results evaluate clinical-summary efficiency, completeness, correctness, and conciseness—not guideline concordance. Searches for guideline concordance, retrieval-augmented generation, fine-tuning, NCCN, and ESGO found no passage establishing the claimed association. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    De la Torre-Trillo et al. reported a kappa of 0.61 in urologic oncology.

    “In urologic oncology, De la Torre-Trillo et al. reported a kappa of 0.61 [22]”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [22] De la Torre-Trillo, J., Yáñez Castillo, Y., Melgarejo Segura, M.T., Carmona Sánchez, E., Zambudio Munuera, A., Mora-Delgado, J., and López Luque, A. (2026). Evaluation of Artificial Intelligence as a Decision-Support Tool in Urological Tumor Boards: A Study in Real Clinical Practice. J. Clin. Med., 15. source

  • May combine two claims.

    Premisecitation

    Pergialiotis et al. evaluated ChatGPT-5 against MDT decisions in 599 cases and found that concordance deteriorated in advanced and recurrent disease, with staging and multimodal decisions driving discordance.

    “Pergialiotis et al. evaluated the same model (ChatGPT-5) against MDT decisions in a larger cohort of 599 cases and likewise found that concordance deteriorated in advanced and recurrent disease, with staging and multimodal decisions driving discordance [15].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [15] Pergialiotis, V., Thomakos, N., Lygizos, V., Fanaki, M., Varthaliti, A., Vlachos, D.E., and Haidopoulos, D. (2026). Discrepancies Between MDT Recommendations and AI-Generated Decisions in Gynecologic Oncology: A Retrospective Comparative Cohort Study. Cancers, 18. source

  • May combine two claims.

    Findingown data

    Full consistency across three queries was achieved in only 37.2% of cases (90/242), meaning inconsistency in 62.8%.

    “Full consistency across three queries was achieved in only 37.2% of cases (90/242).”
    • Also stated in: § Abstract · plain
      “Key limitations include inconsistency in 62.8% of cases”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Among inconsistent cases, the most frequent variation was systemic therapy selection (33.6%), followed by surgical management (23.0%), genetic testing recommendations (15.8%), radiation therapy indications (11.8%), fertility-sparing specifics (9.2%), and novel agent integration (6.6%).

    “Among 152 inconsistent cases, variations most frequently involved systemic therapy selection (33.6%), surgical management (23.0%), genetic testing recommendations (15.8%), radiation therapy indications (11.8%), fertility-sparing specifics (9.2%), and novel agent integration (6.6%)”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In genetic testing cases, deficiencies included failure to recommend germline BRCA1/2 testing, inconsistent Lynch syndrome screening, and omission of somatic HRD testing.

    “Deficiencies included failure to recommend germline BRCA1/2 testing, inconsistent Lynch syndrome screening, and omission of somatic HRD testing.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    By tumor type, AI achieved the highest CPS for endometrial cancer (91.2% ± 5.8), followed by cervical (89.6% ± 6.3), ovarian (88.4% ± 7.1), and rare tumors (84.2% ± 8.4; Kruskal-Wallis p = 0.014).

    “By tumor type, AI achieved the highest CPS for endometrial cancer (91.2% ± 5.8), followed by cervical (89.6% ± 6.3), ovarian (88.4% ± 7.1), and rare tumors (84.2% ± 8.4; Kruskal–Wallis p = 0.014).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In cervical cancer, radical surgery versus chemoradiation selection and fertility-sparing appropriateness were key areas of discordance.

    “In cervical cancer, radical surgery versus chemoradiation selection and fertility-sparing appropriateness were key areas of discordance.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    MDT recommendations received a Likert score of 5 in 71.4% of pooled evaluations, compared with 56.8% for AI.

    “MDT recommendations received a Likert score of 5 in 71.4% of pooled evaluations, compared with 56.8% for AI”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    High inter-rater reliability (κ = 0.761-0.814) mitigates but does not eliminate the concern of specialty-specific evaluative lens.

    “high inter-rater reliability (κ = 0.761–0.814) mitigates but does not eliminate this concern.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In the fertility-sparing subgroup, AI frequently provided overly conservative recommendations or failed to recognize contraindications.

    “in the fertility-sparing subgroup, AI frequently provided overly conservative recommendations or failed to recognize contraindications”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The study design was informed by the DECIDE-AI framework for early-stage evaluation of AI-driven decision support systems.

    “The study design was informed by the DECIDE-AI framework for early-stage evaluation of AI-driven decision support systems [16]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [16] Vasey, 2022, Reporting Guideline for the Early-Stage Clinical Evaluation of Decision Support Systems Driven by Artificial Intelligence: DECIDE-AI, Nat. Med. source

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Multidisciplinary tumor boards (MDTs) represent the gold standard for complex oncologic decision-making.

    “Multidisciplinary tumor boards (MDTs) represent the gold standard for complex oncologic decision-making, integrating expertise across multiple disciplines to formulate individualized treatment strategies [7,8].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [7] Pillay, 2016, The Impact of Multidisciplinary Team Meetings on Patient Assessment, Management and Outcomes in Oncology Settings: A Systematic Review of the Literature, Cancer Treat. Rev. source
    • cites [8] Taylor, 2010, Multidisciplinary Team Working in Cancer: What Is the Evidence?, BMJ source

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    Identical template wording and field structure were applied verbatim across all cases, and default model settings were used without adjustment of any user-modifiable parameters.

    “To maximize standardization, identical template wording and field structure were applied verbatim across all cases, and default model settings were used without adjustment of any user-modifiable parameters.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Safe integration requires that AI complement rather than replace human expertise.

    “Safe integration therefore requires that AI complement rather than replace human expertise [6].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] Topol, 2019, High-Performance Medicine: The Convergence of Human and Artificial Intelligence, Nat. Med. source

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    A standardized clinical summary was constructed by a researcher blinded to the MDT decision.

    “For each case, a standardized clinical summary was constructed by a researcher blinded to the MDT decision”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Each case was queried at three separate time points within the same calendar day using independent conversations.

    “Each case was queried at three separate time points within the same calendar day using independent conversations.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The study was approved by the Çukurova University Faculty of Medicine Non-Interventional Clinical Research Ethics Committee.

    “Approved by the Çukurova University Faculty of Medicine Non-Interventional Clinical Research Ethics Committee (decision no. 81, meeting no. 164, dated 13 March 2026).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisereasoning

    The reliability of a current-generation model under repeated querying, and its behavior across these precision-oncology domains, thus remain undefined.

    “The reliability of a current-generation model under repeated querying, and its behavior across these precision-oncology domains, thus remain undefined.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Conducted per the Declaration of Helsinki.

    “Conducted per the Declaration of Helsinki.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    Both MDT and AI recommendations were independently evaluated by two blinded medical oncologists who were not involved in the MDT discussions or the AI data entry process.

    “Both MDT and AI recommendations were independently evaluated by two blinded medical oncologists (Ertugrul Bayram and Berksoy Sahin), who were not involved in the MDT discussions or the AI data entry process”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Raters were blinded to the source (MDT or AI) of each recommendation.

    “Raters were blinded to the source (MDT or AI) of each recommendation”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Clinical deployment of LLMs carries substantial limitations, including generation of inaccurate or fabricated outputs (hallucinations), variable output quality, and absence of transparent reasoning pathways.

    “However, their clinical deployment carries substantial limitations, including the generation of inaccurate or fabricated outputs (commonly termed “hallucinations”), variable output quality, and the absence of transparent reasoning pathways [4].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [4] Lee, 2023, Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine, N. Engl. J. Med. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The composite performance score is a pragmatic, non-validated metric designed for this analysis.

    “The composite performance score used in this study is a pragmatic, non-validated metric designed for this analysis”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The composite performance score (CPS) is a valid measure of clinical recommendation quality.

  • May combine two claims.

    Interpretationown data

    Advanced-stage disease is associated with lower CPS and consistency across multiple tumor types.

    “Advanced-stage disease was associated with significantly lower CPS and consistency, consistent with findings across multiple tumor types.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The design was informed by the DECIDE-AI framework.

    “the design was informed by the DECIDE-AI framework [16].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [16] Vasey, 2022, Reporting Guideline for the Early-Stage Clinical Evaluation of Decision Support Systems Driven by Artificial Intelligence: DECIDE-AI, Nat. Med. source

  • May combine two claims.

    Premisecitation

    Kaplan et al. found that while temporal stability was modest, consistency remained suboptimal for complex clinical scenarios in gynecologic oncology questions.

    “Kaplan et al., evaluating ChatGPT Omni across 700 gynecologic oncology questions at three-month intervals, found that while temporal stability was modest, consistency remained suboptimal for complex clinical scenarios [29].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [29] Kaplan, 2025, Artificial Intelligence and Gynecological Oncology: A Comparative Study of ChatGPT Omni and Gemini Pro across Repeated Intervals with Case-Scenario and Open-Ended Queries, Oncol. Res. Treat. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Hack et al. demonstrated that retrieval-augmented generation (RAG) approaches yielded superior guideline-concordant recommendations compared with standard LLM outputs in head and neck cancer.

    “In head and neck cancer, Hack et al. demonstrated that retrieval-augmented generation (RAG) approaches yielded superior guideline-concordant recommendations compared with standard LLM outputs [28].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [28] Hack, 2026, Evaluation of Large Language Models as Decision Support Tools for Head and Neck Cancer Management: A Blinded Multidisciplinary Simulation Study, Oral Oncol. source

  • May combine two claims.

    Study design and conductreasoning

    The study has limitations including single-LLM design, same-day-only reproducibility assessment, unstandardized prompt variation, limited statistical power for the rare tumor subgroup (n = 15), and use of concordance as a surrogate without patient outcome data.

    “Additional limitations include the single-LLM design, same-day-only reproducibility assessment, unstandardized prompt variation, limited statistical power for the rare tumor subgroup (n = 15), and the use of concordance as a surrogate without patient outcome data.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    Prompt design and case complexity substantially impact measured agreement.

    “highlighting the substantial impact of prompt design and case complexity on measured agreement.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Liao et al. reported that only 46% of ChatGPT-4.0 outputs fully aligned with expert recommendations across 362 patients, with lower performance in advanced-stage disease and molecular marker-dependent scenarios.

    “In breast cancer, Liao et al. reported that only 46% of ChatGPT-4.0 outputs fully aligned with expert recommendations across 362 patients, with lower performance in advanced-stage disease and molecular marker-dependent scenarios [20].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [20] Liao, 2025, Accuracy and Reproducibility of ChatGPT Responses to Breast Cancer Tumor Board Patients, JCO Clin. Cancer Inform. source
47 of 47

  • The paper relies on this without support.

    Gynecologic oncology presents particular challenges for AI-based decision support owing to the heterogeneous spectrum of malignancies it encompasses—ovarian, endometrial, cervical, and rare tumors—each with distinct molecular profiles, treatment paradigms, and complex considerations such as fertility-sparing approaches and genetic testing for hereditary cancer syndromes.

    “Gynecologic oncology presents particular challenges for AI-based decision support owing to the heterogeneous spectrum of malignancies it encompasses—ovarian, endometrial, cervical, and rare tumors—each with distinct molecular profiles, treatment paradigms, and complex considerations such as fertility-sparing approaches and genetic testing for hereditary cancer syndromes.”

  • The paper relies on this without support.

    MDT decisions are an appropriate reference standard (gold standard) for evaluating AI recommendations.

    “This study evaluated ChatGPT-5 against gynecologic oncology tumor board decisions”

  • The paper relies on this without support.

    The composite performance score (CPS) is a valid measure of clinical recommendation quality.

    “The composite performance score used in this study is a pragmatic, non-validated metric designed for this analysis”

  • The paper relies on this without support.

    Same-day repeated queries are a valid test of reproducibility.

    “ChatGPT-5 produced fully consistent recommendations in only 37.2% of cases upon three same-day queries”

  • The paper relies on this without support.

    The Likert scale used by raters is a valid instrument for assessing recommendation quality.

    “its derivation from widely used Likert scales”

  • The paper relies on this without support.

    ChatGPT-5's training data cutoff is June 2024.

    “the training data cutoff of ChatGPT-5 (June 2024)”

  • The paper relies on this without support.

    A concurrent study evaluating the same model in a larger cohort reached a convergent conclusion.

    “Notably, a concurrent study evaluating the same model in a larger cohort reached a convergent conclusion”

  • The paper relies on this without support.

    MDT recommendations are a valid reference standard for expert consensus in gynecologic cancer management.

    “ChatGPT-5 demonstrated fair concordance with MDT recommendations (κ = 0.258–0.334)”
    “gap between LLM outputs and expert consensus”
8 of 8

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Asoglu, H., Kose, S., Kayim, O., Abaci, A., Khatib, G., Vardar, M. A., Kilic Bagir, E., Gumurdulu, D., Kidi, M. M., Bayram, E., Koseci, T., Sahin, B., & Kara, I. O.
Affiliations
  • Cukurova University (TR)
  • Department of Radiology, Cukurova University Faculty of Medicine, 01330 Adana, Turkey
  • Department of Pathology, Cukurova University Faculty of Medicine, 01330 Adana, Turkey
  • Department of Medical Oncology, Cukurova University Faculty of Medicine, 01330 Adana, Turkey
Corresponding author
Asoglu, Hatice
ORCID
Venue
Journal of Clinical Medicine
Publisher
MDPI AG
Published
2026-06-09
Volume
15
Issue
12
Pages
4451
ISSN
2077-0383 (ISSN-L)
DOI
10.3390/jcm15124451
PMC
PMC13301218
OpenAlex
W7164012829
Semantic Scholar
a567417ac74c3aee52607344fe7667b831faa53f
Cited by
1
References
35
Citations by year
2026: 1
FWCI
1.99
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Clinical Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
Çukurova Üniversitesi
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
ChatGPT-5, large language models, clinical decision support, gynecologic oncology, multidisciplinary tumor board, endometrial cancer, ovarian cancer, cervical cancer, inter-rater reliability, advanced disease, fertility-sparing treatment, genetic testing, new drugs

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-30.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-30.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review