The Role of Artificial Intelligence (ChatGPT-4o) in Supporting Tumor Board Decisions
Open accessAbstract
Background/Objectives: Artificial intelligence (AI) has emerged as a promising field in the era of personalized oncology due to its potential to save time and workforce while serving as a supportive tool in patient management decisions. Although several studies in the literature have explored the integration of AI into oncology practice across different tumor types, available data remain limited. In our study, we aimed to evaluate the role of AI in the management of complex cancer cases by comparing the decisions of an in-house tumor board and ChatGPT-4o for patients with various tumor types. Methods: A total of 102 patients with diverse cancer types were included. Treatment and follow-up decisions proposed by both the tumor board and ChatGPT-4o were independently evaluated by two medical oncologists using a 5-point Likert scale. Results: Analysis of agreement levels showed high inter-rater reliability (κ = 0.722, p < 0.001 for tumor board decisions; κ = 0.794, p < 0.001 for ChatGPT decisions). However, concordance between the tumor board and ChatGPT was low, as reflected in the assessments of both raters (Rater 1: κ = 0.211, p = 0.003; Rater 2: κ = 0.376, p < 0.001). Both raters more frequently agreed with the tumor board decisions, and a statistically significant difference between tumor board and AI decisions was observed for both (Rater 1: Z = +4.548, p < 0.001; Rater 2: Z = +3.990, p < 0.001). Conclusions: These findings suggest that AI, in its current form, is not yet capable of functioning as a standalone decision-maker in the management of challenging oncology cases. Clinical experience and expert judgment remain the most critical factors in guiding patient care.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Karabuğa et al. reported concordance values of κ = 0.211-0.376 between ChatGPT-4o and an in-house tumor board across 102 patients with diverse cancer types.
“Karabuğa et al., using a similar methodological design with two blinded raters and a 5-point Likert scale, reported concordance values of κ = 0.211–0.376 between ChatGPT-4o and an in-house tumor board across 102 patients with diverse cancer types [25]”
From Feasibility and Concordance of a Large Language Model (ChatGPT-5) as a Clinical Decision Support Tool in Gynecologic Oncology Tumor Boards: A Blinded, Multi-Observer Study (2026) · cites this paper as [25]
The Results report 102 patients with different cancer diagnoses and tumor-board–ChatGPT-4o agreement of κ = 0.211 and 0.376 for the two raters. Methods §2.4 confirms two independent raters using a 5-point Likert scale. However, the Discussion says the raters knew whether recommendations came from the tumor board or ChatGPT-4o, so describing them as blinded is incorrect (page numbers unavailable in our copy). (LLM, conf=0.93)
Experimental — accuracy not yet measuredOur concordance rates are consistent with findings from studies in similarly complex clinical settings.
“Our concordance rates are consistent with findings from studies in similarly complex clinical settings.”
The convergence of our ChatGPT-5 results with the earlier GPT-4o data of Karabuğa et al. indicates that advancing model generation does not, by itself, substantially narrow the LLM-expert gap.
“the convergence of our ChatGPT-5 results with the earlier GPT-4o data of Karabuğa et al. indicates that advancing model generation does not, by itself, substantially narrow the LLM–expert gap.”
The concordance values in our study (κ = 0.258-0.334) are strikingly similar to those of Karabuğa et al. despite the use of a newer-generation model and a tumor type-specific cohort.
“values strikingly similar to those observed in our study (κ = 0.258–0.334) despite the use of a newer-generation model and a tumor type-specific cohort.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Lukac, S., Dayan, D., Fink, V., Leinert, E., Hartkopf, A., Veselinovic, K., Janni, W., Rack, B., Pfister, K., Heitmeir, B., et al. (2023) · Archives of Gynecology and Obstetrics · doi:10.1007/s00404-023-07130-5 · Cited by 130 · see citing works
Kaiser, K. N., Hughes, A. J., Yang, A. D., Turk, A. A., Mohanty, S., Gonzalez, A. A., Patzer, R. E., Bilimoria, K. Y., & Ellis, R. J. (2024) · Journal of Surgical Oncology · doi:10.1002/jso.27821 · Cited by 29 · see citing works
Flagged first, then by citations
Research works printed but not linked: 21
- [1]Haemmerli J. Sveikata L. Nouri A. May A. Egervari K. Freyschlag C. Lobrinus J.A. Migliorini D. Momjian S. Sanda N. ChatGPT in glioma adjuvant therapy decision making: Ready to assume the role of a doctor in the tumour board? BMJ Health Care Inform. 2023 30 e100775 10.1136/bmjhci-2023-100775 37399360 PMC10314415 · DOI
- [2]Lee J. Yoon W. Kim S. Kim D. Kim S. So C.H. Kang J. BioBERT: A pre-trained biomedical language representation model for biomedical text mining Bioinformatics 2020 36 1234 1240 10.1093/bioinformatics/btz682 31501885 PMC7703786 · DOI · Paper page
- [3]Topol E.J. High-performance medicine: The convergence of human and artificial intelligence Nat. Med. 2019 25 44 56 10.1038/s41591-018-0300-7 30617339 · DOI · Paper page
- [4]Nori H. King N. McKinney S.M. Carignan D. Horvitz E. Capabilities of gpt-4 on medical challenge problems arXiv 2023 2303.13375
- [5]Mehrabi N. Morstatter F. Saxena N. Lerman K. Galstyan A. A survey on bias and fairness in machine learning ACM Comput. Surv. 2021 54 1 35 10.1145/3457607 · DOI · Paper page
- [6]Gilson A. Safranek C.W. Huang T. Socrates V. Chi L. Taylor R.A. Chartash D. How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment JMIR Med. Educ. 2023 9 e45312 10.2196/45312 36753318 PMC9947764 · DOI · Paper page
- [7]Schmidl B. Hütten T. Pigorsch S. Stögbauer F. Hoch C.C. Hussain T. Wollenberg B. Wirth M. Assessing the role of advanced artificial intelligence as a tool in multidisciplinary tumor board decision-making for primary head and neck cancer cases Front. Oncol. 2024 14 1353031 10.3389/fonc.2024.1353031 38854718 PMC11157509 · DOI
- [8]Zabaleta J. Aguinagalde B. Lopez I. Fernandez-Monge A. Lizarbe J.A. Mainer M. Ferrer-Bonsoms J.A. de Assas M. Utility of Artificial Intelligence for Decision Making in Thoracic Multidisciplinary Tumor Boards J. Clin. Med. 2025 14 399 10.3390/jcm14020399 39860405 PMC11765867 · DOI
- [10]Rao A. Kim J. Kamineni M. Pang M. Lie W. Dreyer K.J. Succi M.D. Evaluating GPT as an Adjunct for Radiologic Decision Making: GPT-4 Versus GPT-3.5 in a Breast Imaging Pilot J. Am. Coll. Radiol. 2023 20 990 997 10.1016/j.jacr.2023.05.003 37356806 PMC10733745 · DOI
- [11]Liang R. Zhao A. Peng L. Xu X. Zhong J. Wu F. Yi F. Zhang S. Wu S. Hou J. Enhanced Artificial Intelligence Strategies in Renal Oncology: Iterative Optimization and Comparative Analysis of GPT 3.5 Versus 4.0 Ann. Surg. Oncol. 2024 31 3887 3893 10.1245/s10434-024-15107-0 38472675 · DOI
- [12]Li C.P. Jakob J. Menge F. Reißfelder C. Hohenberger P. Yang C. Comparing ChatGPT-3.5 and ChatGPT-4’s alignments with the German evidence-based S3 guideline for adult soft tissue sarcoma iScience 2024 27 111493 10.1016/j.isci.2024.111493 39759026 PMC11699281 · DOI
- [13]Kus F. Chalabiyev E. Yıldırım H.C. Ilgin K. Sırvan F. Dizdar O. Yalcin S. Artificial Intelligence (ChatGPT-4o) in Adjuvant Treatment Decision-Making for Stage II Colon Cancer: A Comparative Analysis with Clinician Recommendations and NCCN/ESMO Guidelines Int. J. Hematol. Oncol. 2025 35 68 74 10.4999/uhod.258149 · DOI
- [14]Erdat E.C. Yalçıner M. Örüncü M.B. Ürün Y. Şenler F. Assessing the accuracy of the GPT-4 model in multidisciplinary tumor board decision prediction Clinical and Translational Oncology Springer Berlin/Heidelberg, Germany 2025 PMC12399707 40133589
- [16]Alami K. Willemse E. Quiriny M. Lipski S. Laurent C. Donquier V. Digonnet A. Evaluation of ChatGPT-4’s Performance in Therapeutic Decision-Making During Multidisciplinary Oncology Meetings for Head and Neck Squamous Cell Carcinoma Cureus 2024 16 e68808 10.7759/cureus.68808 39376890 PMC11456411 · DOI
- [17]Ammo T. Guillaume V.G.J. Hofmann U.K. Ulmer N.M. Buenting N. Laenger F. Beier J.P. Leypold T. Evaluating ChatGPT-4o as a decision support tool in multidisciplinary sarcoma tumor boards: Heterogeneous performance across various specialties Front. Oncol. 2024 14 1526288 10.3389/fonc.2024.1526288 39896191 PMC11782276 · DOI
- [18]Valentini M. Szkandera J. Smolle M.A. Scheipl S. Leithner A. Andreou D. Artificial intelligence large language model ChatGPT: Is it a trustworthy and reliable source of information for sarcoma patients? Front. Public Health 2024 12 1303319 10.3389/fpubh.2024.1303319 38584922 PMC10995284 · DOI
- [19]Luo M. Trivedi S. Kurian A.W. Ward K. Keegan T.H.M. Rubin D. Banerjee I. Automated Extraction of Patient-Centered Outcomes After Breast Cancer Treatment: An Open-Source Large Language Model-Based Toolkit JCO Clin. Cancer Inform. 2024 8 e2300258 10.1200/CCI.23.00258 39167746 PMC11867221 · DOI
- [20]Lorenzi A. Pugliese G. Maniaci A. Lechien J.R. Allevi F. Boscolo-Rizzo P. Vaira L.A. Saibene A.M. Reliability of large language models for advanced head and neck malignancies management: A comparison between ChatGPT 4 and Gemini Advanced Eur. Arch. Otorhinolaryngol. 2024 281 5001 5006 10.1007/s00405-024-08746-2 38795148 PMC11392976 · DOI
- [21]Sim J. Wright C.C. The kappa statistic in reliability studies: Use, interpretation, and sample size requirements Phys. Ther. 2005 85 257 268 10.1093/ptj/85.3.257 15733050 · DOI · Paper page
- [22]Bujang M.A. Baharum N. Guidelines of the minimum sample size requirements for Kappa agreement test Epidemiol. Biostat. Public Health 2017 14 17614 10.2427/12267 · DOI
- [23]Benkirane K. Kay J. Perez-Ortiz M. How Can We Diagnose and Treat Bias in Large Language Models for Clinical Decision-Making? arXiv 2024 2410.16574
Cited by
Sorted by citations
Asoglu, H., Kose, S., Kayim, O., Abaci, A., Khatib, G., Vardar, M. A., Kilic Bagir, E., Gumurdulu, D., Kidi, M. M., Bayram, E., et al. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15124451 · Cited by 1 · see citing works
- Authors
- Karabuğa, B., Karaçin, C., Büyükkör, M., Bayram, D., Aydemir, E., Kaya, O. B., Yılmaz, M. E., Çamöz, E. S., & Ergün, Y.
- Affiliations
- Ankara Onkoloji Eğitim ve Araştırma Hastanesi (TR)
- Gülhane Eğitim ve Araştırma Hastanesi (TR)
- Department of Medical Oncology, Dr. Abdurrahman Yurtaslan Ankara Oncology Research and Training Hospital, 06200 Ankara, Turkey
- Department of Medical Oncology, Bower Hospital, 21100 Diyarbakır, Turkey
- Corresponding author
- Karabuğa, Berkan
- ORCID
- Venue
- Journal of Clinical Medicine
- Publisher
- MDPI AG
- Published
- 2025-05-18
- Volume
- 14
- Issue
- 10
- Pages
- 3535
- ISSN
- 2077-0383 (ISSN-L)
- PMC
- PMC12112035
- OpenAlex
- W4410504026
- Cited by
- 22
- References
- 2
- Citations by year
- 2026: 17 · 2025: 5
- FWCI
- 5.30
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Journal of Clinical Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- GPT-4o, artificial intelligence in oncology, multidisciplinary tumor board, clinical decision-making, inter-rater reliability, expert opinion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
