When AI Joins the Table: Evaluating Large Language Model Performance in Soft Tissue Sarcoma Tumor Board Decisions
Open accessAbstract
Objectives: Multidisciplinary tumor boards (MDTs) are critical for the personalized management of soft tissue sarcomas (STS), but they are limited by time, costs, and resource demands. With recent advances in large language models (LLMs) like ChatGPT, there is growing interest in evaluating their potential role in augmenting MDT workflows. This study aimed to assess the clinical performance of ChatGPT-4o in real-world STS cases using predefined evaluation criteria, comparing its treatment suggestions with expert MDT decisions. Materials and methods: This retrospective study included 152 patients presented to the multidisciplinary sarcoma tumor board. ChatGPT-4o was prompted to generate guideline-based treatment recommendations based on anonymized tumor board registration letters. Outputs were scored by blinded expert reviewers using a five-domain framework: diagnostic modalities, therapeutic modalities, treatment sequencing/timing, chemotherapy regimen, and clinical contextualization. Descriptive statistics and non-parametric ANOVA with post hoc tests assessed performance, including subgroup analysis by sarcoma subtype. Results: ChatGPT-4o scores were significantly lower than the maximum achievable value of 1.0 across all five criteria (all p < 0.0001). Among individual domains, clinical contextualization significantly outperformed all other criteria in pairwise comparisons (all p < 0.05). No significant performance differences were observed across sarcoma subtypes (H = 19.74, p = 0.138). Conclusions: ChatGPT-4o demonstrated substantial expert-rated performance in generating tumor board recommendations for soft tissue sarcoma cases, particularly excelling in personalized contextualization. Discrepancies in treatment sequencing and chemotherapy selection highlight the need for expert oversight. These findings support the feasibility of LLM integration into oncology workflows, warranting further refinement toward safe, supportive clinical use.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Dehdab et al. found deficiencies in treatment sequencing and chemotherapy selection in sarcoma despite reasonable overall performance.
“in sarcoma, Dehdab et al. found deficiencies in treatment sequencing and chemotherapy selection despite reasonable overall performance [23].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Detecting hallucinations in large language models using semantic entropy
Farquhar, S., Kossen, J., Kuhn, L., & Gal, Y. (2024) · Nature · doi:10.1038/s41586-024-07421-0 · Cited by 934 · cited 1× · see citing works
“A key issue is their tendency to produce hallucinations or confabulations, in which the model generates outputs that may sound clinically reasonable but are not actually supported by the provided data or current medical guidelines (Farquhar et al. 2024; Kalai et al. 2025).”
Anh-Hoang, D., Tran, V., & Nguyen, L.-M. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1622292 · Cited by 146 · cited 1× · see citing works
“Such techniques may reduce the likelihood of hallucinations and improve the consistency of guideline-based recommendations (Anh-Hoang et al. 2025; Amugongo et al. 2025 ).”
ESMO guidance on the use of Large Language Models in Clinical Practice (ELCAP)
Wong, E. Y. T., Verlingue, L., Aldea, M., Franzoi, M. A., Umeton, R., Halabi, S., Harbeck, N., Indini, A., Prelaj, A., Romano, E., et al. (2025) · Annals of Oncology · doi:10.1016/j.annonc.2025.09.001 · Cited by 44 · see citing works
Flagged first, then by citations
Research works printed but not linked: 33
- Aghamaliyev U, Karimbayli J, Giessen-Jung C et al (2024) ChatGPT’s gastrointestinal tumor board tango: a limping dance partner? Eur J Cancer. 10.1016/j.ejca.2024.114100 38729055 · DOI
- Ali SR, Dobbs TD, Hutchings HA, Whitaker IS (2023) Using ChatGPT to write patient clinic letters. Lancet Digit Health 5(4):e179–e181. 10.1016/S2589-7500(23)00048-1 36894409 · DOI · Paper page
- Ammo T, Guillaume VGJ, Hofmann UK et al (2025) Evaluating ChatGPT-4o as a decision support tool in multidisciplinary sarcoma tumor boards: heterogeneous performance across various specialties. Front Oncol 14:1526288. 10.3389/FONC.2024.1526288 39896191 PMC11782276 · DOI
- Amugongo LM, Mascheroni P, Brooks S, Doering S, Seidel J (2025) Retrieval augmented generation for large language models in healthcare: a systematic review. PLOS Digital Health 4(6):e0000877. 10.1371/JOURNAL.PDIG.0000877 40498738 PMC12157099 · DOI · Paper page
- Aubreville M, Ganz J, Ammeling J et al (2025) Prediction of tumor board procedural recommendations using large language models. Eur Arch Otorhinolaryngol 282(3):1619–1629. 10.1007/S00405-024-08947-9 39266750 · DOI
- Beattie J, Owens D, Navar AM et al (2025) ChatGPT augmented clinical trial screening. Mach Learn Health 1(1):015005. 10.1088/3049-477X/ADBD47 · DOI
- Calukovic B, Benzler K, Carter ME et al (2025) An analysis of the distribution of bone and soft tissue sarcoma diagnoses and their disparities in Southwest Germany: a multicenter approach. Front Oncol 15:1592004. 10.3389/FONC.2025.1592004 41199839 PMC12586119 · DOI
- Dehdab R, Brendlin A, Werner S et al (2024) Evaluating ChatGPT-4V in chest CT diagnostics: a critical image interpretation assessment. Jpn J Radiol 42(10):1168. 10.1007/S11604-024-01606-3 38867035 PMC11442562 · DOI
- Erdat EC, Yalçıner M, Örüncü MB, Ürün Y, Şenler FÇ (2025) Assessing the accuracy of the GPT-4 model in multidisciplinary tumor board decision prediction. Clin Transl Oncol 27(9):3793–3802. 10.1007/S12094-025-03905-1 40133589 PMC12399707 · DOI
- Goker B, Shea M, Zhang R et al (2024) The evolution of the multidisciplinary tumor board in orthopedic oncology: from its historical roots to its future potential. Holistic Integr Oncol 3(1):1–8. 10.1007/S44178-024-00107-7/FIGURES/2 · DOI
- Huang J, Yang DM, Rong R et al (2024) A critical assessment of using ChatGPT for extracting structured data from clinical notes. NPJ Digit Med 7(1):1–13. 10.1038/S41746-024-01079-8;SUBJMETA 38693429 PMC11063058 · DOI
- Kalai AT, Ofir O, Openai N, Vempala SS, Tech G, Openai EZ (2025) Why Language models hallucinate. Published online September 4, 2025. Accessed 12 Oct 2025. https://arxiv.org/pdf/2509.04664 · arXiv
- Lam T, Dogan I, Kadir Bartin M et al (2025) Chat GPT performance in multi-disciplinary boards—Should AI be a member of cancer boards? Healthcare 13(18):2254. 10.3390/HEALTHCARE13182254 41008386 PMC12469382 · DOI · Paper page
- Lamb BW, Brown KF, Nagpal K, Vincent C, Green JSA, Sevdalis N (2011) Quality of care management decisions by multidisciplinary cancer teams: a systematic review. Ann Surg Oncol 18(8):2116–2125. 10.1245/S10434-011-1675-6 21442345 · DOI
- Lammert J, Dreyer T, Mathes S et al (2024) Expert-guided large language models for clinical decision support in precision oncology. JCO Precis Oncol. 10.1200/PO-24-00478 39475661 · DOI
- Li J (2023) Security implications of AI chatbots in health care. J Med Internet Res 25(1):e47551. 10.2196/47551 38015597 PMC10716748 · DOI
- Li Y, Fu X, Verma G, Buitelaar P, Liu M (2025) Mitigating hallucination in large language models (LLMs): an application-oriented survey on RAG, reasoning, and agentic systems. Published online October 28, 2025. Accessed 26 Dec 2025. https://arxiv.org/pdf/2510.24476v1 · arXiv
- Liu J, Wang C, Liu S (2023) Utility of ChatGPT in clinical practice. J Med Internet Res. 10.2196/48568 37379067 PMC10365580 · DOI · Paper page
- Luchini C, Lawlor RT, Milella M, Scarpa A (2020) Molecular tumor boards in clinical practice. Trends Cancer 6(9):738–744. 10.1016/j.trecan.2020.05.008 32517959 · DOI
- Makary MA (2011) Multidisciplinary teams and clinics: better care or just more care. Ann Surg Oncol 18(8):2105–2106. 10.1245/s10434-011-1678-3 21559751 · DOI
- Mu Y, He D (2024) The potential applications and challenges of ChatGPT in the medical field. Int J Gen Med 17:817. 10.2147/IJGM.S456659 38476626 PMC10929156 · DOI
- Nardone V, Marmorino F, Germani MM et al (2024) The role of artificial intelligence on tumor boards: perspectives from surgeons, medical oncologists and radiation oncologists. Curr Oncol 31(9):4984. 10.3390/CURRONCOL31090369 39329997 PMC11431448 · DOI
- Nishisako S, Higashi T, Wakao F (2025) Reducing hallucinations and trade-offs in responses in generative AI chatbots for cancer information: development and evaluation study. JMIR Cancer. 10.2196/70176 40934488 PMC12425422 · DOI
- Park YE, Chae H (2024) The fidelity of artificial intelligence to multidisciplinary tumor board recommendations for patients with gastric cancer: a retrospective study. J Gastrointest Cancer 55(1):365–372. 10.1007/S12029-023-00967-8 37702851 PMC11096204 · DOI
- Schmidl B, Hütten T, Pigorsch S et al (2024) Assessing the use of the novel tool Claude 3 in comparison to ChatGPT 4.0 as an artificial intelligence tool in the diagnosis and therapy of primary head and neck cancer cases. Eur Arch Otorhinolaryngol 281(11):6099. 10.1007/S00405-024-08828-1 39112556 PMC11512878 · DOI
- Seitl F, Kovářík T, Mirshahi S et al (2024) Assessing the quality of information extraction. Published online April 5, 2024. Accessed 12 Oct 2025. https://arxiv.org/pdf/2404.04068 · arXiv
- Sorin V, Klang E, Sklair-Levy M et al (2023a) Large language model (ChatGPT) as a support tool for breast tumor board. NPJ Breast Cancer 9(1):1–4. 10.1038/S41523-023-00557-8 37253791 PMC10229606 · DOI
- Sorin V, Klang E, Sklair-Levy M et al (2023b) Large language model (ChatGPT) as a support tool for breast tumor board. NPJ Breast Cancer 9(1):44. 10.1038/S41523-023-00557-8 37253791 PMC10229606 · DOI
- Sorin V, Klang E, Sklair-Levy M et al (2023c) Large language model (ChatGPT) as a support tool for breast tumor board. NPJ Breast Cancer 9(1):1–4. 10.1038/S41523-023-00557-8;SUBJMETA 37253791 PMC10229606 · DOI
- Stalp JL, Denecke A, Jentschke M, Hillemanns P, Klapdor R (2024) Quality of ChatGPT-generated therapy recommendations for breast cancer treatment in gynecology. Curr Oncol 31(7):3845–3854. 10.3390/CURRONCOL31070284 39057156 PMC11275284 · DOI
- Stiller CA, Trama A, Serraino D et al (2013) Descriptive epidemiology of sarcomas in Europe: report from the RARECARE project. Eur J Cancer 49(3):684–695. 10.1016/j.ejca.2012.09.011 23079473 · DOI
- Tangsrivimol JA, Darzidehkalani E, Virk HUH et al (2025) Benefits, limits, and risks of ChatGPT in medicine. Front Artif Intell 8:1518049. 10.3389/FRAI.2025.1518049 39949509 PMC11821943 · DOI
- Zhao X, Zhang Y, Ma X et al (2020) Concordance between treatment recommendations provided by IBM Watson for Oncology and a multidisciplinary tumor board for breast cancer in China. Jpn J Clin Oncol 50(8):852–858. 10.1093/JJCO/HYAA051 32419014 · DOI
Cited by
Sorted by citations
Asoglu, H., Kose, S., Kayim, O., Abaci, A., Khatib, G., Vardar, M. A., Kilic Bagir, E., Gumurdulu, D., Kidi, M. M., Bayram, E., et al. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15124451 · Cited by 1 · see citing works
“In urologic oncology, De la Torre-Trillo et al. reported a kappa of 0.61 [22], and in sarcoma, Dehdab et al. found deficiencies in treatment sequencing and chemotherapy selection despite reasonable overall performance [23].”
Influential
- Authors
- Dehdab, R., Afat, S., Mankertz, F., Brendel, J. M., Maalouf, N., Werner, S., Brendlin, A., Herrmann, J., Nikolaou, K., Kloker, L. D., Calukovic, B., Benzler, K., Zender, L., & Deinzer, C. K. W.
- Affiliations
- Universitätsklinikum Tübingen (DE)
- Corresponding author
- Deinzer, Christoph K. W.
- ORCID
- Venue
- Journal of Cancer Research and Clinical Oncology
- Published
- 2026-02-27
- Volume
- 152
- Issue
- 2
- ISSN
- 0084-5353 (ISSN-L)
- 1432-1335
- PMC
- PMC12948744
- OpenAlex
- W7131858462
- Cited by
- 5
- References
- 3
- Citations by year
- 2026: 5
- FWCI
- 9.83
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Journal of Cancer Research and Clinical Oncology (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Eberhard Karls Universität Tübingen
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Pulmonary and Respiratory Medicine › Sarcoma Diagnosis and Treatment
- Keywords
- soft tissue sarcoma, large language models, GPT-4o, multidisciplinary tumor board, AI-assisted clinical decision-making, treatment guidelines, treatment sequencing, chemotherapy selection, sarcoma subtypes
- MeSH terms
- Clinical Decision-Making, Large Language Models, Generative Artificial Intelligence, Adult, Aged, Artificial Intelligence, Female, Humans, Male, Middle Aged, Retrospective Studies, Sarcoma, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
