Capacity of ChatGPT to Identify Guideline-Based Treatments for Advanced Solid Tumors
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
BACKGROUND: ChatGPT, created by OpenAI, is a large language model which has become the fastest growing consumer application in history, recognized for its expansive knowledge of varied subjects. The field of oncology is highly specialized and requires nuanced understanding of medications and conditions. Herein, we sought to better qualify the ability of ChatGPT to name applicable treatments for patients with advanced solid cancers. METHODS: This observational study was conducted utilizing ChatGPT. The capacity of ChatGPT to tabulate appropriate systemic therapies for new diagnoses of advanced solid malignancies was ascertained through standardized prompts. A ratio of those medications listed by ChatGPT to those suggested in the National Comprehensive Cancer Network (NCCN) guidelines was produced and called the valid therapy quotient (VTQ). Additional descriptive analyses of the VTQ and its association with incidence and type of treatment were performed. RESULTS: Some 51 distinct diagnoses were utilized within this experiment. ChatGPT was able to identify 91 distinct medications in response to prompts related to advanced solid tumors. The overall VTQ is 0.77. In all cases, ChatGPT was able to provide at least one example of systemic therapy suggested by the NCCN. There was a weak association between incidence of each malignancy and the VTQ. CONCLUSION: The capacity of ChatGPT to identify medications used to treat advanced solid tumors indicates a level of concordance with the NCCN guidelines. As it stands, the role of ChatGPT to assist oncologists and patients in treatment decision making remains unknown. Nonetheless, in future iterations, it may be anticipated that accuracy and consistency in this domain will improve, and further studies will be needed to better quantify its capabilities.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Siegel, R. L., Miller, K. D., Wagle, N. S., & Jemal, A. (2023) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21763 · Cited by 16,303
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · cited 2×
“Recent literature has indicated that ChatGPT may have the ability to pass modern licensing examinations, while other models have conquered complex communication-oriented strategy games [6, 13].”
BackgroundKung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
ChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842
Vasey, B., Nagendran, M., Campbell, B., Clifton, D. A., Collins, G. S., Denaxas, S., Denniston, A. K., Faes, L., Geerts, B., Ibrahim, M., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01772-9 · Cited by 1,039 · cited 1×
“Leaders within healthcare and technology have sought to assuage these fears, as well as outline means of qualifying abilities of these new tools [2-4].”
BackgroundHopkins, A. M., Logan, J. M., Kichenadasse, G., & Sorich, M. J. (2023) · JNCI Cancer Spectrum · doi:10.1093/jncics/pkad010 · Cited by 259 · cited 1×
“While other groups have described the ability of ChatGPT to interface and answer patient-oriented questions, it is important to take its accuracy in doing so in context [14].”
Human-level play in the game of Diplomacy by combining language models with strategic reasoning
Meta Fundamental AI Research Diplomacy Team (FAIR)†, Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., et al. (2022) · Science · doi:10.1126/science.ade9097 · Cited by 241
Applicability of ChatGPT in Assisting to Solve Higher Order Problems in Pathology
Sinha, R. K., Deb Roy, A., Kumar, N., & Mondal, H. (2023) · Cureus · doi:10.7759/cureus.35237 · Cited by 187 · cited 1×
“Within the last few months, ChatGPT has garnered attention through nearly all domains of society, inspiring individuals from multiple fields, medicine included, to probe its flexibility and aptitude at a wide range of tasks [5-6].”
BackgroundPRISMA AI reporting guidelines for systematic reviews and meta-analyses on AI in healthcare
Cacciamani, G. E., Chu, T. N., Sanford, D. I., Abreu, A., Duddalwar, V., Oberai, A., Kuo, C.-C. J., Liu, X., Denniston, A. K., Vasey, B., et al. (2023) · Nature Medicine · doi:10.1038/s41591-022-02139-w · Cited by 143 · cited 1×
“Leaders within healthcare and technology have sought to assuage these fears, as well as outline means of qualifying abilities of these new tools [2-4].”
Background
Cited by
Large Language Models for Chatbot Health Advice Studies: A Systematic Review
Huo, B., Boyle, A., Marfo, N., Tangamornsuksan, W., Steen, J. P., McKechnie, T., Lee, Y., Mayol, J., Antoniou, S. A., Thirunavukarasu, A. J., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.57879 · Cited by 283
Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions
Lotter, W., Hassett, M. J., Schultz, N., Kehl, K. L., Van Allen, E. M., & Cerami, E. (2024) · Cancer Discovery · doi:10.1158/2159-8290.cd-23-1199 · Cited by 196
Applications of large language models in cancer care: current evidence and future perspectives
Iannantuono, G. M., Bracken-Clarke, D., Floudas, C. S., Roselli, M., Gulley, J. L., & Karzai, F. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1268915 · Cited by 115
Reimagining cancer treatments in the era of generative AI
Derbal, Y. (2025) · DIGITAL HEALTH · doi:10.1177/20552076251394631 · Cited by 4
Brown, E. D. L., Shah, H. A., Donnelly, B. M., Ward, M., Vojnic, M., & D'Amico, R. S. (2025) · Cureus · doi:10.7759/cureus.81097 · Cited by 2
Metadata
- Authors
- Schulte, B.
- Venue
- Cureus
- Published
- 2023-04-21
- Volume
- 15
- Issue
- 4
- Pages
- e37938-e37938
- ISSN
- 2168-8184 (ISSN-L)
- OpenAlex
- W4366683734
- Cited by
- 50
- References
- 9
- FWCI
- 1.73
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.