Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index; 1 of the 6 sources its claims cite were checked for support.
Abstract
Data about the quality of cancer information that chatbots and other artificial intelligence systems provide are limited. Here, we evaluate the accuracy of cancer information on ChatGPT compared with the National Cancer Institute’s (NCI’s) answers by using the questions on the “Common Cancer Myths and Misconceptions” web page. The NCI’s answers and ChatGPT answers to each question were blinded, and then evaluated for accuracy (accurate: yes vs no). Ratings were evaluated independently for each question, and then compared between the blinded NCI and ChatGPT answers. Additionally, word count and Flesch-Kincaid readability grade level for each individual response were evaluated. Following expert review, the percentage of overall agreement for accuracy was 100% for NCI answers and 96.9% for ChatGPT outputs for questions 1 through 13 (ĸ = ‒0.03, standard error = 0.08). There were few noticeable differences in the number of words or the readability of the answers from NCI or ChatGPT. Overall, the results suggest that ChatGPT provides accurate information about common cancer myths and misconceptions.
Overview
CoChat Summary
TL;DR
This brief report evaluated whether ChatGPT provides accurate answers to 13 common cancer myths and misconceptions compared with National Cancer Institute (NCI) answers. Expert reviewers found ChatGPT outputs largely accurate and similar to NCI answers, though with some hedging/uncertainty noted.
Key findings
- NCI answers were rated accurate by 5 of 5 reviewers for 13 of 13 questions, giving 100% interrater agreement.
- ChatGPT outputs were rated accurate by 5 of 5 reviewers for 11 of 13 questions, with 96.9% overall agreement; the remaining 2 questions were rated accurate by 4 of 5 reviewers (Fleiss ĸ = ‒0.03, SE = 0.08, P = .6).
- There were few noticeable differences in word count or Flesch-Kincaid readability between NCI and ChatGPT answers, though both were written at a less readable level than health literacy advocates would likely recommend.
- Five repeated runs for each of the 13 questions showed subtle sentence-structure changes but consistent answers, suggesting accuracy remained stable with repetitive questioning.
Methods
- On December 20, 2022, the authors asked ChatGPT (December 15, 2022, version) the 13 questions from the NCI “Common Cancer Myths and Misconceptions” web page.
- NCI and ChatGPT answers were blinded and independently rated for accuracy (yes vs no) by 5 scientific reviewers with expertise in cancer treatment and cancer misinformation between December 20, 2022, and January 3, 2023.
- Interrater agreement was assessed with Fleiss ĸ; word count and Flesch-Kincaid grade level were evaluated for each response, and analyses used Stata version 17.
Limitations
- The paper states that it evaluated only common cancer misinformation in English, so more obscure myths may have incomplete or English-only information and may not be answered accurately.
- The paper states that ChatGPT outputs may be trained on models that could be scientifically outdated, and ChatGPT was limited to data collected before 2021, so accuracy may be delayed as new scientific information emerges.
- Evident from the text: this was a small-scale study of 13 questions using one ChatGPT version at one time point; the authors did not code for specific message features, though multiple observers perceived hedging and uncertainty in ChatGPT messages.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
1 Valid · 0 Not valid · 5 Not checked · 17 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There were few noticeable differences in the number of words or the readability of the answers from NCI or ChatGPT.”
- Also stated in: § Abstract · plain
“There were few noticeable differences in the number of words or the readability of the answers from NCI or ChatGPT.”
§ Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The remaining 2 questions (question 5, “Does my attitude—positive or negative—determine my risk of getting or surviving cancer?” and question 9, “Do power lines cause cancer?”) were rated as accurate by 4 of 5 reviewers, with a single unique reviewer rating each question as inaccurate (ĸ = ‒0.03, SE = 0.08, P = .6).”
§ Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we completed 5 runs repeatedly from the same input for each of the 13 questions (Supplementary Table 1, available online). On review, although subtle sentence structure changes were noted, the answers remained consistent”
§ Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Answers from both sources were written at a less readable rate than would likely be recommended by health literacy advocates, for example.”
§ Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information > Abstract
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“suggesting that answers remain accurate with repetitive questioning.”
§ Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information > Abstract
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
3 more claims
Create a free account to see all 23 claims and to verify the 5 references still open.
- Probably not a substantial claim. 10
- It may state two separate claims as one. 5
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“it was observed that ChatGPT responses were relevant and had a high level of accuracy, similar to the responses of the National Cancer Institute [32].”
From Is there any room for ChatGPT AI bot in speech-language pathology? (2025) · cites this paper as [32]
The cited source says what the claim attributes to it.
In the citing paper
“ChatGPT was shown to exhibit good performance at answering questions about cancer myths and misconceptions (159).”
From Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions (2024) · cites this paper as [159]
Citations
Cites
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · cited 1×
“Interrater agreement was evaluated by Fleiss Œ (5).”
MethodDissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“There is concern that algorithms reinforce current health disparities and inequities (9), although the extent to which this is true about health information specifically is currently unknown.”
BackgroundChatbots in the fight against the COVID-19 pandemic
Miner, A. S., Laranjo, L., & Kocaballi, A. B. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0280-0 · Cited by 322 · cited 1×
“spaces (3); recently, 1 such AI system receiving a lot of public and”
BackgroundAn Initial Model of Trust in Chatbots for Customer Service—Findings from a Questionnaire Study
Nordheim, C. B., Følstad, A., & Bjørkli, C. A. (2019) · Interacting with Computers · doi:10.1093/iwc/iwz022 · Cited by 233 · cited 1×
“Other practical considerations about how people will interpret AI-generated chat responses in terms of trust and credibility (10) exist, particularly as they relate to medical and health information, behaviors, and services compared with consumer products, behaviors, and services.”
BackgroundCancer misinformation and harmful information on Facebook and other social media: a brief report
Johnson, S. B., Parsons, M., Dorff, T., Moran, M. S., Ward, J. H., Cohen, S. A., Akerley, W., Bauman, J., Hubbard, J., Spratt, D. E., et al. (2022) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djab141 · Cited by 210 · cited 1×
“JNCI Cancer Spectrum, 2023, 7(2), pkad015”
Self-citationCalixte, R., Rivera, A., Oridota, O., Beauchamp, W., & Camacho-Rivera, M. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17186856 · Cited by 112 · cited 1×
“Patients are increasingly turning to the internet for information about cancer, with 80% of US adults reportedly using the internet to seek health information (1).”
Background
Research works printed but not linked: 1
- Hundt A, Agnew W, Zeng V, Kacianka S, Gombolay M. Robots enact malignant stereotypes. FACCT’22: 2022 ACM Conference on Fairness, Accountability, and Transparency. 2022; 743- 756.
Cited by
Utility of ChatGPT in Clinical Practice
Liu, J., Wang, C., & Liu, S. (2023) · Journal of Medical Internet Research · doi:10.2196/48568 · Cited by 505
“• Cancer: Johnson et al [12] used questions from the “Common Cancer Myths and Misconceptions” web page to assess the accuracy of ChatGPT and National Cancer Institute (NCI) answers to the questions.”
ChatGPT for healthcare services: An emerging stage for an innovative perspective
Javaid, M., Haleem, A., & Singh, R. P. (2023) · BenchCouncil Transactions on Benchmarks, Standards and Evaluations · doi:10.1016/j.tbench.2023.100105 · Cited by 429
The impact of generative artificial intelligence on socioeconomic inequalities and policy making
Capraro, V., Lentsch, A., Acemoglu, D., Akgun, S., Akhmedova, A., Bilancini, E., Bonnefon, J.-F., Brañas-Garza, P., Butera, L., Douglas, K. M., et al. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae191 · Cited by 371
Self-citationChatGPT in healthcare: A taxonomy and systematic review
Li, J., Dada, A., Puladi, B., Kleesiek, J., & Egger, J. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108013 · Cited by 452
ChatGPT for shaping the future of dentistry: the potential of multi-modal large language model
Huang, H., Zheng, O., Wang, D., Yin, J., Wang, Z., Ding, S., Yin, H., Xu, C., Yang, R., Zheng, Q., et al. (2023) · International Journal of Oral Science · doi:10.1038/s41368-023-00239-y · Cited by 324
Lim, Z. W., Pushpanathan, K., Yew, S. M. E., Lai, Y., Sun, C.-H., Lam, J. S. H., Chen, D. Z., Goh, J. H. L., Tan, M. C. J., Sheng, B., et al. (2023) · eBioMedicine · doi:10.1016/j.ebiom.2023.104770 · Cited by 310
Academic integrity and artificial intelligence: is ChatGPT hype, hero or heresy?
Currie, G. M. (2023) · Seminars in Nuclear Medicine · doi:10.1053/j.semnuclmed.2023.04.008 · Cited by 283
Potential of ChatGPT and GPT-4 for Data Mining of Free-Text CT Reports on Lung Cancer
Fink, M. A., Bischoff, A., Fink, C. A., Moll, M., Kroschke, J., Dulz, L., Heußel, C. P., Kauczor, H.-U., & Weber, T. F. (2023) · Radiology · doi:10.1148/radiol.231362 · Cited by 277
Bernstein, I. A., Zhang, Y. (., Govil, D., Majid, I., Chang, R. T., Sun, Y., Shue, A., Chou, J. C., Schehlein, E., Christopher, K. L., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.30320 · Cited by 247
Garg, R. K., Urs, V. L., Agrawal, A. A., Chaudhary, S. K., Paliwal, V., & Kar, S. K. (2023) · Health Promotion Perspectives · doi:10.34172/hpp.2023.22 · Cited by 202
Evaluation of ChatGPT-generated medical responses: A systematic review and meta-analysis
Wei, Q., Yao, Z., Cui, Y., Wei, B., Jin, Z., & Xu, X. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104620 · Cited by 201
Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions
Lotter, W., Hassett, M. J., Schultz, N., Kehl, K. L., Van Allen, E. M., & Cerami, E. (2024) · Cancer Discovery · doi:10.1158/2159-8290.cd-23-1199 · Cited by 196
Decker, H., Trang, K., Ramirez, J., Colley, A., Pierce, L., Coleman, M., Bongiovanni, T., Melton, G. B., & Wick, E. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.36997 · Cited by 178
Wang, L., Wan, Z., Ni, C., Song, Q., Li, Y., Clayton, E., Malin, B., & Yin, Z. (2024) · Journal of Medical Internet Research · doi:10.2196/22769 · Cited by 165
“…output; Lee et al [78] concerned that health disparity may result from low readability made by the sophistication of LLM wording; and Johnson et al [48] noted that LLM algorithms tend to reinforce the health disparity and to prevent LLM algorithms from exacerbating current disparity in health.”
Zhu, L., Mou, W., & Chen, R. (2023) · Journal of Translational Medicine · doi:10.1186/s12967-023-04123-5 · Cited by 143
“Lingxuan Zhu ,2† , Weiming Mou 2† and Rui Chen 1* To the editor, Large language models (LLMs) represented by ChatGPT have shown promising potential in the field of medicine [1, 2].”
BackgroundChatGPT: promise and challenges for deployment in low- and middle-income countries
Wang, X., Sanders, H. M., Liu, Y., Seang, K., Tran, B. X., Atanasov, A. G., Qiu, Y., Tang, S., Car, J., Wang, Y. X., et al. (2023) · The Lancet Regional Health - Western Pacific · doi:10.1016/j.lanwpc.2023.100905 · Cited by 138
Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data
Xu, C., Guo, D., Duan, N., & McAuley, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.385 · Cited by 123
AI for social science and social science of AI: A survey
Xu, R. F., Sun, Y., Ren, M., Guo, S., Pan, R., Lin, H., Sun, L., & Han, X. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103665 · Cited by 225
Gibson, D., Jackson, S., Shanmugasundaram, R., Seth, I., Siu, A., Ahmadi, N., Kam, J., Mehan, N., Thanigasalam, R., Jeffery, N., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/55939 · Cited by 122
“Additionally, the most significant limitation of ChatGPT is its potential for biased, outdated, or misleading content generation [1,4,6,53].”
Is ChatGPT accurate and reliable in answering questions regarding head and neck cancer?
Kuşcu, O., Pamuk, A. E., Sütay Süslü, N., & Hosal, S. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1256459 · Cited by 117
Loh, E. (2024) · BMJ Leader · doi:10.1136/leader-2023-000797 · Cited by 116
“26 A recent study assessed the feasibility of using an AI chatbot to answer patient questions as an adjunct to patient-provider communication and found that the general public appear to trust the use of chatbots to answer low risk health questions.”
BackgroundApplications of large language models in cancer care: current evidence and future perspectives
Iannantuono, G. M., Bracken-Clarke, D., Floudas, C. S., Roselli, M., Gulley, J. L., & Karzai, F. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1268915 · Cited by 115
Sun, H., Zhang, K., Lan, W., Gu, Q., Jiang, G., Yang, X., Qin, W., & Han, D. (2023) · Journal of Medical Internet Research · doi:10.2196/51300 · Cited by 101
“Previous research in medical LLMs has demonstrated that ChatGPT excels in responding to open-ended questions, but its performance tends to falter during examinations [27,28].”
Lahat, A., Shachar, E., Avidan, B., Glicksberg, B., & Klang, E. (2023) · Diagnostics · doi:10.3390/diagnostics13111950 · Cited by 92
“[21] investigated the accuracy of ChatGPT in generating responses to common cancer myths and misconceptions, concluding that it often provided accurate information.”
BackgroundHancı, V., Ergün, B., Gül, Ş., Uzun, Ö., Erdemir, İ., & Hancı, F. B. (2024) · Medicine · doi:10.1097/md.0000000000039305 · Cited by 112
The Use of Large Language Models to Generate Education Materials about Uveitis
Kianian, R., Sun, D., Crowell, E. L., & Tsui, E. (2024) · Ophthalmology Retina · doi:10.1016/j.oret.2023.09.008 · Cited by 83
Jo, H. (2025) · Information Development · doi:10.1177/02666669231202764 · Cited by 84
“On the other hand, skeptics express concerns regarding the potential loss of human touch, privacy issues, and the risk of reinforcing biases and misinformation (Cotton et al., 2023; Johnson et al., 2023).”
Fatima, A., Shafique, M. A., Alam, K., Fadlalla Ahmed, T. K., & Mustafa, M. S. (2024) · Medicine · doi:10.1097/md.0000000000039250 · Cited by 81
Large language model use in clinical oncology
Carl, N., Schramm, F., Haggenmüller, S., Kather, J. N., Hetz, M. J., Wies, C., Michel, M. S., Wessels, F., & Brinker, T. J. (2024) · npj Precision Oncology · doi:10.1038/s41698-024-00733-4 · Cited by 80
“Sources encompassed of fi cial information forums (3/34) 14,16,18 , guidelines (7/34) 15,17,28,29,33,35,45 , frequently asked questions (FAQs) from various sources, social media, hospital websites and Google Trends © (8/34) 19,30,31,36,37,39,40,42 , clinical cases (8/34) 20 – 22, 24 – 27 ,”
BackgroundMethodInfluentialBenefits, limits, and risks of ChatGPT in medicine
Tangsrivimol, J. A., Darzidehkalani, E., Virk, H. U. H., Wang, Z., Egger, J., Wang, M., Hacking, S., Glicksberg, B. S., Strauss, M., & Krittanawong, C. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1518049 · Cited by 76
“In medical query responses, ChatGPT demonstrated high accuracy rates for common retinal diseases (median scores of 4–5 out of 5 across different aspects) (Potapenko et al., 2023), and achieved 96.9% accuracy in addressing cancer myths and misconceptions (Johnson et al., 2023).”
InfluentialEvaluating large language models and agents in healthcare: key challenges in clinical applications
Chen, X., Xiang, J., Lu, S., Liu, Y., He, M., & Shi, D. (2025) · Intelligent Medicine · doi:10.1016/j.imed.2025.03.002 · Cited by 68
Generative AI, IoT, and blockchain in healthcare: application, issues, and solutions
Mazhar, T., Khan, S., Shahzad, T., Khan, M. A., Saeed, M. M. A., Awotunde, J. B., & Hamam, H. (2025) · Discover Internet of Things · doi:10.1007/s43926-025-00095-8 · Cited by 100
“By integrating the two technologies, healthcare practitioners can enhance patient outcomes, ensure data security, and facilitate the creation of successful treatment plans [51].”
Gödde, D., Nöhl, S., Wolf, C., Rupert, Y., Rimkus, L., Ehlers, J., Breuckmann, F., & Sellmann, T. (2023) · Journal of Medical Internet Research · doi:10.2196/49368 · Cited by 69
Pagallo, U., O’Sullivan, S., Nevejans, N., Holzinger, A., Friebe, M., Jeanquartier, F., Jean-Quartier, C., & Miernik, A. (2024) · Health and Technology · doi:10.1007/s12553-023-00806-7 · Cited by 69
“Some attempts indicate that AI-driven language systems can be used to correct misconceptions, to inform patients on cancer [87], and may be of further use in clinical scenarios [88].”
BackgroundGolan, R., Ripps, S. J., Reddy, R., Loloi, J., Bernstein, A., Connelly, Z. M., Golan, N. S., & Ramasamy, R. (2023) · Cureus · doi:10.7759/cureus.42214 · Cited by 79
“ChatGPT imitates and interprets human language with high precision and has been demonstrated to perform well for a variety of broad purposes, including performance on standardized examinations, such as the USMLE Step 1, research queries, drug development, and discovery [7-13].”
BackgroundA systematic review and meta-analysis on ChatGPT and its utilization in medical and dental research
Bagde, H., Dhopte, A., Alam, M. K., & Basri, R. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e23050 · Cited by 66
“The study by Johnson SB [33] et al. assessed ChatGPT ’ s accuracy in answering queries about cancer and reported an accuracy of 96.9 % for the obtained responses and 100 % for completeness.”
BackgroundInfluentialLahat, A., Sharif, Κ., Zoabi, N., Patt, Y. S., Sharif, Y., Fisher, L., Shani, U., Arow, M., Levin, R., & Klang, E. (2024) · Journal of Medical Internet Research · doi:10.2196/54571 · Cited by 74
“5, were recognized for generating human-like text and have potential across various fields, including medicine, where they can assist in diagnostics, therapeutic planning, telehealth, and patient education [5-15].”
Haver, H. L., Gupta, A., Ambinder, E. B., Bahl, M., Oluyemi, E. T., Jeudy, J. J., & Yi, P. H. (2024) · Radiology Imaging Cancer · doi:10.1148/rycan.230086 · Cited by 61
de Hond, A., Leeuwenberg, T., Bartels, R., van Buchem, M., Kant, I., Moons, K. G., & van Smeden, M. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00111-0 · Cited by 53
Abi-Rafeh, J., Xu, H. H., Kazan, R., Tevlin, R., & Furnas, H. (2024) · Aesthetic Surgery Journal · doi:10.1093/asj/sjad260 · Cited by 49
Shao, C.-Y., Li, H., Liu, X.-L., Li, C., Yang, L.-Q., Zhang, Y.-J., Luo, J., & Zhao, J. (2023) · Interactive Journal of Medical Research · doi:10.2196/46900 · Cited by 49
Makhortykh, M., Zucker, E. M., Simon, D. J., Bultmann, D., & Ulloa, R. (2023) · Discover Artificial Intelligence · doi:10.1007/s44163-023-00072-6 · Cited by 48
“…is yet to be studied, generative AI platforms, such as chatGPT, have demonstrated the ability to identify whether claims entered by the users are false in relation to diverse subjects [34] and health-related matters in particular [41], and to evaluate the degree of news outlets’ credibility [94].”
BackgroundAugmented Reality for Maintenance Tasks with ChatGPT for Automated Text-to-Action
Xu, F., Nguyen, T., & Du, J. (2024) · Journal of Construction Engineering and Management · doi:10.1061/jcemd4.coeng-14142 · Cited by 58
(2023) · International Research Journal of Modernization in Engineering Technology and Science · doi:10.56726/irjmets45213 · Cited by 45
Kim, H.-W., Shin, D.-H., Kim, J., Lee, G.-H., & Cho, J. W. (2024) · Seizure: European Journal of Epilepsy · doi:10.1016/j.seizure.2023.11.013 · Cited by 48
“…in a study conducted in the United States on cancer information, ChatGPT demonstrated a 96.9 % accuracy in answering questions from the “ Common Cancer Myths and Misconceptions ” page, indicating that it generally provides satisfactory information on common cancer myths and misconceptions [18].”
InfluentialKim, J. H., Kim, S. K., Choi, J., & Lee, Y. (2024) · DIGITAL HEALTH · doi:10.1177/20552076241227132 · Cited by 47
Cherrez-Ojeda, I., Gallardo-Bastidas, J. C., Robles-Velasco, K., Osorio, M. F., Velez Leon, E. M., Leon Velastegui, M., Pauletto, P., Aguilar-Díaz, F. C., Squassi, A., González Eras, S. P., et al. (2024) · JMIR Medical Education · doi:10.2196/51757 · Cited by 46
Answering head and neck cancer questions: An assessment of ChatGPT responses
Wei, K., Fritz, C., & Rajasekaran, K. (2024) · American Journal of Otolaryngology · doi:10.1016/j.amjoto.2023.104085 · Cited by 46
Arça, D. Ö., Erdemir, İ., Kara, F., Shermatov, N., Odacioğlu, M., İbişoğlu, E., Hancı, F. B., Sağıroğlu, G., & Hancı, V. (2024) · Medicine · doi:10.1097/md.0000000000038352 · Cited by 60
Evaluation of ChatGPT’s responses to information needs and information seeking of dementia patients
Saeidnia, H. R., Kozak, M., Lund, B. D., & Hassanzadeh, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-61068-5 · Cited by 55
Readability and Information Quality in Cancer Information From a Free vs Paid Chatbot
Musheyev, D., Pan, A., Gross, P. W., Kamyab, D., Kaplinsky, P., Spivak, M., Bragg, M. A., Loeb, S., & Kabarriti, A. E. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.22275 · Cited by 48
A Systematic Review of ChatGPT and Other Conversational Large Language Models in Healthcare
Wang, L., Wan, Z., Ni, C., Song, Q., Li, Y., Clayton, E. W., Malin, B. A., & Yin, Z. (2024) · medRxiv · doi:10.1101/2024.04.26.24306390 · Cited by 41
“…biased output; Lee et al. [74] concerned health disparity may result from low readability made by the sophistication of LLM wording; and Johnson et al. [56] noted that LLM algorithms tend to reinforce the health disparity and to prevent LLM algorithms from exacerbating current disparity in health.”
BackgroundInnovating Healthcare: The Role of ChatGPT in Streamlining Hospital Workflow in the Future
Zheng, Y., Wang, L., Feng, B., Zhao, A., & Wu, Y. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03323-w · Cited by 45
“For instance, chatGPT can assist in the identification of tumors and abnormalities in medical images, which can help healthcare professionals to make informed treatment decisions [6].”
ChatGPT’s ability to comprehend and answer cirrhosis related questions in Arabic
Samaan, J. S., Yeo, Y. H., Ng, W. H., Ting, P.-S., Trivedi, H., Vipani, A., Yang, J. D., Liran, O., Spiegel, B., Kuo, A., et al. (2023) · Arab Journal of Gastroenterology · doi:10.1016/j.ajg.2023.08.001 · Cited by 35
Artificial intelligence across the cancer care continuum
Riaz, I. B., Khan, M. A., & Osterman, T. J. (2025) · Cancer · doi:10.1002/cncr.70050 · Cited by 40
Roster, K., Kann, R., Farabi, B., Gronbeck, C., Brownstone, N. D., & Lipner, S. R. (2024) · JMIR Dermatology · doi:10.2196/50163 · Cited by 37
Performance of large language models (LLMs) in providing prostate cancer information
Alasker, A., Alsalamah, S., Alshathri, N. N., Almansour, N., Alsalamah, F., Alghafees, M. A., Alkhamees, M. A., & Alsaikhan, B. H. (2024) · BMC Urology · doi:10.1186/s12894-024-01570-0 · Cited by 45
“[24, 25] Furthermore, Haifeng Song et al. demonstrated the extraordinary potential of LLMs in patient education and medical health consultations.”
BackgroundGarg, R. K., Urs, V. L., Agrawal, A. A., Chaudhary, S. K., Paliwal, V., & Kar, S. K. (2023) · medRxiv · doi:10.1101/2023.06.13.23291311 · Cited by 36
Gül, Ş., Erdemir, İ., Hancı, V., Aydoğmuş, E., & Erkoç, Y. S. (2024) · Medicine · doi:10.1097/md.0000000000038009 · Cited by 48
“[37] According to a study conducted by Musheyev et al [38] the responses provided by ChatGPT, perplexity, Chat Sonic, and Microsoft Bing AI on the topic of urological cancers were found to be fairly difficult to read, moderately difficult to understand, and lacked clear instructions for users on…”
MethodAn Entity Extraction Pipeline for Medical Text Records Using Large Language Models: Analytical Study
Wang, L., Ma, Y., Bi, W., Lv, H., & Li, Y. (2024) · Journal of Medical Internet Research · doi:10.2196/54580 · Cited by 46
“The advent of large language models (LLMs) has paved a new path for the dilemma in clinical text extraction [21-23].”
Rebitschek, F. G., Carella, A., Kohlrausch-Pazin, S., Zitzmann, M., Steckelberg, A., & Wilhelm, C. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01752-6 · Cited by 45
Svestkova, A., Huang, Y., & Šmahel, D. (2025) · Digital Health · doi:10.1177/20552076251360973 · Cited by 39
Assessing the accuracy and reliability of ChatGPT’s medical responses about thyroid cancer
Helvacı, B. C., Hepşen, S., Candemir, B., Boz, O., Durantaş, H., Housseın, M., & Çakal, E. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105593 · Cited by 35
Nishisako, S., Higashi, T., & Wakao, F. (2025) · JMIR Cancer · doi:10.2196/70176 · Cited by 46
“…[32], prostate cancer [33], liver cancer [34], kidney cancer [35], breast cancer [36,37], lung cancer [20], colon cancer, pancreatic cancer [38], cervical cancer [39], as well as topics like cancer treatment [13,40], ophthalmic information [11,41,42], and cardiovascular disease prevention [43].”
Scott, I. A., & Zuccon, G. (2024) · Internal Medicine Journal · doi:10.1111/imj.16393 · Cited by 39
Evaluation of Artificial Intelligence–generated Responses to Common Plastic Surgery Questions
Copeland‐Halperin, L. R., O’Brien, L., & Copeland, M. (2023) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000005226 · Cited by 30
Considering ChatGPT to be the first aid for young adults on mental health issues
Aminah, S., Hidayah, N. E. F., & Ramli, M. (2023) · Journal of Public Health · doi:10.1093/pubmed/fdad065 · Cited by 30
Toward Clinical-Grade Evaluation of Large Language Models
Moreno, A. C., & Bitterman, D. S. (2024) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2023.11.012 · Cited by 28
Kaiser, K. N., Hughes, A. J., Yang, A. D., Turk, A. A., Mohanty, S., Gonzalez, A. A., Patzer, R. E., Bilimoria, K. Y., & Ellis, R. J. (2024) · Journal of Surgical Oncology · doi:10.1002/jso.27821 · Cited by 29
Innovations in Medicine: Exploring ChatGPT’s Impact on Rare Disorder Management
Zampatti, S., Peconi, C., Megalizzi, D., Calvino, G., Trastulli, G., Cascella, R., Strafella, C., Caltagirone, C., & Giardina, E. (2024) · Genes · doi:10.3390/genes15040421 · Cited by 37
“In fact, despite much harmful information available online about cancer [60], the chatbot demonstrated good accuracy in its response to cancer myths [61].”
BackgroundChatGPT in Occupational Medicine: A Comparative Study with Human Experts
Padovan, M., Cosci, B., Petillo, A., Nerli, G., Porciatti, F., Scarinci, S., Carlucci, F., Dell’Amico, L., Meliani, N., Necciari, G., et al. (2024) · Bioengineering · doi:10.3390/bioengineering11010057 · Cited by 31
“Johnson et al. [36] assessed the precision of cancer-related information delivered by ChatGPT when compared to the responses from the National Cancer Institute (NCI).”
MethodApplicability of Online Chat-Based Artificial Intelligence Models to Colorectal Cancer Screening
Atarere, J. O., Naqvi, H., Haas, C. J., Adewunmi, C. Y., Bandaru, S. K., Allamneni, R., Ugonabo, O. S., Egbo, O. J., Umoren, M., & Kanth, P. (2024) · Digestive Diseases and Sciences · doi:10.1007/s10620-024-08274-3 · Cited by 27
“Other researchers have also found that ChatGPT provides accurate information about common cancer myths and mis-conceptions [21].”
Large Language Models in Oncology: Revolution or Cause for Concern?
Caglayan, A., Slusarczyk, W., Rabbani, R. D., Ghose, A., Papadopoulos, V. P., & Boussios, S. (2024) · Current Oncology · doi:10.3390/curroncol31040137 · Cited by 30
“Not soon after the release of ChatGPT, it was shown to be capable of providing responses to common cancer misconceptions that are accurate and similar to answers provided by the National Cancer Institute’s (NCI) ‘Common Cancer Myths and Misconceptions’ web page [50].”
Method“Good” and “Bad” Machine Agency in the Context of Human-AI Communication: The Case of ChatGPT
Brandtzaeg, P. B., You, Y., Wang, X., & Lao, Y. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-48057-7_1 · Cited by 25
Ostrowska, M., Kacała, P., Onolememen, D., Vaughan-Lane, K., Joseph, A. S., Ostrowski, A., Pietruszewska, W., Banaszewski, J., & Wróbel, M. (2024) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-024-08643-8 · Cited by 31
“Within the online communication environment, cancer mis-information and harmful information remain a serious concern [20].”
BackgroundSchmidl, B., Hütten, T., Pigorsch, S. U., Stögbauer, F., Hoch, C. C., Hussain, T., Wollenberg, B., & Wirth, M. (2024) · Frontiers in Oncology · doi:10.3389/fonc.2024.1455413 · Cited by 35
Alanezi, F. (2025) · Nutrition and Health · doi:10.1177/02601060241244563 · Cited by 24
“…many existing problems related to ill-health-related practices, myths, and misconceptions (Ashley et al., 2023; National Cancer Institute, 2023e; Skyler et al., 2023; Yin et al., 2007), but also signi fi cantly reduce the healthcare costs for both governments and public (Berkman et al., 2011).”
Floyd, W., Kleber, T. J., Carpenter, D. J., Pasli, M., Qazi, J. J., Huang, C. C., Leng, J. X., Ackerson, B. G., Pierpoint, M., Salama, J. K., et al. (2023) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2023.10.020 · Cited by 28
Exploring the role of ChatGPT in oncology: providing information and support for cancer patients
Cè, M., Chiarpenello, V., Bubba, A., Felisaz, P. F., Oliva, G., Irmici, G., & Cellina, M. (2024) · BioMedInformatics · doi:10.3390/biomedinformatics4020049 · Cited by 24
“The Internet has definitely transformed how patients navigate medical information, reshaping the dynamics of patient empowerment and communication with the doctor [2].”
BackgroundChatGPT’s adherence to otolaryngology clinical practice guidelines
Tessler, I., Wolfovitz, A., Alon, E. E., Gecel, N. A., Livneh, N., Zimlichman, E., & Klang, E. (2024) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-024-08634-9 · Cited by 27
Mohammadi, S. S., Khatri, A., Jain, T., Thng, Z. X., Yoo, W.-S., Yavari, N., Bazojoo, V., Mobasserian, A., Akhavanrezayat, A., Than, N. T. T., et al. (2024) · Ophthalmology Science · doi:10.1016/j.xops.2024.100594 · Cited by 24
The impact of generative artificial intelligence on socioeconomic inequalities and policy makingPreprint
Capraro, V., Lentsch, A., Acemoğlu, D., Akgün, S., Akhmedova, A., Bilancini, E., Bonnefon, J., Brañas‐Garza, P., Butera, L., Douglas, K. M., et al. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/6fd2y · Cited by 48
Self-citationComparing ChatGPT's and Surgeon's Responses to Thyroid-related Questions From Patients
Guo, S., Li, R., Li, G., Chen, W., Huang, J., He, L., Ma, Y., Wang, L., Zheng, H., Tian, C., et al. (2024) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgae235 · Cited by 24
Joshi, S., Ha, E., Amaya, A., Mendoza, M., Rivera, Y. M., & Singh, V. K. (2024) · JMIR Formative Research · doi:10.2196/60939 · Cited by 34
Camalan, B. V., Doluoğlu, S., Taraf, N. H., Günay, M. M., & Özlügedik, S. (2025) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-025-09524-4 · Cited by 23
“This result is supported by other publications in the literature, which have demonstrated discrepancies in ChatGPT performance across various medical fields [20–23], as well as considerable variability in accuracy rates [24, 25], including specific documentation within otolaryngology [26–29].”
The Role of Artificial Intelligence in Emergency Medicine: A Comprehensive Overview
Mijwil, M. M., Unogwu, O. J., & Kumar, K. (2023) · Mesopotamian Journal of Artificial Intelligence in Healthcare · doi:10.58496/mjaih/2023/001 · Cited by 25
Freitas, M. D., López‐Pintor, R. M., Santos‐Silva, A. R., Warnakulasuriya, S., & Dios, P. D. (2024) · Exploration of Digital Health Technologies · doi:10.37349/edht.2024.00032 · Cited by 20
Connors, C., Gupta, K., Khusid, J. A., Khargi, R., Yaghoubian, A. J., Levy, M., Gallante, B. E., Atallah, W. M., & Gupta, M. (2024) · Journal of Endourology · doi:10.1089/end.2023.0696 · Cited by 22
Grilo, A. M., Marques, C. P., Corte-Real, M., Carolino, E., & Caetano, M. (2025) · JMIR Cancer · doi:10.2196/63677 · Cited by 24
“Recently, artificial intelligence (AI)–powered chatbots such as ChatGPT have signified a potential paradigm shift in how patients with cancer can access a vast amount of medical information [1,3,4].”
Alhajahjeh, A. A., & Nazha, A. (2023) · Current Hematologic Malignancy Reports · doi:10.1007/s11899-023-00716-5 · Cited by 22
Finch, L. A., Broach, V. A., Feinberg, J., Al‐Niaimi, A., Abu‐Rustum, N. R., Zhou, Q. C., Iasonos, A. E., & Dennis, S. (2024) · Gynecologic Oncology · doi:10.1016/j.ygyno.2024.07.007 · Cited by 20
Kaiser, K. N., Hughes, A. J., Yang, A. D., Mohanty, S., Maatman, T. K., Gonzalez, A. A., Patzer, R. E., Bilimoria, K. Y., & Ellis, R. J. (2025) · Surgery · doi:10.1016/j.surg.2025.109267 · Cited by 22
Assessment of ChatGPT-3.5's Knowledge in Oncology: Comparative Study with ASCO-SEP Benchmarks
Odabashian, R., Bastin, D., Jones, G., Manzoor, M., Tangestaniapour, S., Assad, M., Lakhani, S., Odabashian, M., & McGee, S. (2024) · JMIR AI · doi:10.2196/50442 · Cited by 18
“Additionally, while a previous study by Johnson et al [35] demonstrated that ChatGPT can be used by patients to answer common cancer myths and questions, the questions used in this study were already featured on the National Cancer Institute’s webpage and were likely part of ChatGPT’s training data…”
MethodWhat is the impact of artificial intelligence-based chatbots on infodemic management?
Morita, P. P., Lotto, M., Kaur, J., Chumachenko, D., Oetomo, A., Espiritu, K. D., & Hussain, I. Z. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1310437 · Cited by 22
An assessment of ChatGPT’s responses to frequently asked questions about cervical and breast cancer
Ye, Z., Zhang, B., Zhang, K., González, M. J., Yan, H., Wu, T., Qu, Y., Jiang, Y., Xue, P., & Qiao, Y. (2024) · BMC Women s Health · doi:10.1186/s12905-024-03320-8 · Cited by 26
“However, ChatGPT’s answers exceeded this recommended grade level of 6th to 8th grade for popular science articles [17, 22], making them challenging for a significant portion of the population with limited literacy skills [23, 24].”
BackgroundResultInfluentialBased on Medicine, The Now and Future of Large Language Models
Su, Z., Tang, G., Huang, R., Yang, Q. W., Zhang, Z., & Dai, X. (2024) · Cellular and Molecular Bioengineering · doi:10.1007/s12195-024-00820-3 · Cited by 20
“Research shows that ChatGPT achieves an accuracy rate of up to 96.9% when answering questions on the "Common Cancer Myths and Misconceptions" web page [108].”
InfluentialEmpowering patients: how accurate and readable are large language models in renal cancer education
Halawani, A., Almehmadi, S. G., Alhubaishy, B. A., Alnefaie, Z. A., & Hasan, M. N. (2024) · Frontiers in Oncology · doi:10.3389/fonc.2024.1457516 · Cited by 22
Utilization of Generative Artificial Intelligence in Nursing Education: A Topic Modeling Analysis
Seo, W. J., & Kim, M. (2024) · Education Sciences · doi:10.3390/educsci14111234 · Cited by 19
ChatGPT and medicine: how AI language models are shaping the future and health related careers
Yan, M., Cerri, G. G., & de Moraes, F. Y. (2023) · Nature Biotechnology · doi:10.1038/s41587-023-02011-3 · Cited by 20
Trends in research on ChatGPT and adoption-related issues discussed in articles: a narrative review
Kim, S.-J. (2023) · Science Editing · doi:10.6087/kcse.321 · Cited by 24
Metadata
- Authors
- Johnson, S. B., King, A. J., Warner, E. L., Aneja, S., Kann, B. H., & Bylund, C. L.
- Affiliations
- University of Utah (US)
- Yale University (US)
- Brigham and Women's Hospital (US)
- University of Florida Health (US)
- Corresponding author
- Johnson, Skyler B
- Venue
- JNCI Cancer Spectrum
- Publisher
- Oxford University Press (OUP)
- Published
- 2023-03-01
- Volume
- 7
- Issue
- 2
- ISSN
- 2515-5091 (ISSN-L)
- PMC
- PMC10020140
- OpenAlex
- W4327681325
- Cited by
- 289
- References
- 6
- Citations by year
- 2026: 21 · 2025: 77 · 2024: 116 · 2023: 75
- FWCI
- 11.54
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- JNCI Cancer Spectrum (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Huntsman Cancer Institute
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Social Sciences › Health › Social Media in Health Education
- Keywords
- ChatGPT, cancer myths, cancer misconceptions, artificial intelligence, chatbots, health information accuracy, National Cancer Institute, information quality, readability
- MeSH terms
- Artificial Intelligence, Humans, Neoplasms, United States, National Cancer Institute (U.S.)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.