Reliability of Medical Information Provided by ChatGPT: Assessment Against Clinical Guidelines and Patient Information Quality Instrument
Open accessPublished version of a preprint
Abstract
BACKGROUND: ChatGPT-4 is the latest release of a novel artificial intelligence (AI) chatbot able to answer freely formulated and complex questions. In the near future, ChatGPT could become the new standard for health care professionals and patients to access medical information. However, little is known about the quality of medical information provided by the AI. OBJECTIVE: We aimed to assess the reliability of medical information provided by ChatGPT. METHODS: Medical information provided by ChatGPT-4 on the 5 hepato-pancreatico-biliary (HPB) conditions with the highest global disease burden was measured with the Ensuring Quality Information for Patients (EQIP) tool. The EQIP tool is used to measure the quality of internet-available information and consists of 36 items that are divided into 3 subsections. In addition, 5 guideline recommendations per analyzed condition were rephrased as questions and input to ChatGPT, and agreement between the guidelines and the AI answer was measured by 2 authors independently. All queries were repeated 3 times to measure the internal consistency of ChatGPT. RESULTS: Five conditions were identified (gallstone disease, pancreatitis, liver cirrhosis, pancreatic cancer, and hepatocellular carcinoma). The median EQIP score across all conditions was 16 (IQR 14.5-18) for the total of 36 items. Divided by subsection, median scores for content, identification, and structure data were 10 (IQR 9.5-12.5), 1 (IQR 1-1), and 4 (IQR 4-5), respectively. Agreement between guideline recommendations and answers provided by ChatGPT was 60% (15/25). Interrater agreement as measured by the Fleiss κ was 0.78 (P<.001), indicating substantial agreement. Internal consistency of the answers provided by ChatGPT was 100%. CONCLUSIONS: ChatGPT provides medical information of comparable quality to available static internet information. Although currently of limited quality, large language models could become the future standard for patients and health care professionals to gather medical information.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The output generated by Chat GPT is dependent on the complexity and clarity of the input.
“The output generated by Chat GPT is dependent on the complexity and clarity of the input, continuous evaluation of generated output will be required to ensure safe implementation in clinical settings [32, 33].”
From Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review (2025) · cites this paper as [32]
The Discussion > Limitations (page unavailable) says the study used medical terms and that less technical vocabulary might have affected the results, supporting a possible effect of input wording. It does not establish that output depends on both the complexity and clarity of the input; the Discussion > Principal Findings instead reports that ChatGPT handled complex questions and spelling or grammatical errors with ease. (LLM, conf=0.78)
Experimental — accuracy not yet measuredIn practice, generative AI systems in health care have been found to experience integrity threats, such as generating disinformation and misinformation, and making biased decisions.
“In practice, generative AI systems in health care have been found to experiencing integrity threats, such as generating disinformation and misinformation, and making biased decisions [81].”
From Generative AI in medical practice: in-depth exploration of privacy and security challenges (2024) · cites this paper as [81]
The Results (Consistency of ChatGPT Answers to Guideline Recommendations; page unavailable) report that ChatGPT’s answers agreed with clinical guidelines in 15 of 25 cases, supporting concern about inaccurate medical information. The Discussion (Principal Findings; page unavailable) warns of a danger of perpetuating biases, but does not report observed biased decisions or establish that ChatGPT generated deliberate disinformation. (LLM, conf=0.87)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
EASL Clinical Practice Guidelines: Management of hepatocellular carcinoma
Galle, P. R., Forner, A., Llovet, J. M., Mazzaferro, V., Piscaglia, F., Raoul, J.-L., Schirmacher, P., & Vilgrain, V. (2018) · Journal of Hepatology · doi:10.1016/j.jhep.2018.03.019 · Cited by 11,406 · see citing works
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · see citing works
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023) · JMIR Medical Education · doi:10.2196/45312 · Cited by 2,078 · see citing works
ChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842 · cited 1× · see citing works
“…in academia and beyond, including maintenance of human verification, development of rules for accountability, and advocating for availability of the source code, while simultaneously embracing its potential and endorsing public discussion about the significance and impact of AI applications [9].”
ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.9 · Cited by 1,806 · cited 2× · see citing works
“Although development of the software began in the last decade, its emergence took the medical community by surprise [5].”
Artificial hallucinations in ChatGPT: implications in scientific writing
Alkaissi, H., & McFarlane, S. I. (2023) · Cureus · doi:10.7759/cureus.35179 · Cited by 1,540 · see citing works
Tran, K. B., Lang, J. J., Compton, K., Xu, R., Acheson, A., Henrikson, H. J., Kocarnik, J., Penberthy, L., Aali, A., Abbas, Q., et al. (2022) · The Lancet · doi:10.1016/s0140-6736(22)01438-6 · Cited by 959 · see citing works
Eysenbach, G. (2023) · JMIR Medical Education · doi:10.2196/46885 · Cited by 840 · see citing works
Can artificial intelligence help for scientific writing?
Salvagno, M., Taccone, F. S., & Gerli, A. G. (2023) · Critical Care · doi:10.1186/s13054-023-04380-2 · Cited by 827 · see citing works
Generating scholarly content with ChatGPT: Ethical challenges for medical publishing
Liebrenz, M., Schleifer, R., Buadze, A., Bhugra, D., & Smith, A. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00019-5 · Cited by 594 · see citing works
Hassani, H., & Silva, E. S. (2023) · Big Data and Cognitive Computing · doi:10.3390/bdcc7020062 · Cited by 401 · see citing works
Neoadjuvant therapy for pancreatic cancer
Springfeld, C., Ferrone, C. R., Katz, M. H. G., Philip, P. A., Hong, T. S., Hackert, T., Büchler, M. W., & Neoptolemos, J. (2023) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-023-00746-1 · Cited by 377 · see citing works
Moult, B., Franck, L. S., & Brady, H. (2004) · Health Expectations · doi:10.1111/j.1369-7625.2004.00273.x · Cited by 334 · see citing works
Health, T. L. D. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00023-7 · Cited by 257 · see citing works
National Institute for Health and Care Excellence (NICE)
(2018) · Palgrave Macmillan UK eBooks · doi:10.1007/978-1-349-94186-5_836 · Cited by 226 · see citing works
Measuring quality of patient information documents with an expanded EQIP scale
Charvet-Bérard, A., Chopard, P., & Perneger, T. (2008) · Patient Education and Counseling · doi:10.1016/j.pec.2007.11.018 · Cited by 111 · see citing works
Do ChatGPT and Other AI Chatbots Pose a Cybersecurity Risk?
Sebastian, G. (2023) · International Journal of Security and Privacy in Pervasive Computing · doi:10.4018/ijsppc.320225 · Cited by 88 · see citing works
Fan, K. S., Ghani, S., Machairas, N., Lenti, L., Fan, K. H., Richardson, D., Scott, A., & Raptis, D. A. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-040487 · Cited by 85 · see citing works
Self-citationChatGPT at Universities – The Least of Our ConcernsPreprint
Willems, J. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4334162 · Cited by 55 · see citing works
Persuading Patients Using Rhetoric to Improve Artificial Intelligence Adoption: Experimental Study
Sebastian, G., George, A., & Jackson Jr, G. (2023) · Journal of Medical Internet Research · doi:10.2196/41430 · Cited by 48 · see citing works
Quality Assessment of Information on Bariatric Surgery Websites
Vetter, D., Ruhwinkel, H., Raptis, D. A., & Bueter, M. (2017) · Obesity Surgery · doi:10.1007/s11695-017-2983-0 · Cited by 30 · see citing works
Self-citationGhani, S., Fan, K. S., Fan, K. H., Lenti, L., & Raptis, D. A. (2021) · Journal of Medical Internet Research · doi:10.2196/22618 · Cited by 28 · see citing works
Self-citationVos, T. (2020) · Oxford University Press eBooks · doi:10.1093/med/9780190082543.003.0002 · Cited by 26 · see citing works
Raptis, D. A., Sinanyan, M., Ghani, S., Soggiu, F., Gilliland, J. J., & Imber, C. J. (2019) · HPB · doi:10.1016/j.hpb.2019.03.355 · Cited by 22 · see citing works
Self-citationMethodological aspects of studying the mechanisms of consciousness
Tsytsarev, V. (2021) · Behavioural Brain Research · doi:10.1016/j.bbr.2021.113684 · Cited by 16 · see citing works
A quality assessment of online patient information regarding tonsillitis using the EQIP tool
Kwan, L. Y., Yip, H. C. A., Tan, S., & Fan, K. S. (2022) · International Journal of Pediatric Otorhinolaryngology · doi:10.1016/j.ijporl.2022.111224 · Cited by 10 · see citing works
Evaluating the Quality of Online Information Regarding Neoadjuvant Therapy for Pancreatic Cancer
Stevens, L., Guo, M. Z., Brown, Z. J., Ejaz, A. M., Pawlik, T. M., & Cloyd, J. M. (2022) · Journal of Gastrointestinal Cancer · doi:10.1007/s12029-022-00879-z · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [18]Cirrhosis in over 16s: assessment and management National Institute for Health and Care Excellence 2023-06-20 https://www.nice.org.uk/guidance/ng50/chapter/Recommendations 37797152
- [19]Pancreatic cancer in adults: diagnosis and management National Institute for Health and Care Excellence 2023-06-20 https://www.nice.org.uk/guidance/ng85/chapter/Recommendations-for-research 29505215
Cited by
Sorted by citations
Large language models in healthcare and medical domain: A review
Nazi, Z. A., & Peng, W. (2024) · Informatics · doi:10.3390/informatics11030057 · Cited by 415 · see citing works
Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs
Li, W., Chen, X., Deng, X., Wen, H., You, M., Liu, W., Li, Q., & Li, J. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01029-4 · Cited by 400 · see citing works
Generative AI in medical practice: in-depth exploration of privacy and security challenges
Chen, Y., & Esmaeilzadeh, P. (2024) · Journal of Medical Internet Research · doi:10.2196/53008 · Cited by 353 · see citing works
“In practice, generative AI systems in health care have been found to experiencing integrity threats, such as generating disinformation and misinformation, and making biased decisions [81].”
Large Language Models for Chatbot Health Advice Studies: A Systematic Review
Huo, B., Boyle, A., Marfo, N., Tangamornsuksan, W., Steen, J. P., McKechnie, T., Lee, Y., Mayol, J., Antoniou, S. A., Thirunavukarasu, A. J., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.57879 · Cited by 283 · see citing works
Moulaei, K., Yadegari, A., Baharestani, M., Farzanbakhsh, S., Sabet, B., & Reza Afrash, M. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105474 · Cited by 244 · see citing works
Wang, L., Wan, Z., Ni, C., Song, Q., Li, Y., Clayton, E. W., Malin, B., & Yin, Z. (2024) · Journal of Medical Internet Research · doi:10.2196/22769 · Cited by 226 · see citing works
Mohammad‐Rahimi, H., Ourang, S. A., Pourhoseingholi, M. A., Omid Dianat, A., Dummer, P. M. H., & Nosrat, A. (2023) · International Endodontic Journal · doi:10.1111/iej.14014 · Cited by 138 · see citing works
Will artificial intelligence drive the advancements in higher education? A tri-phased exploration
Kumar, S., Rao, P., Singhania, S., Verma, S., & Kheterpal, M. (2024) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2024.123258 · Cited by 131 · see citing works
Ayre, J., Mac, O., McCaffery, K., McKay, B. R., Liu, M., Shi, Y., Rezwan, A., & Dunn, A. G. (2024) · Journal of General Internal Medicine · doi:10.1007/s11606-023-08469-w · Cited by 124 · see citing works
Gibson, D., Jackson, S., Shanmugasundaram, R., Seth, I., Siu, A., Ahmadi, N., Kam, J., Mehan, N., Thanigasalam, R., Jeffery, N., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/55939 · Cited by 122 · see citing works
Kernberg, A., Gold, J. A., & Mohan, V. (2024) · Journal of Medical Internet Research · doi:10.2196/54419 · Cited by 120 · see citing works
“However, in another study by Walker et al [17] aimed at evaluating the reliability of medical information provided by ChatGPT-4, multiple iterations of their queries executed through the model yielded a remarkable 100% internal consistency among the generated outputs [17].”
Suicide Risk Assessments Through the Eyes of ChatGPT-3.5 Versus ChatGPT-4: Vignette Study
Levkovich, I., & Elyoseph, Z. (2023) · JMIR Mental Health · doi:10.2196/51232 · Cited by 119 · see citing works
Blease, C., Worthen, A., & Torous, J. (2024) · Psychiatry Research · doi:10.1016/j.psychres.2024.115724 · Cited by 113 · see citing works
Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review
Bracken, A., Reilly, C., Feeley, A., Sheehan, E., Merghani, K., & Feeley, I. (2025) · Journal of Medical Systems · doi:10.1007/s10916-025-02157-4 · Cited by 113 · see citing works
Assessment of artificial intelligence applications in responding to dental trauma
Özden, İ., Gökyar, M., Ozden, M. E., & Öveçoğlu, H. S. (2024) · Dental Traumatology · doi:10.1111/edt.12965 · Cited by 95 · see citing works
Deng, L., Wang, T., Yangzhang, Zhai, Z., Tao, W., Li, J., Zhao, Y., Luo, S., & Xu, J. (2024) · International Journal of Surgery · doi:10.1097/js9.0000000000001066 · Cited by 93 · see citing works
Chow, J. C. L., Wong, V., & Li, K. (2024) · BioMedInformatics · doi:10.3390/biomedinformatics4010047 · Cited by 87 · see citing works
Hancı, V., Ergün, B., Gül, Ş., Uzun, Ö., Erdemir, İ., & Hancı, F. B. (2024) · Medicine · doi:10.1097/md.0000000000039305 · Cited by 87 · see citing works
The Potential Applications and Challenges of ChatGPT in the Medical Field
Mu, Y.-L., & He, D. (2024) · International Journal of General Medicine · doi:10.2147/ijgm.s456659 · Cited by 67 · see citing works
Sallam, M., Barakat, M. M., & Sallam, M. (2024) · Interactive Journal of Medical Research · doi:10.2196/54704 · Cited by 88 · see citing works
Nasra, M., Jaffri, R., Pavlin‐Premrl, D., Kok, H. K., Khabaza, A., Barras, C. D., Slater, L., Yazdabadi, A., Moore, J. M., Russell, J. H., et al. (2024) · Internal Medicine Journal · doi:10.1111/imj.16607 · Cited by 83 · see citing works
Chow, J. C. L., & Li, K. (2024) · JMIR Bioinformatics and Biotechnology · doi:10.2196/64406 · Cited by 75 · see citing works
Çiçek, F. E., Ülker, M., Özer, M., & Kıyak, Y. S. (2024) · Postgraduate Medical Journal · doi:10.1093/postmj/qgae170 · Cited by 82 · see citing works
A bibliometric analysis of the rise of ChatGPT in medical research
Barrington, N. M., Gupta, N., Musmar, B., Doyle, D., Panico, N., Godbole, N., Reardon, T., & D’Amico, R. S. (2023) · Medical Sciences · doi:10.3390/medsci11030061 · Cited by 69 · see citing works
Can deepseek and ChatGPT be used in the diagnosis of oral pathologies?
Kaygisiz, Ö. F., & Teke, M. T. (2025) · BMC Oral Health · doi:10.1186/s12903-025-06034-x · Cited by 65 · see citing works
Lyu, W., Zhang, S., Chung, T., Sun, Y., & Zhang, Y. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100383 · Cited by 68 · see citing works
“On the other hand, recent work has also highlighted issues of the use of GenAI in higher education [60], such as accuracy [58], reliability [58, 115], and plagiarism [57].”
Yurdakurban, E., Topsakal, K. G., & Duran, G. S. (2023) · Journal of Stomatology Oral and Maxillofacial Surgery · doi:10.1016/j.jormas.2023.101757 · Cited by 63 · see citing works
Improving radiology workflow using ChatGPT and artificial intelligence
Meşe, İ., Taşlıçay, C. A., & Sivrioğlu, A. K. (2023) · Clinical Imaging · doi:10.1016/j.clinimag.2023.109993 · Cited by 58 · see citing works
Yang, J., Ardavanis, K. S., Slack, K. E., Fernando, N. D., Valle, C. J. D., & Hernandez, N. M. (2024) · The Journal of Arthroplasty · doi:10.1016/j.arth.2024.01.029 · Cited by 53 · see citing works
Pal, A., Wangmo, T., Bharadia, T., Ahmed-Richards, M., Bhanderi, M., Kachhadiya, R., Allemann, S. S., & Elger, B. S. (2025) · Patient Preference and Adherence · doi:10.2147/ppa.s527922 · Cited by 53 · see citing works
Hu, J.-M., Liu, F.-C., Chu, C.-M., & Chang, Y.-T. (2023) · Journal of Medical Internet Research · doi:10.2196/49385 · Cited by 50 · see citing works
Johnson, A. J., Singh, T. K., Gupta, A., Sankar, H., Gill, I., Shalini, M., & Mohan, N. (2024) · Dental Traumatology · doi:10.1111/edt.13000 · Cited by 48 · see citing works
Performance of AI chatbots on controversial topics in oral medicine, pathology, and radiology
Mohammad‐Rahimi, H., Khoury, Z. H., Alamdari, M. I., Rokhshad, R., Motie, P., Parsa, A., Tavares, T., Sciubba, J. J., Price, J. B., & Sultan, A. S. (2024) · Oral Surgery Oral Medicine Oral Pathology and Oral Radiology · doi:10.1016/j.oooo.2024.01.015 · Cited by 50 · see citing works
Zhou, S., Luo, X., Chen, C., Jiang, H., Yang, C., Ran, G., Yu, J., & Yin, C. (2024) · International Journal of Surgery · doi:10.1097/js9.0000000000001850 · Cited by 47 · see citing works
ChatGPT: Transforming Healthcare with AI
Neha, F., Bhati, D., Shukla, D. K., & Amiruzzaman, M. (2024) · AI · doi:10.3390/ai5040126 · Cited by 47 · see citing works
A Systematic Review of ChatGPT and Other Conversational Large Language Models in HealthcarePreprint
Wang, L., Wan, Z., Ni, C., Song, Q., Li, Y., Clayton, E. W., Malin, B., & Yin, Z. (2024) · medRxiv · doi:10.1101/2024.04.26.24306390 · Cited by 41 · see citing works
Readability analysis of ChatGPT's responses on lung cancer
Gencer, A. (2024) · Scientific Reports · doi:10.1038/s41598-024-67293-2 · Cited by 40 · see citing works
Esmailpour, H., Rasaie, V., Hemmati, Y. B., & Falahchai, M. (2025) · BMC Oral Health · doi:10.1186/s12903-025-05965-9 · Cited by 40 · see citing works
Can Artificial Intelligence Language Models Effectively Address Dental Trauma Questions?
Kuru, E., Aşık, A., & Demir, D. M. (2025) · Dental Traumatology · doi:10.1111/edt.13063 · Cited by 39 · see citing works
Reicher, L., Lutsker, G., Michaan, N., Grisaru, D., & Laskov, I. (2024) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.15869 · Cited by 38 · see citing works
Do ChatGPT and Gemini Provide Appropriate Recommendations for Pediatric Orthopaedic Conditions?
Pirkle, S., Yang, J., & Blumberg, T. J. (2024) · Journal of Pediatric Orthopaedics · doi:10.1097/bpo.0000000000002797 · Cited by 38 · see citing works
Salman, I. M., Ameer, O. Z., Khanfar, M. A., & Hsieh, Y.-H. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1495378 · Cited by 36 · see citing works
Medical accuracy of artificial intelligence chatbots in oncology: a scoping review
Chen, D. J., Avison, K., Alnassar, S., Huang, R. S., & Raman, S. (2025) · The Oncologist · doi:10.1093/oncolo/oyaf038 · Cited by 40 · see citing works
McMahon, H. V., & McMahon, B. D. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1287186 · Cited by 34 · see citing works
Assessing the Current Limitations of Large Language Models in Advancing Health Care Education
Kim, J., & Vajravelu, B. N. (2024) · JMIR Formative Research · doi:10.2196/51319 · Cited by 35 · see citing works
Deep Learning Applications in Pancreatic Cancer
Patel, H., Zanos, T. P., & Hewitt, D. B. (2024) · Cancers · doi:10.3390/cancers16020436 · Cited by 34 · see citing works
AI in Dental Radiology—Improving the Efficiency of Reporting With ChatGPT: Comparative Study
Stephan, D., Bertsch, A., Burwinkel, M., Vinayahalingam, S., Al‐Nawas, B., Kämmerer, P. W., & Thiem, D. G. E. (2024) · Journal of Medical Internet Research · doi:10.2196/60684 · Cited by 37 · see citing works
Evaluating the Reliability of ChatGPT for Health-Related Questions: A Systematic Review
Beheshti, M., Toubal, I. E., Alaboud, K., Almalaysha, M., Ogundele, O. B., Turabieh, H., Abdalnabi, N., Boren, S. A., Scott, G. J., & Dahu, B. M. (2025) · Informatics · doi:10.3390/informatics12010009 · Cited by 37 · see citing works
ChatGPT's performance in dentistry and allergyimmunology assessments: a comparative study.
Fuchs, A., Trachsel, T., Weiger, R., & Eggmann, F. (2023) · Swiss dental journal · doi:10.61872/sdj-2024-06-01 · Cited by 31 · see citing works
Scquizzato, T., Semeraro, F. F., Swindell, P., Simpson, R. F. G., Angelini, M., Gazzato, A., Sajjad, U., Bignami, E. G., Landoni, G., Keeble, T., et al. (2023) · Resuscitation · doi:10.1016/j.resuscitation.2023.110077 · Cited by 34 · see citing works
ChatGPT in Answering Queries Related to Lifestyle-Related Diseases and Disorders
Mondal, H., Dash, I., Mondal, S., & Behera, J. K. (2023) · Cureus · doi:10.7759/cureus.48296 · Cited by 32 · see citing works
Rebitschek, F. G., Carella, A., Kohlrausch-Pazin, S., Zitzmann, M., Steckelberg, A., & Wilhelm, C. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01752-6 · Cited by 32 · see citing works
The Future of Patient Education: AI-Driven Guide for Type 2 Diabetes
Hernandez, C. A., Gonzalez, A. E. V., Polianovskaia, A., Sanchez, R. A., Arce, V. M., Mustafa, A., Vypritskaya, E., Gutierrez, O. P., Bashir, M., & Sedeh, A. E. (2023) · Cureus · doi:10.7759/cureus.48919 · Cited by 30 · see citing works
Yan, Z., Liu, J., Fan, Y., Lu, S., Xu, D., Yang, Y., Wang, H., Mao, J., Tseng, H.-C., Chang, T.-H., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/62857 · Cited by 31 · see citing works
Large language models: a new frontier in paediatric cataract patient education
Dihan, Q. A., Chauhan, M. Z., Eleiwa, T. K., Brown, A. D., Hassan, A. K., Khodeiry, M. M., ElSheikh, R. H., Oke, I., Nihalani, B. R., VanderVeen, D. K., et al. (2024) · British Journal of Ophthalmology · doi:10.1136/bjo-2024-325252 · Cited by 29 · see citing works
Kamminga, N. C. W., Kievits, J. E. C., Plaisier, P. W., Burgers, J. S., van der Veldt, A. A. M., van den Brand, J. A. G. J., Mulder, M., Wakkee, M., Lugtenberg, M., & Nijsten, T. E. C. (2024) · British Journal of Dermatology · doi:10.1093/bjd/ljae377 · Cited by 29 · see citing works
McLean, A. L., & Hristidis, V. (2025) · Journal of Cancer Education · doi:10.1007/s13187-025-02592-4 · Cited by 29 · see citing works
Huo, B., Calabrese, E., Sylla, P., Kumar, S., Ignacio, R. C., Oviedo, R., Hassan, I., Slater, B. J., Kaiser, A., Walsh, D. S., et al. (2024) · Surgical Endoscopy · doi:10.1007/s00464-024-10807-w · Cited by 28 · see citing works
Stephan, D., Bertsch, A., Schumacher, S., Puladi, B., Burwinkel, M., Al‐Nawas, B., Kämmerer, P. W., & Thiem, D. G. E. (2025) · Journal of Medical Internet Research · doi:10.2196/73337 · Cited by 33 · see citing works
Guo, S., Song, Y., Chen, G., Han, H., Wu, H. Y., & Ma, J. (2025) · Digital Health · doi:10.1177/20552076251374121 · Cited by 29 · see citing works
Abuyaman, O. (2023) · JMIR Formative Research · doi:10.2196/49459 · Cited by 29 · see citing works
Kharko, A., McMillan, B., Hagström, J., Muli, I., Davidge, G., Hägglund, M., & Blease, C. (2024) · DIGITAL HEALTH · doi:10.1177/20552076241291384 · Cited by 27 · see citing works
Lang, S., Vitale, J. A., Galbusera, F., Fekete, T. F., Boissière, L., Charles, Y. P., Yücekul, A., Yılgör, Ç., Núñez-Pereira, S., Haddad, S., et al. (2024) · Spine Deformity · doi:10.1007/s43390-024-00955-3 · Cited by 27 · see citing works
Alkalbani, A. M., Alrawahi, A. S., Salah, A., Haghighi, V., Zhang, Y., Alkindi, S. S. A., & Sheng, Q. Z. (2025) · Information · doi:10.3390/info16060489 · Cited by 27 · see citing works
Vavekanand, R., Laghari, A. A., & Kumar, T. (2026) · Iran Journal of Computer Science · doi:10.1007/s42044-025-00360-7 · Cited by 29 · see citing works
Yan, W., Hu, B., Liu, Y., Li, C., & Song, C. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103867 · Cited by 27 · see citing works
Siu, A. H. Y., Gibson, D., Chiu, C., Kwok, A. M. F., Irwin, M., Christie, A., Koh, C., Keshava, A., Reece, M. M., Suen, M., et al. (2024) · Colorectal Disease · doi:10.1111/codi.17267 · Cited by 25 · see citing works
The utility of ChatGPT as a generative medical translator
Grimm, D. R., Lee, Y.-J., Hu, K., Liu, L., Garcia, O. S., Balakrishnan, K., & Ayoub, N. (2024) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-024-08708-8 · Cited by 24 · see citing works
Tong, L., Zhang, C., Liu, R., Yang, J. N., & Sun, Z. (2024) · Journal of Orthopaedic Surgery and Research · doi:10.1186/s13018-024-04996-2 · Cited by 24 · see citing works
Revolutionizing MASLD: How Artificial Intelligence Is Shaping the Future of Liver Care
Pugliese, N., Bertazzoni, A., Hassan, C., Schattenberg, J. M., & Aghemo, A. (2025) · Cancers · doi:10.3390/cancers17050722 · Cited by 23 · see citing works
Wu, Y., Zhang, Y., Xu, M., Chen, J., Xue, Y., & Zheng, Y. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-02972-2 · Cited by 23 · see citing works
AI algorithm for personalized resource allocation and treatment of hemorrhage casualties
Jin, X., Frock, A., Nagaraja, S., Wallqvist, A., & Jaques Reifman, A. (2024) · Frontiers in Physiology · doi:10.3389/fphys.2024.1327948 · Cited by 23 · see citing works
ChatGPT and oral cancer: a study on informational reliability
Çıtır, M. (2025) · BMC Oral Health · doi:10.1186/s12903-025-05479-4 · Cited by 22 · see citing works
Gao, Z., Li, L., Ma, S., Wang, Q., Hemphill, L., & Xu, R. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03385-w · Cited by 22 · see citing works
Buldur, M., & Sezer, B. (2024) · BMC Oral Health · doi:10.1186/s12903-024-04358-8 · Cited by 27 · see citing works
Comparing ChatGPT's and Surgeon's Responses to Thyroid-related Questions From Patients
Guo, S., Li, R., Li, G., Chen, W., Huang, J., He, L., Ma, Y., Wang, L., Zheng, H., Tian, C., et al. (2024) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgae235 · Cited by 21 · see citing works
Mastrokostas, P. G., Mastrokostas, L. E., Emara, A. K., Wellington, I. J., Ginalis, E. E., Houten, J. K., Khalsa, A. S., Saleh, A., Razi, A. E., & Ng, M. K. (2024) · Global Spine Journal · doi:10.1177/21925682241241241 · Cited by 21 · see citing works
Bosco, C., Otenen, E., Torres, J. O., Nguyen, V. V., Chheda, D., Peng, X., Jessup, N. M., Himes, A. K., Cureton, B., Lu, Y. Y.-F., et al. (2024) · JMIR Aging · doi:10.2196/60566 · Cited by 20 · see citing works
Shiraishi, M., Tanigawa, K., Tomioka, Y., Miyakuni, A., Moriwaki, Y., Yang, R., Oba, J., & Okazaki, M. (2024) · Aesthetic Plastic Surgery · doi:10.1007/s00266-024-04002-4 · Cited by 20 · see citing works
Large Language Models for Clinical Text Cleansing Enhance Medical Concept Normalization
Abdulnazar, A., Roller, R., Schulz, S., & Kreuzthaler, M. (2024) · IEEE Access · doi:10.1109/access.2024.3472500 · Cited by 25 · see citing works
Exploring the Utility of ChatGPT in Cleft Lip Repair Education
Mahedia, M., Rohrich, R. N., Sadiq, K. O., Bailey, L., Harrison, L. M., & Hallac, R. R. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14030993 · Cited by 19 · see citing works
Grilo, A. M., Marques, C. P., Corte-Real, M., Carolino, E., & Caetano, M. (2025) · JMIR Cancer · doi:10.2196/63677 · Cited by 19 · see citing works
Evaluating Chat Generative Pre-trained Transformer Responses to Common Pediatric In-toeing Questions
Amaral, J. Z., Schultz, R. J., Martin, B. M., Taylor, T. N., Touban, B. M., McGraw-Heinrich, J. A., McKay, S. D., Rosenfeld, S. B., & Smith, B. G. (2024) · Journal of Pediatric Orthopaedics · doi:10.1097/bpo.0000000000002695 · Cited by 19 · see citing works
Huo, B., Marfo, N. Y. A., Sylla, P. A., Calabrese, E., Kumar, S. S., Slater, B. J., Walsh, D. S., & Vosburg, R. W. (2024) · Surgical Endoscopy · doi:10.1007/s00464-024-11155-5 · Cited by 20 · see citing works
Kıyak, Y. S. (2024) · Medical Science Educator · doi:10.1007/s40670-024-02146-1 · Cited by 18 · see citing works
Zhiqiang, W., Li, X., Ma, C., & Zhang, Z. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1801829 · Cited by 18 · see citing works
Thongsri, N., Tripak, O., & Bao, Y.-K. (2024) · Interactive Technology and Smart Education · doi:10.1108/itse-01-2024-0001 · Cited by 13 · see citing works
Cao, Z., & Peng, L. (2024) · Journal of Information Science · doi:10.1177/01655515241297329 · Cited by 21 · see citing works
Çıracıoğlu, A. M., & Erdoğan, S. D. (2025) · European Journal of Orthopaedic Surgery & Traumatology · doi:10.1007/s00590-025-04198-4 · Cited by 17 · see citing works
Jeon, S., Lee, S., Kim, E. H., Eun, J., Lee, K., Lim, H., & Lee, J. (2025) · JMIR Formative Research · doi:10.2196/72553 · Cited by 17 · see citing works
Future Perspective of Risk Prediction in Aesthetic Surgery: Is Artificial Intelligence Reliable?
Duran, A., Çörtük, O., & Ok, B. (2024) · Aesthetic Surgery Journal · doi:10.1093/asj/sjae140 · Cited by 17 · see citing works
Tsai, C.-Y., Cheng, P., Deng, J., Jaw, F., & Yii, S.-C. (2024) · Digital Health · doi:10.1177/20552076241269538 · Cited by 17 · see citing works
Shamil, E., Ko, T. K., Fan, K. S., Schuster‐Bruce, J., Jaafar, M., Khwaja, S., Eynon‐Lewis, N., D'Souza, A. R., & Andrews, P. J. (2024) · Facial Plastic Surgery · doi:10.1055/a-2413-3675 · Cited by 17 · see citing works
Wu, H., Yao, S., Bao, H., Guo, Y., Xu, C., & Ma, J. (2025) · The Knee · doi:10.1016/j.knee.2025.06.007 · Cited by 16 · see citing works
Ahmed, H. S., & Thrishulamurthy, C. J. (2024) · European Journal of Ophthalmology · doi:10.1177/11206721241272251 · Cited by 15 · see citing works
Humanizing AI in Education: A Readability Comparison of LLM and Human-Created Educational Content
Rashid, M. M., Atılgan, N., Dobres, J., Day, S., Penkova, V., Küçük, M., Clapp, S. R., & Sawyer, B. D. (2024) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813241261689 · Cited by 15 · see citing works
Identifying healthcare needs with patient experience reviews using ChatGPT
Li, J., Yang, Y., Chen, R., Zheng, D., Pang, P. C., Lam, C. K., Wong, D., & Wang, Y. (2025) · PLoS ONE · doi:10.1371/journal.pone.0313442 · Cited by 15 · see citing works
Lund, B. D., Mannuru, N. R., katta, M., Hota, S. S. L. M., Pamukuntla, A., Uppala, S., Kola, S. M., & Mannuru, A. (2025) · Journal of Health Communication · doi:10.1080/10810730.2025.2533820 · Cited by 18 · see citing works
Hanai, A., Ishikawa, T., Kawauchi, S., Iida, Y., & Kawakami, E. (2024) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2023-100924 · Cited by 15 · see citing works
Piao, Y., Chen, H., Wu, S., Li, X., Li, Z., & Yang, D. (2024) · Digital Health · doi:10.1177/20552076241284771 · Cited by 13 · see citing works
- Authors
- Walker, H. L., Ghani, S., Kuemmerli, C., Nebiker, C. A., Müller, B. P., Raptis, D. A., & Staubli, S. M.
- Affiliations
- Royal Free London NHS Foundation Trust (GB)
- University of Basel (CH)
- Kantonsspital Aarau (CH)
- King Faisal Specialist Hospital & Research Centre (SA)
- Corresponding author
- Staubli, Sebastian Manuel
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2023-06-30
- Volume
- 25
- Pages
- e47479
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC10365578
- OpenAlex
- W4380997513
- Cited by
- 363
- References
- 27
- Citations by year
- 2026: 92 · 2025: 137 · 2024: 116 · 2023: 18
- FWCI
- 14.50
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Social Sciences › Health › Social Media in Health Education
- Keywords
- ChatGPT-4, large language models, health information quality, clinical guidelines, gallstone disease, pancreatitis, liver cirrhosis, pancreatic cancer, hepatocellular carcinoma, guideline agreement, patient information
- MeSH terms
- Artificial Intelligence, Health Personnel, Humans, Language, Reproducibility of Results, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
