ChatGPT-Generated Differential Diagnosis Lists for Complex Case–Derived Clinical Vignettes: Diagnostic Accuracy Evaluation
Open accessPublished version of a preprint
Abstract
Background: The diagnostic accuracy of differential diagnoses generated by artificial intelligence chatbots, including ChatGPT models, for complex clinical vignettes derived from general internal medicine (GIM) department case reports is unknown. Objective: This study aims to evaluate the accuracy of the differential diagnosis lists generated by both third-generation ChatGPT (ChatGPT-3.5) and fourth-generation ChatGPT (ChatGPT-4) by using case vignettes from case reports published by the Department of GIM of Dokkyo Medical University Hospital, Japan. Methods: We searched PubMed for case reports. Upon identification, physicians selected diagnostic cases, determined the final diagnosis, and displayed them into clinical vignettes. Physicians typed the determined text with the clinical vignettes in the ChatGPT-3.5 and ChatGPT-4 prompts to generate the top 10 differential diagnoses. The ChatGPT models were not specially trained or further reinforced for this task. Three GIM physicians from other medical institutions created differential diagnosis lists by reading the same clinical vignettes. We measured the rate of correct diagnosis within the top 10 differential diagnosis lists, top 5 differential diagnosis lists, and the top diagnosis. Results: In total, 52 case reports were analyzed. The rates of correct diagnosis by ChatGPT-4 within the top 10 differential diagnosis lists, top 5 differential diagnosis lists, and top diagnosis were 83% (43/52), 81% (42/52), and 60% (31/52), respectively. The rates of correct diagnosis by ChatGPT-3.5 within the top 10 differential diagnosis lists, top 5 differential diagnosis lists, and top diagnosis were 73% (38/52), 65% (34/52), and 42% (22/52), respectively. The rates of correct diagnosis by ChatGPT-4 were comparable to those by physicians within the top 10 (43/52, 83% vs 39/52, 75%, respectively; P=.47) and within the top 5 (42/52, 81% vs 35/52, 67%, respectively; P=.18) differential diagnosis lists and top diagnosis (31/52, 60% vs 26/52, 50%, respectively; P=.43) although the difference was not significant. The ChatGPT models’ diagnostic accuracy did not significantly vary based on open access status or the publication date (before 2011 vs 2022). Conclusions: This study demonstrates the potential diagnostic accuracy of differential diagnosis lists generated using ChatGPT-3.5 and ChatGPT-4 for complex clinical vignettes from case reports published by the GIM department. The rate of correct diagnoses within the top 10 and top 5 differential diagnosis lists generated by ChatGPT-4 exceeds 80%. Although derived from a limited data set of case reports from a single department, our findings highlight the potential utility of ChatGPT-4 as a supplementary tool for physicians, particularly for those affiliated with the GIM department. Further investigations should explore the diagnostic accuracy of ChatGPT by using distinct case materials beyond its training data. Such efforts will provide a comprehensive insight into the role of artificial intelligence in enhancing clinical decision-making.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
ChatGPT demonstrates strong performance on straightforward cases but reduced reliability in mixed acid-base disorders typical of critically ill ICU populations.
“ChatGPT demonstrates strong performance on straightforward cases but reduced reliability in mixed acid–base disorders typical of critically ill ICU populations9.”
From Structural error asymmetry and harm-weighted analysis of ChatGPT versus ICU Physicians in acid–base interpretation: a prospective observational study (2026) · cites this paper as [9]
In “Comparison With Prior Work” (page not provided), the paper reports lower ChatGPT-3.5 diagnostic accuracy for complex general-internal-medicine vignettes than for more common clinical presentations. It does not examine mixed acid–base disorders or critically ill ICU patients, so the claim’s specific application to those cases is unsupported. (LLM, conf=0.86)
Experimental — accuracy not yet measuredPerformance declines with increasing complexity.
“This confirms prior observations that performance declines with increasing complexity3,9.”
From Structural error asymmetry and harm-weighted analysis of ChatGPT versus ICU Physicians in acid–base interpretation: a prospective observational study (2026) · cites this paper as [9]
In “Comparison With Prior Work” (page not specified), the paper reports lower diagnostic accuracy for ChatGPT-3.5 on its complex vignettes than in a prior study of more common presentations, and largely attributes the difference to case complexity. That supports the observation for ChatGPT-3.5, but not an unqualified decline in performance across models or settings. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · see citing works
When to use the <scp>B</scp> onferroni correction
Armstrong, R. A. (2014) · Ophthalmic and Physiological Optics · doi:10.1111/opo.12131 · Cited by 3,031 · see citing works
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Lee, P., Bubeck, S., & Petro, J. (2023) · New England Journal of Medicine · doi:10.1056/nejmsr2214184 · Cited by 1,738 · see citing works
Improving Diagnosis in Health Care
Committee on Diagnostic Error in Health Care, Board on Health Care Services, Institute of Medicine, & The National Academies of Sciences, E. (2015) · doi:10.17226/21794 · Cited by 1,376 · see citing works
Artificial Intelligence and Machine Learning in Clinical Medicine, 2023
Haug, C. J., & Drazen, J. M. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2302038 · Cited by 1,215 · see citing works
ChatGPT and the future of medical writing
Biswas, S. (2023) · Radiology · doi:10.1148/radiol.223312 · Cited by 571 · see citing works
Accuracy of a Generative Artificial Intelligence Model in a Complex Diagnostic Challenge
Kanjee, Z., Crowe, B., & Rodman, A. (2023) · JAMA · doi:10.1001/jama.2023.8288 · Cited by 449 · see citing works
Hirosawa, T., Harada, Y., Yokose, M., Sakamoto, T., Kawamura, R., & Shimizu, T. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20043378 · Cited by 345 · see citing works
Self-citationDocumenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus
Dodge, J., Sap, M., Marasović, A., Agnew, W., Ilharco, G., Groeneveld, D., Mitchell, M., & Gardner, M. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.98 · Cited by 306 · see citing works
ChatGPT: Is this version good for healthcare and research?
Vaishya, R., Misra, A., & Vaish, A. (2023) · Diabetes & Metabolic Syndrome: Clinical Research & Reviews · doi:10.1016/j.dsx.2023.102744 · Cited by 293 · see citing works
Will ChatGPT transform healthcare?
(2023) · Nature Medicine · doi:10.1038/s41591-023-02289-5 · Cited by 214 · see citing works
Meunier, P.-Y., Raynaud, C., Guimaraes, E., Gueyffier, F., & Letrilliart, L. (2023) · The Annals of Family Medicine · doi:10.1370/afm.2908 · Cited by 149 · cited 1× · see citing works
“However, despite the potential of CDS systems to boost diagnostic accuracy and efficiency, they often increase clinicians’ workload [9], particularly due to the need for structured input data.”
BackgroundRiches, N., Panagioti, M., Alam, R., Cheraghi‐Sohi, S., Campbell, S., Esmail, A., & Bower, P. (2016) · PLoS ONE · doi:10.1371/journal.pone.0148991 · Cited by 139 · see citing works
To ChatGPT or not to ChatGPT? The Impact of Artificial Intelligence on Academic Publishing
Curtis, N. C., & ChatGPT (2023) · The Pediatric Infectious Disease Journal · doi:10.1097/inf.0000000000003852 · Cited by 138 · see citing works
Wikipedia and Medicine: Quantifying Readership, Editors, and the Significance of Natural Language
Heilman, J., & West, A. G. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4069 · Cited by 135 · see citing works
Wani, S. U. D., Khan, N. A., Thakur, G. S., Gautam, S. P., Ali, M., Alam, P., Alshehri, S., Ghoneim, M. M., & Shakeel, F. (2022) · Healthcare · doi:10.3390/healthcare10040608 · Cited by 128 · see citing works
ChatGPT in Scientific Writing: A Cautionary Tale
Zheng, H., & Zhan, H. (2023) · The American Journal of Medicine · doi:10.1016/j.amjmed.2023.02.011 · Cited by 104 · see citing works
Skinner, T., Scott, I., & Martin, J. (2016) · International Journal of General Medicine · doi:10.2147/ijgm.s96741 · Cited by 99 · see citing works
Triage Accuracy of Symptom Checker Apps: 5-Year Follow-up Evaluation
Schmieding, M. L., Kopka, M., Schmidt, K. F. R., Schulz-Niethammer, S., Balzer, F., & Feufel, M. A. (2022) · Journal of Medical Internet Research · doi:10.2196/31810 · Cited by 99 · see citing works
Assessing clinical reasoning: moving from in vitro to in vivo
Holmboe, E. S., & Durning, S. J. (2014) · Diagnosis · doi:10.1515/dx-2013-0029 · Cited by 83 · see citing works
Harada, Y., Katsukura, S., Kawamura, R., & Shimizu, T. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18042086 · Cited by 44 · see citing works
Self-citationAvoiding premature closure and reaching diagnostic accuracy: some key predictive factors
Krupat, E., Wormwood, J. B., Schwartzstein, R. M., & Richards, J. B. (2017) · Medical Education · doi:10.1111/medu.13382 · Cited by 37 · see citing works
Kawamura, R., Harada, Y., Sugimoto, S., Nagase, Y., Katsukura, S., & Shimizu, T. (2022) · JMIR Medical Informatics · doi:10.2196/35225 · Cited by 34 · see citing works
Self-citationChen, J. H., Dhaliwal, G., & Yang, D. (2022) · JAMA · doi:10.1001/jama.2022.13735 · Cited by 32 · see citing works
A Brief History of Clinical Decision Support
Greenes, R. A. (2014) · Elsevier eBooks · doi:10.1016/b978-0-12-398476-0.00002-6 · Cited by 17 · see citing works
Zong, M., & Krishnamachari, B. (2022) · arXiv · doi:10.48550/arxiv.2212.00857 · Cited by 13 · see citing works
Harada, Y., Otaka, Y., Katsukura, S., & Shimizu, T. (2023) · BMJ Quality & Safety · doi:10.1136/bmjqs-2022-015436 · Cited by 11 · see citing works
Self-citationEnhancing clinical reasoning with Chat Generative Pre-trained Transformer: a practical guide
Hirosawa, T., & Shimizu, T. (2023) · Diagnosis · doi:10.1515/dx-2023-0116 · Cited by 10 · see citing works
Self-citationHepatic portal venous gas after diving
Jinno, A., Hirosawa, T., & Shimizu, T. (2018) · BMJ Case Reports · doi:10.1136/bcr-2017-223844 · Cited by 7 · see citing works
Self-citationOpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
Flagged first, then by citations
Cited by
Sorted by citations
Large Language Models for Chatbot Health Advice Studies: A Systematic Review
Huo, B., Boyle, A., Marfo, N., Tangamornsuksan, W., Steen, J. P., McKechnie, T., Lee, Y., Mayol, J., Antoniou, S. A., Thirunavukarasu, A. J., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.57879 · Cited by 283 · see citing works
Takita, H., Kabata, D., Walston, S. L., Tatekawa, H., Saito, K., Tsujimoto, Y., Miki, Y., & Ueda, D. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01543-z · Cited by 203 · see citing works
Önder, Ç. E., Koç, G., Gökbulut, P., Taşkaldıran, I., & Kuşkonmaz, Ş. M. (2024) · Scientific Reports · doi:10.1038/s41598-023-50884-w · Cited by 126 · see citing works
Influence of believed AI involvement on the perception of digital medical advice
Reis, M., Reis, F., & Kunde, W. (2024) · Nature Medicine · doi:10.1038/s41591-024-03180-7 · Cited by 130 · see citing works
Hoppe, J. M., Auer, M. K., Strüven, A. K., Maßberg, S., & Stremmel, C. (2024) · Journal of Medical Internet Research · doi:10.2196/56110 · Cited by 95 · see citing works
Sallam, M., Barakat, M. M., & Sallam, M. (2024) · Interactive Journal of Medical Research · doi:10.2196/54704 · Cited by 88 · see citing works
Shan, G., Chen, X., Wang, C., Liu, L., Gu, Y., Jiang, H., & Shi, T. (2025) · JMIR Medical Informatics · doi:10.2196/64963 · Cited by 71 · see citing works
Shiferaw, M. W., Zheng, T., Winter, A., Mike, L. A., & Chan, L. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02824-5 · Cited by 75 · see citing works
Large Language Models in Medical Diagnostics: Scoping Review With Bibliometric Analysis
Su, H., Sun, Y., Li, R., Zhang, A., Yang, Y., Xiao, F., Duan, Z., Chen, J., Hu, Q., Yang, T., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/72062 · Cited by 66 · see citing works
Wang, L., Li, J., Zhuang, B., Huang, S., Fang, M., Wang, C., Li, W., Zhang, M., & Gong, S. (2025) · Journal of Medical Internet Research · doi:10.2196/64486 · Cited by 51 · see citing works
Chen, Y., Huang, X., Yang, F., Lin, H., Lin, H., Zheng, Z., Liang, Q., Zhang, J., & Li, X. (2024) · BMC Medical Education · doi:10.1186/s12909-024-06309-x · Cited by 50 · see citing works
Nia, M. F., Ahmadi, M., & Irankhah, E. (2025) · Frontiers in Dental Medicine · doi:10.3389/fdmed.2024.1456208 · Cited by 60 · see citing works
A large language model improves clinicians’ diagnostic performance in complex critical illness cases
Wu, X. C., Huang, Y., & He, Q. (2025) · Critical Care · doi:10.1186/s13054-025-05468-7 · Cited by 45 · see citing works
Analysis of ChatGPT Responses to Ophthalmic Cases: Can ChatGPT Think like an Ophthalmologist?
Chen, J., Reddy, A. J., Al-Sharif, E. M., Shoji, M. K., Kalaw, F. G. P., Eslani, M., Lang, P. Z., Arya, M., Koretz, Z. A., Bolo, K. A., et al. (2024) · Ophthalmology Science · doi:10.1016/j.xops.2024.100600 · Cited by 44 · see citing works
Silva, T. P., Andrade‐Bortoletto, M. F. S., Ocampo, T. S. C., Alencar‐Palha, C., Bornstein, M. M., Oliveira‐Santos, C., & Oliveira, M. L. (2024) · Clinical Oral Investigations · doi:10.1007/s00784-024-05587-5 · Cited by 40 · see citing works
Hirosawa, T., Harada, Y., Mizuta, K., Sakamoto, T., Tokumasu, K., & Shimizu, T. (2024) · JMIR Formative Research · doi:10.2196/59267 · Cited by 38 · see citing works
Self-citationDiagnostic accuracy of large language models in psychiatry
Gargari, O. K., Fatehi, F., Mohammadi, I., Firouzabadi, S. R., Shafiee, A., & Habibi, G. (2024) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2024.104168 · Cited by 37 · see citing works
Clinical, Research, and Educational Applications of ChatGPT in Dentistry: A Narrative Review
Puleio, F., Giudice, G. L., Bellocchio, A. M., Boschetti, C. E., & Giudice, R. L. (2024) · Applied Sciences · doi:10.3390/app142310802 · Cited by 39 · see citing works
Diagnostic Accuracy of a Custom Large Language Model on Rare Pediatric Disease Case Reports
Young, C. C., Enichen, E., Rivera, C., Auger, C. A., Grant, N., Rao, A. S., & Succi, M. D. (2024) · American Journal of Medical Genetics Part A · doi:10.1002/ajmg.a.63878 · Cited by 37 · see citing works
Hudon, A., Kiepura, B., Pelletier, M., & Phan, V. (2024) · JMIR Medical Education · doi:10.2196/54067 · Cited by 32 · see citing works
Ho, C., Tian, T., Ayers, A. T., Aaron, R. E., Phillips, V., Wolf, R. M., Mathioudakis, N. N., Dai, T., & Klonoff, D. C. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02757-z · Cited by 31 · see citing works
Ono, D., Dickson, D. W., & Koga, S. (2024) · Neuropathology and Applied Neurobiology · doi:10.1111/nan.12997 · Cited by 30 · see citing works
Radha Krishnan, R. P., Hung, E. H., Ashford, M., Edillo, C. E., Gardner, C., Hatrick, H. B., Kim, B., Lai, A. W. Y., Li, X., Zhao, Y. X., et al. (2024) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.16275 · Cited by 29 · see citing works
Hassanein, F. E. A., Barbary, A. E., Hussein, R. R., Ahmed, Y., Elguindy, J., Sarhan, S., & Abou‐Bakr, A. (2025) · Oral Diseases · doi:10.1111/odi.70007 · Cited by 28 · see citing works
Quality of ChatGPT-Generated Therapy Recommendations for Breast Cancer Treatment in Gynecology
Stalp, J. L., Denecke, A., Jentschke, M., Hillemanns, P., & Klapdor, R. (2024) · Current Oncology · doi:10.3390/curroncol31070284 · Cited by 26 · see citing works
Diagnostic performance of generative artificial intelligences for a series of complex case reports
Hirosawa, T., Harada, Y., Mizuta, K., Sakamoto, T., Tokumasu, K., & Shimizu, T. (2024) · Digital Health · doi:10.1177/20552076241265215 · Cited by 24 · see citing works
Self-citationChatGPT Performance in Diagnostic Clinical Microbiology Laboratory-Oriented Case Scenarios
Sallam, M., Al‐Salahat, K., & Al-Ajlouni, E. (2023) · Cureus · doi:10.7759/cureus.50629 · Cited by 26 · see citing works
Bias Sensitivity in Diagnostic Decision-Making: Comparing ChatGPT with Residents
Schmidt, H. G., Rotgans, J. I., & Mamede, S. (2024) · Journal of General Internal Medicine · doi:10.1007/s11606-024-09177-9 · Cited by 22 · see citing works
Cuevas‐Nunez, M., Silberberg, V. A., Arregui, M., Jham, B. C., Victoria, R. B., Koptseva, I., de Tejada, M. J. B. G., Posada-Caez, R., Gil-Manich, V., Bara-Casaus, J., et al. (2024) · Oral Surgery Oral Medicine Oral Pathology and Oral Radiology · doi:10.1016/j.oooo.2024.11.087 · Cited by 22 · see citing works
Bhasuran, B., Jin, Q., Xie, Y., Yang, C., Hanna, K., Costa, J., Shavor, C., Han, W., Lu, Z., & He, Z. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01556-8 · Cited by 25 · see citing works
Shikino, K., Shimizu, T., Otsuka, Y., Tago, M., Takahashi, H., Watari, T., Sasaki, Y., Iizuka, G., Tamura, H., Nakashima, K., et al. (2024) · JMIR Medical Education · doi:10.2196/58758 · Cited by 21 · see citing works
Self-citationFrom text to image: challenges in integrating vision into ChatGPT for medical image interpretation
Koga, S., & Du, W. (2024) · Neural Regeneration Research · doi:10.4103/nrr.nrr-d-24-00165 · Cited by 20 · see citing works
Bibliometric Analysis on ChatGPT Research with CiteSpace
Nan, D., Zhao, X., Chen, C., Sun, S., Lee, K. R., & Kim, J. H. (2025) · Information · doi:10.3390/info16010038 · Cited by 20 · see citing works
Accuracy Evaluation of GPT-Assisted Differential Diagnosis in Emergency Department
Shah-Mohammadi, F., & Finkelstein, J. (2024) · Diagnostics · doi:10.3390/diagnostics14161779 · Cited by 22 · see citing works
Sallam, M., Al‐Farajat, A., & Egger, J. (2024) · Jordan Medical Journal · doi:10.35516/jmj.v58i1.2285 · Cited by 16 · see citing works
Innovating dental diagnostics: ChatGPT's accuracy on diagnostic challenges
Danesh, A., Danesh, A., & Danesh, F. (2024) · Oral Diseases · doi:10.1111/odi.15082 · Cited by 18 · see citing works
Bridges, J. M. (2024) · Diagnosis · doi:10.1515/dx-2024-0033 · Cited by 17 · see citing works
Harada, Y., Sakamoto, T., Sugimoto, S., & Shimizu, T. (2024) · JMIR Formative Research · doi:10.2196/53985 · Cited by 17 · see citing works
Self-citationHassanein, F. E. A., Hussein, R. R., Almalahy, H. G., Sarhan, S., Ahmed, Y., & Abou‐Bakr, A. (2025) · Scientific Reports · doi:10.1038/s41598-025-28862-1 · Cited by 16 · see citing works
ChatGPT's Potential in Enhancing Physician Efficiency: A Japanese Case Study
Kaneda, Y., Takita, M., Hamaki, T., Ozaki, A., & Tanimoto, T. E. (2023) · Cureus · doi:10.7759/cureus.48235 · Cited by 16 · see citing works
Ćirković, A., & Katz, T. R. (2023) · JMIR Formative Research · doi:10.2196/51798 · Cited by 16 · see citing works
Generative artificial intelligence performs rudimentary structural biology modeling
Ille, A. M., Markosian, C., Burley, S. K., Mathews, M. B., Pasqualini, R., & Arap, W. (2024) · Scientific Reports · doi:10.1038/s41598-024-69021-2 · Cited by 16 · see citing works
Du, W., Jin, X., Harris, J. C., Brunetti, A., Johnson, E., Leung, O. J., Li, X., Walle, S., Yu, Q., Zhou, X. L., et al. (2024) · Annals of Diagnostic Pathology · doi:10.1016/j.anndiagpath.2024.152392 · Cited by 15 · see citing works
Mansoor, M. A., Ibrahim, A. M., Grindem, D., & Baig, A. (2025) · JMIRx Med · doi:10.2196/65263 · Cited by 15 · see citing works
Mizuta, K., Hirosawa, T., Harada, Y., & Shimizu, T. (2024) · Diagnosis · doi:10.1515/dx-2024-0027 · Cited by 14 · see citing works
Self-citationMascarenhas, M., Ribeiro, T., Agudo, B., Afonso, J. P. L., Mendes, F., Martins, M., Cardoso, P., Mota, J., Almeida, M. J., da Costa, A. M. P., et al. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14020572 · Cited by 14 · see citing works
Can artificial intelligence diagnose seizures based on patients' descriptions? A study of GPT ‐4
Ford, J., Pevy, N., Grünewald, R. A., Howell, S. J., & Reuber, M. (2025) · Epilepsia · doi:10.1111/epi.18322 · Cited by 14 · see citing works
Artificial intelligence in sleep medicine: assessing the diagnostic precision of ChatGPT-4
Patel, A., & Cheung, J. (2025) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.11732 · Cited by 12 · see citing works
Educational Utility of Clinical Vignettes Generated in Japanese by ChatGPT-4: Mixed Methods Study
Takahashi, H., Shikino, K., Kondo, T., Komori, A., Yamada, Y., Saita, M., & Naito, T. (2024) · JMIR Medical Education · doi:10.2196/59133 · Cited by 14 · see citing works
Large language model application in emergency medicine and critical care
Hwai, H., Ho, Y.-J., Wang, C., & Huang, C. (2024) · Journal of the Formosan Medical Association · doi:10.1016/j.jfma.2024.08.032 · Cited by 13 · see citing works
Ghanta, S. N., Alaref, S. J., Lala, A., Nadkarni, G. N., Ganatra, S., Dani, S. S., & Mehta, J. L. (2024) · Diagnostics · doi:10.3390/diagnostics14212393 · Cited by 11 · see citing works
Hu, X., Xu, D., Zhang, H., Tang, M., & Gao, Q. (2025) · The Spine Journal · doi:10.1016/j.spinee.2024.12.035 · Cited by 11 · see citing works
ChatGPT neglects a neglected disease
Nacher, M., Françoise, U., & Adenis, A. A. (2024) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(23)00750-8 · Cited by 9 · see citing works
The Utility and Limitations of Artificial Intelligence-Powered Chatbots in Healthcare
Hayat, J., Lari, M., Alherz, M., & Lari, A. (2024) · Cureus · doi:10.7759/cureus.73127 · Cited by 11 · see citing works
Jin, Z., Abola, R. E., Bargnes, V., Tsivitis, A., Rahman, S., Schwartz, J. A., Bergese, S. D., & Schabel, J. E. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1582096 · Cited by 9 · see citing works
Comparing the accuracy of large language models and prompt engineering in diagnosing realworld cases
Yao, G., Zhang, W. D., Zhu, Y., Wong, U.-K., Zhang, Y., Cui, Y., Shen, G., Li, Z., & Gao, H. Y. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106026 · Cited by 10 · see citing works
Mazzucchelli, M., Salzano, S., Caltabiano, R., Magro, G. G., Certo, F., Barbagallo, G. M. V., & Broggi, G. (2025) · Neuropathology · doi:10.1111/neup.70023 · Cited by 8 · see citing works
Wu, X., Huang, Y., & He, Q. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106088 · Cited by 8 · see citing works
Unveiling the Algorithm: The Role of Explainable Artificial Intelligence in Modern Surgery
Lopes, S., Mascarenhas, M., Fonseca, J., Fernandes, G., & Leite-Moreira, A. F. (2025) · Healthcare · doi:10.3390/healthcare13243208 · Cited by 10 · see citing works
Chen, M., Wu, Y., Ma, J., Jia, X., Gao, C., Zhao, F., & Qiao, Y. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02409-8 · Cited by 13 · see citing works
Integrating AI in medicine: Lessons from Chat-GPT's limitations in medical imaging
Koga, S., & Du, W. D. (2024) · Digestive and Liver Disease · doi:10.1016/j.dld.2024.02.014 · Cited by 8 · see citing works
The potential, limitations, and future of diagnostics enhanced by generative artificial intelligence
Hirosawa, T., & Shimizu, T. (2024) · Diagnosis · doi:10.1515/dx-2024-0095 · Cited by 7 · see citing works
Self-citationDiagnostic efficacy of large language models in the pediatric emergency department: a pilot study
Monte, F. D., Barolo, R., Circhetta, M., Delmonaco, A. G., Castagno, E., Pivetta, E. E., Bergamasco, L., Franco, M., Olmo, G., & Bondone, C. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1624786 · Cited by 7 · see citing works
Evaluating Large Language Models for Burning Mouth Syndrome Diagnosis
Suga, T., Uehara, O., Abiko, Y., & Toyofuku, A. (2025) · Journal of Pain Research · doi:10.2147/jpr.s509845 · Cited by 7 · see citing works
Kutbi, D., Abou-Bakr, E., & Haidar, H. M. (2025) · JMIR Formative Research · doi:10.2196/84251 · Cited by 6 · see citing works
Medical Student Experiences With ChatGPT: National Cross-Sectional Study
Xu, A., Speakman, S., Piranio, V. S., Medina, R., Liu, M., Lamprecht, C. B., Abchee, N. J., & Brennan, M. M. (2026) · JMIR Formative Research · doi:10.2196/76838 · Cited by 6 · see citing works
Patel, A., Contractor, H., Heninger, H., Vallamchetla, S. K., Li, P., Tao, C., & Cheung, J. (2026) · Scientific Reports · doi:10.1038/s41598-025-34939-8 · Cited by 6 · see citing works
Naeem, A., Khan, O. A., Baqir, S. M., Jana, K. R., Shankar, P., Kaur, A., Zaaya, M., Sajid, F., Mohsin, F., Boadla, M. R., et al. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14041131 · Cited by 5 · see citing works
Evaluating the accuracy of ChatGPT model versions for giving care-seeking advice
Kopka, M., He, L., & Feufel, M. A. (2026) · Communications Medicine · doi:10.1038/s43856-026-01466-0 · Cited by 5 · see citing works
Hassanein, F. E. A., Ibrahim, S. S. A., Tomo, S., Alsahhaf, A., & Abou‐Bakr, A. (2026) · Odontology · doi:10.1007/s10266-026-01380-w · Cited by 7 · see citing works
ChatGPT: A game-changer in oral and maxillofacial surgery
Abdalla, A. Q., & Aziz, T. A. (2024) · Journal of Medicine Surgery and Public Health · doi:10.1016/j.glmedi.2024.100078 · Cited by 5 · see citing works
Yang, S., Luo, Y., Fotedar, N., Jiao, M., Rao, V. R., Ju, X., Wu, S., Xian, X., Sun, H., Karakis, I., et al. (2024) · medRxiv · doi:10.1101/2024.05.26.24307955 · Cited by 5 · see citing works
Nasef, H., Patel, H., Amin, Q., Baum, S., Ratnasekera, A., Ang, D., Havron, W. S., Nakayama, D., & Elkbuli, A. (2025) · The American Surgeon™ · doi:10.1177/00031348241256075 · Cited by 4 · see citing works
Khalid, M. U., El‐Kefraoui, C., Wang, A., Wang, J., Phang, P. T., Brown, C. J., Ghuman, A., Raval, M. J., & Karimuddin, A. A. (2025) · Surgery · doi:10.1016/j.surg.2025.109975 · Cited by 4 · see citing works
Hack, S., Craig, J. R., Lin, C.-F., Fu, C., Kwiatkowska, M. A., Kocum, P., Allevi, F., & Saibene, A. M. (2026) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-026-10040-2 · Cited by 4 · see citing works
Evaluating LLMs for Diagnosis Summarization
Santos, J., dos Santos, H. D. P., Ulbrich, A. H. D. P. S., Faccio, D., Tabalipa, F. D. O., Nogueira, R. F., & Costa, M. M. (2024) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc53108.2024.10782231 · Cited by 5 · see citing works
The Effect of LLM Assistance on Diagnostic Accuracy: A Systematic Review and Meta-AnalysisPreprint
Tretow, I., Ergon, E., Schwebel, M., Treffers, T., Welpe, I. M., & Feuerriegel, S. (2025) · medRxiv · doi:10.64898/2025.12.11.25341476 · Cited by 3 · see citing works
Lightweight Denoising Diffusion Implicit Model for Medical Segmentation
Oh, R., & Gonsalves, T. (2025) · Electronics · doi:10.3390/electronics14040676 · Cited by 3 · see citing works
Szydlak, R., Kıyak, Y. S., Hege, I., Górski, S., Linglart, L., Shchudrova, T. S., Torre, D. M., & Kononowicz, A. A. (2025) · Medical Teacher · doi:10.1080/0142159x.2025.2583403 · Cited by 3 · see citing works
Fabre, B. L., Filho, M. A. F. M., Aguiar, P. N., da Costa, F. M., Gutierres, B. D. S., William, W. N., & del Giglio, A. (2024) · ESMO Real World Data and Digital Oncology · doi:10.1016/j.esmorw.2024.100042 · Cited by 3 · see citing works
Advancing dental diagnostics with OpenAI's o1-preview
Danesh, A., Danesh, F., & Danesh, F. (2025) · The Journal of the American Dental Association · doi:10.1016/j.adaj.2025.04.003 · Cited by 2 · see citing works
Performance of Microsoft Copilot in the Diagnostic Process of Pulmonary Embolism
Arslan, B., Sutaşır, M. N., & Altınbilek, E. (2025) · Western Journal of Emergency Medicine · doi:10.5811/westjem.24995 · Cited by 2 · see citing works
Accuracy of ChatGPT‐4 Plus in Providing Information on Oral Cancer Management
Yılmaz, B., Kahraman, E. N., Brennan, M. T., Grewal, A. S., & Aktaş, A. (2025) · Oral Diseases · doi:10.1111/odi.70110 · Cited by 3 · see citing works
Srinivasan, S., Ai, X., Lo, T. W. S., Gilson, A., Zou, M., Zou, K., Kim, H., Yang, M., Pushpanathan, K., Yew, S. M. E., et al. (2025) · Ophthalmology Science · doi:10.1016/j.xops.2025.101050 · Cited by 4 · see citing works
Kopka, M., & Feufel, M. A. (2026) · JMIR Biomedical Engineering · doi:10.2196/88053 · Cited by 2 · see citing works
Bayala, Y. L. T., Kaboré, F., Sougué, C., Ouédraogo, A., Zongo, E., Tiendrébeogo, W. J. S. Z., & Ouedraogo, D. (2026) · Health Informatics Journal · doi:10.1177/14604582261448687 · Cited by 2 · see citing works
Harnessing the Power of Artificial Intelligence in EducationPreprint
Tiwari, E. S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4747352 · Cited by 2 · see citing works
Gulen, D., Gözden, H. E., Ekin, S., & Ceylan, I. (2026) · Scientific Reports · doi:10.1038/s41598-026-44576-4 · Cited by 1 · see citing works
“This confirms prior observations that performance declines with increasing complexity 3,9 .”
The double-edged nature of ChatGPT in self-diagnosis
Koga, S. (2024) · Wiener klinische Wochenschrift · doi:10.1007/s00508-024-02343-3 · Cited by 1 · see citing works
Large language models and their performance for the diagnosis of histoplasmosis
Nacher, M., Françoise, U., & Adenis, A. A. (2025) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0013151 · Cited by 1 · see citing works
Brooks, J. S., Blankson, P., Campbell, P. M., Cowley, R. A., Yang, T. Y., Oseni, T. I. A., Rodriguez, A., & Idris, M. Y. (2025) · JMIR Formative Research · doi:10.2196/82487 · Cited by 1 · see citing works
Jagadeesh, Y., Rizvi, N., & Nair, M. K. (2026) · Oral · doi:10.3390/oral6010010 · Cited by 1 · see citing works
Physicians’ expectations of the use of conversational agents in healthcare: a qualitative study
Wutz, M., Söling, S., & Köberlein-Neu, J. (2026) · BMC Health Services Research · doi:10.1186/s12913-026-14321-8 · Cited by 1 · see citing works
Seo, D., Chung, J., Choi, Y., Shin, Y., & Park, W. (2026) · JMIR Medical Informatics · doi:10.2196/86145 · Cited by 1 · see citing works
Yu, C., Li, F., Zhang, N., Hu, H., Huang, H., Wang, J., Tao, Y., & Wu, Y. (2026) · JMIR Medical Education · doi:10.2196/87959 · Cited by 2 · see citing works
Evaluating ChatGPT's effectiveness and tendencies in Japanese internal medicine
Kaneda, Y., Tayuinosho, A., Tomoyose, R., Takita, M., Hamaki, T., Tanimoto, T. E., & Ozaki, A. (2024) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.14011 · Cited by 2 · see citing works
Esposito, E., Cardakli, N., Christoff, A., & Kraus, C. L. (2025) · Clinical ophthalmology · doi:10.2147/opth.s556186 · Cited by 1 · see citing works
Nacanabo, W. M., Seghda, T. A. A., Bayala, Y. L. T., Millogo, G. R. C., Thiam, A. T., Yaméogo, N. V., SAMADOULOUGOU, K. A., & Zabsonré, P. (2024) · Research Square · doi:10.21203/rs.3.rs-4578427/v1 · Cited by 1 · see citing works
WITHDRAWN: EpiSemoLLM: A Fine-tuned Large Language Model for Epileptogenic ZonePreprint
Yang, S., Luo, Y., Jiao, M., Fotedar, N., Rao, V. R., Ju, X., Wu, S., Xian, X., Sun, H., Karakis, I., et al. (2024) · medRxiv · doi:10.1101/2024.09.16.24313764 · Cited by 1 · see citing works
- Authors
- Hirosawa, T., Kawamura, R., Harada, Y., Mizuta, K., Tokumasu, K., Kaji, Y., Suzuki, T., & Shimizu, T.
- Affiliations
- Dokkyo Medical University (JP)
- Okayama University (JP)
- International University of Health and Welfare Narita Hospital (JP)
- Urasoe General Hospital (JP)
- Corresponding author
- Hirosawa, Takanobu
- ORCID
- Venue
- JMIR Medical Informatics
- Publisher
- JMIR Publications Inc.
- Published
- 2023-10-09
- Volume
- 11
- Pages
- e48808
- ISSN
- 2291-9694 (ISSN-L)
- OpenAlex
- W4386710076
- Cited by
- 130
- References
- 30
- Citations by year
- 2026: 34 · 2025: 43 · 2024: 48 · 2023: 4
- FWCI
- 5.15
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Medical Informatics (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Japan Society for the Promotion of Science (22K10421)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- ChatGPT, differential diagnosis, diagnostic accuracy, clinical vignettes, artificial intelligence in healthcare, ChatGPT-3.5, ChatGPT-4, general internal medicine, physician comparison, large language models, clinical decision support, physician-patient communication, medical education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
