Mind + Machine: ChatGPT as a Basic Clinical Decisions Support Tool
Open accessAbstract
Background Generative artificial intelligence (AI) has integrated into various industries as it has demonstrated enormous potential in automating elaborate processes and enhancing complex decision-making. The ability of these chatbots to critically triage, diagnose, and manage complex medical conditions, remains unknown and requires further research. Objective This cross-sectional study sought to quantitatively analyze the appropriateness of ChatGPT (OpenAI, San Francisco, CA, US) in its ability to triage, synthesize differential diagnoses, and generate treatment plans for nine diverse but common clinical scenarios. Methods Various common clinical scenarios were developed. Each was input into ChatGPT, and the chatbot was asked to develop diagnostic and treatment plans. Five practicing physicians independently scored ChatGPT's responses to the clinical scenarios. Results The average overall score for the triage ranking was 4.2 (SD 0.7). The lowest overall score was for the completeness of the differential diagnosis at 4.1 (0.5). The highest overall scores were seen with the accuracy of the differential diagnosis, initial treatment plan, and overall usefulness of the response (all with an average score of 4.4). Variance among physician scores ranged from 0.24 for accuracy of the differential diagnosis to 0.49 for appropriateness of triage ranking. Discussion ChatGPT has the potential to augment clinical decision-making. More extensive research, however, is needed to ensure accuracy and appropriate recommendations are provided.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A 2023 study that directly compared ChatGPT and Google revealed that ChatGPT struggled with specific and nuanced queries, particularly in specialized fields like health care.
“A 2023 study that directly compared ChatGPT and Google revealed that ChatGPT struggled with specific and nuanced queries, particularly in specialized fields like health care [63].”
From Evolving Health Information–Seeking Behavior in the Context of Google AI Overviews, ChatGPT, and Alexa: Interview Study Using the Think-Aloud Protocol (2025) · cites this paper as [63]
The Materials and methods section describes testing ChatGPT-4 on nine clinical scenarios, not directly comparing it with Google. The Discussion notes that clinical nuance and prompt wording can affect evaluation, which partly supports the concern about nuanced healthcare queries. But the Results report relatively high scores overall, so the paper does not establish that ChatGPT generally struggled with such queries. (LLM, conf=0.84)
Experimental — accuracy not yet measuredThe use of the think-aloud protocol to observe search behavior has the potential for producing the Hawthorne effect, in which participants change their behavior because they know they are being observed.
“Second, our use of the think-aloud protocol to observe search behavior has the potential for producing the Hawthorne effect [63], in which participants change their behavior because they know they are being observed.”
From Evolving Health Information–Seeking Behavior in the Context of Google AI Overviews, ChatGPT, and Alexa: Interview Study Using the Think-Aloud Protocol (2025) · cites this paper as [63]
The Materials and methods section describes clinical scenarios entered into ChatGPT and graded by physicians; the Discussion addresses limitations of that evaluation. Neither discusses a think-aloud protocol, observed search behavior, or the Hawthorne effect (searched those terms and the claim's description of behavior changing under observation). Our copy therefore does not verify the point attributed to this source. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · see citing works
Medical error—the third leading cause of death in the US
Makary, M. A., & Daniel, M. (2016) · BMJ · doi:10.1136/bmj.i2139 · Cited by 3,264 · see citing works
Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., et al. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1838 · Cited by 2,613 · see citing works
Dave, T., Athaluri, S. A., & Singh, S. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1169595 · Cited by 1,204 · see citing works
Prompt Engineering with ChatGPT: A Guide for Academic Writers
Giray, L. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03272-4 · Cited by 776 · see citing works
Sarraju, A., Bruemmer, D., Iterson, E. H. V., Cho, L., Rodríguez, F., & Laffin, L. J. (2023) · JAMA · doi:10.1001/jama.2023.1044 · Cited by 455 · see citing works
Murray, N. M., Unberath, M., Hager, G. D., & Hui, F. K. (2019) · Journal of NeuroInterventional Surgery · doi:10.1136/neurintsurg-2019-015135 · Cited by 310 · see citing works
Will ChatGPT transform healthcare?
(2023) · Nature Medicine · doi:10.1038/s41591-023-02289-5 · Cited by 214 · see citing works
Comparison Between ChatGPT and Google Search as Sources of Postoperative Patient Instructions
Ayoub, N., Lee, Y.-J., Grimm, D. R., & Balakrishnan, K. (2023) · JAMA Otolaryngology–Head & Neck Surgery · doi:10.1001/jamaoto.2023.0704 · Cited by 98 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Large Language Models for Chatbot Health Advice Studies: A Systematic Review
Huo, B., Boyle, A., Marfo, N., Tangamornsuksan, W., Steen, J. P., McKechnie, T., Lee, Y., Mayol, J., Antoniou, S. A., Thirunavukarasu, A. J., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.57879 · Cited by 283 · see citing works
Moulaei, K., Yadegari, A., Baharestani, M., Farzanbakhsh, S., Sabet, B., & Reza Afrash, M. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105474 · Cited by 244 · see citing works
Wardle, C., Urbani, S., & Wang, E. (2025) · Journal of Medical Internet Research · doi:10.2196/79961 · Cited by 43 · see citing works
“A 2023 study that directly compared ChatGPT and Google revealed that ChatGPT struggled with specific and nuanced queries, particularly in specialized fields like health care [63].”
Kaboudi, N., Firouzbakht, S., Eftekhar, M. S., Fayazbakhsh, F., Joharivarnoosfaderani, N., Ghaderi, S., Dehdashti, M., Kia, Y. M., Afshari, M., Gharamaleki, M. V., et al. (2024) · medRxiv · doi:10.22037/aaem.v12i1.2384 · Cited by 42 · see citing works
Kunze, K. N., Varady, N. H., Mazzucco, M. R., Lu, A. Z., Chahla, J. A., Martin, R. K., Ranawat, A. S., Pearle, A. D., & Williams, R. J. (2024) · Arthroscopy The Journal of Arthroscopic and Related Surgery · doi:10.1016/j.arthro.2024.06.021 · Cited by 40 · see citing works
Evaluating the Reliability of ChatGPT for Health-Related Questions: A Systematic Review
Beheshti, M., Toubal, I. E., Alaboud, K., Almalaysha, M., Ogundele, O. B., Turabieh, H., Abdalnabi, N., Boren, S. A., Scott, G. J., & Dahu, B. M. (2025) · Informatics · doi:10.3390/informatics12010009 · Cited by 37 · see citing works
Kapsali, M. Z., Livanis, E., Tsalikidis, C., Oikonomou, P., Voultsos, P. P., & Tsaroucha, A. (2024) · Cureus · doi:10.7759/cureus.54759 · Cited by 27 · see citing works
Moglia, A., Georgiou, K. E., Cerveri, P., Mainardi, L. T., Satava, R. M., & Cuschieri, A. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10849-5 · Cited by 24 · see citing works
Forecasting the impact of artificial intelligence on clinical pharmacy practice
Wong, A., Flanagan, T., Covington, E. W., Nguyen, E., Linn, D. D., Brummel, G. L., Hoffmaster, B. S., Isaacs, D. M., & Kane‐Gill, S. L. (2025) · JACCP JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY · doi:10.1002/jac5.70004 · Cited by 19 · see citing works
Hack, S., Attal, R., Yogev, D., Alsleibi, S., Gvili, B., Shahal, D., & Glikson, E. (2025) · Family Practice · doi:10.1093/fampra/cmaf087 · Cited by 16 · see citing works
de Araujo, B. M. D. M., Freitas, P. F. J., Schröder, Â. G. D., Küchler, É. C., Baratto‐Filho, F., Westphalen, V. P. D., Carneiro, E., Neto, U. X. D. S., & de Araújo, C. M. (2024) · Journal of Endodontics · doi:10.1016/j.joen.2024.09.008 · Cited by 16 · see citing works
Hahne, J., & Carpenter, B. D. (2026) · Patient Education and Counseling · doi:10.1016/j.pec.2026.109492 · Cited by 12 · see citing works
Large Language Models in Neurosurgery
Ieva, A. D., Stewart, C., & Molina, E. S. (2024) · Advances in experimental medicine and biology · doi:10.1007/978-3-031-64892-2_11 · Cited by 11 · see citing works
Geracitano, J., Anderson, B., Coffel, M., Rosenzweig, M., Dorn, S. D., Khairat, S., & Conklin, J. (2025) · Advances in Health Information Science and Practice · doi:10.63116/vxul2925 · Cited by 10 · see citing works
ChatGPT4’s diagnostic accuracy in inpatient neurology: A retrospective cohort study
Cano-Besquet, S., Rice-Canetto, T. E., Abou‐El‐Hassan, H., Alarcon, S., Zimmerman, J., Issagholian, L., Salomon, N., Rojas, I., Dhahbi, J. M., & Neeki, M. M. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e40964 · Cited by 8 · see citing works
Evaluating the Utility of ChatGPT in Diagnosing and Managing Maxillofacial Trauma
Rothchild, E., Baker, C., Smith, I. T., Tanna, N., & Ricci, J. A. (2024) · Journal of Craniofacial Surgery · doi:10.1097/scs.0000000000010931 · Cited by 7 · see citing works
Lázaro, C., & Ángulo, C. (2024) · Algorithms · doi:10.3390/a17120591 · Cited by 6 · see citing works
Generative AI in healthcare: foundations, applications, challenges and future directions
Zheng, J., Li, B., Li, H., & Lu, Y. (2026) · Journal of Management Analytics · doi:10.1080/23270012.2026.2647940 · Cited by 5 · see citing works
Shaari, A. L., Bhalla, S. R., & Salehi, P. P. (2025) · Facial Plastic Surgery & Aesthetic Medicine · doi:10.1089/fpsam.2024.0376 · Cited by 4 · see citing works
An Evaluation on the Potential of Large Language Models for Use in Trauma Triage
Le, K., Chen, J., Mai, D., & Le, K. D. R. (2024) · Emergency Care and Medicine · doi:10.3390/ecm1040035 · Cited by 4 · see citing works
Leveraging chatbots for enhanced decision-making: a comprehensive literature review
Simelane, P. M., & Kittur, J. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1748544 · Cited by 2 · see citing works
Role of ChatGPT in antenatal care
Shetty, V., Gul, A., Patel, H., Salcedo, Y. E., Gurmessa, K., Bassi, R., Ullah, R., Khan, W. A., Irfan, R., & Hidayat, A. (2024) · World Journal of Advanced Research and Reviews · doi:10.30574/wjarr.2024.23.1.2045 · Cited by 2 · see citing works
Rothchild, E., Jung, G., Aiello, C., Tanna, N., & Ricci, J. A. (2025) · Journal of Hand and Microsurgery · doi:10.1016/j.jham.2025.100260 · Cited by 1 · see citing works
Escobar, S., Rhee, J., Zaki, H. A., Dibble, E. H., Ward, R. C., Lourenço, A. P., & Baird, G. L. (2026) · Academic Radiology · doi:10.1016/j.acra.2026.03.015 · Cited by 1 · see citing works
Bazieva, A., & Камчиева, А. (2025) · Вестник Ошского государственного университета Педагогика Психология · doi:10.52754/16948742_1(6)_12-2025
A Brief Survey of Generative AI for Healthcare Applications
Essadik, I., Badiy, M., Tammouch, I., Temghart, A. A., & Amounas, F. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-10895-1_1
Rathmann, R., Bozkurt, A., Kaindl, A., Piazza, A., & Garg, R. (2026) · Edition HMD · doi:10.1007/978-3-658-51795-3_19
Dwina, E., Kartika, A. P. T., Akbar, R., & Atmaji, W. A. (2026) · Jurnal Ilmiah Kesehatan (JIKA) · doi:10.36590/jika.v8i2.2042
- Authors
- Ayoub, M., Ballout, A. A., Zayek, R. A., & Ayoub, N. F.
- Affiliations
- Northwell Health (US)
- Corresponding author
- Ayoub, Noel F
- Venue
- Cureus
- Published
- 2023-08-18
- Volume
- 15
- Issue
- 8
- Pages
- e43690-e43690
- ISSN
- 2168-8184 (ISSN-L)
- PMC
- PMC10505276
- OpenAlex
- W4385984708
- Cited by
- 25
- References
- 9
- Citations by year
- 2026: 6 · 2025: 8 · 2024: 11
- FWCI
- 1.00
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Cureus (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Keywords
- ChatGPT, generative AI, clinical decision support, triage, differential diagnosis, treatment planning, chatbots, clinical scenarios
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
