Artificial intelligence in healthcare education: evaluating the accuracy of ChatGPT, Copilot, and Google Gemini in cardiovascular pharmacology
Open accessAbstract
Background: Artificial intelligence (AI) is revolutionizing medical education; however, its limitations remain underexplored. This study evaluated the accuracy of three generative AI tools—ChatGPT-4, Copilot, and Google Gemini—in answering multiple-choice questions (MCQ) and short-answer questions (SAQ) related to cardiovascular pharmacology, a key subject in healthcare education. Methods: Using free versions of each AI tool, we administered 45 MCQs and 30 SAQs across three difficulty levels: easy, intermediate, and advanced. AI-generated answers were reviewed by three pharmacology experts. The accuracy of MCQ responses was recorded as correct or incorrect, while SAQ responses were rated on a 1–5 scale based on relevance, completeness, and correctness. Results: ChatGPT, Copilot, and Gemini demonstrated high accuracy scores in easy and intermediate MCQs (87–100%). While all AI models showed a decline in performance on the advanced MCQ section, only Copilot (53% accuracy) and Gemini (20% accuracy) had significantly lower scores compared to their performance on easy-intermediate levels. SAQ evaluations revealed high accuracy scores for ChatGPT (overall 4.7 ± 0.3) and Copilot (overall 4.5 ± 0.4) across all difficulty levels, with no significant differences between the two tools. In contrast, Gemini’s SAQ performance was markedly lower across all levels (overall 3.3 ± 1.0). Conclusion: ChatGPT-4 demonstrates the highest accuracy in addressing both MCQ and SAQ cardiovascular pharmacology questions, regardless of difficulty level. Copilot ranks second after ChatGPT, while Google Gemini shows significant limitations in handling complex MCQs and providing accurate responses to SAQ-type questions in this field. These findings can guide the ongoing refinement of AI tools for specialized medical education.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Appropriate use of AI devices can enhance academic achievement, boost interest in learning, and improve clinical decision-making.
“The initial empirical data indicate that in case of appropriate use, AI devices have the ability to enhance academic achievement, boost the interest in learning, and improve clinical decision-making.”
From HOW AI TOOLS (LIKE CHATGPT/GEMINI) AFFECT CRITICAL THINKING IN NURSING STUDENTS. A SYSTEMATIC REVIEW (2026) · cites this paper as [5]
The Discussion > Perspectives says AI tools can supplement basic factual learning, which supports a limited educational benefit. It does not report improved academic achievement or interest in learning, and it cautions that AI-generated information requires critical evaluation in clinical decision-making. The study assessed answers to pharmacology questions, not those learner or clinical outcomes. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · see citing works
Multiple comparisons among means
Dunn, O. J. (1961) · Journal of the American Statistical Association · doi:10.1080/01621459.1961.10482090 · Cited by 5,006 · see citing works
Artificial intelligence: A powerful paradigm for scientific research
Xu, Y., Liu, X., Cao, X., Huang, C., Liu, E., Qian, S., Liu, X., Wu, Y., Dong, F., Qiu, C.-W., et al. (2021) · The Innovation · doi:10.1016/j.xinn.2021.100179 · Cited by 1,726 · see citing works
Prompt Engineering with ChatGPT: A Guide for Academic Writers
Giray, L. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03272-4 · Cited by 776 · see citing works
Boscardin, C. K., Gin, B., Golde, P. B., & Hauer, K. E. (2024) · Academic Medicine · doi:10.1097/acm.0000000000005439 · Cited by 533 · cited 1× · see citing works
“…in medical, pharmacy, nursing, dental and allied health programs to assist with studying, personal tutoring, exam preparation, and completing assignments, due to their remarkable ability to provide real-time assistance and tailored information relevant to coursework and research activities (1, 2).”
Walker, H. L., Ghani, S., Kuemmerli, C., Nebiker, C. A., Müller, B. P., Raptis, D. A., & Staubli, S. M. (2023) · Journal of Medical Internet Research · doi:10.2196/47479 · Cited by 363 · see citing works
Nikolic, S., Daniel, S., Haque, R., Belkina, M., Hassan, G. M., Grundy, S., Lyden, S., Neal, P., & Sandison, C. (2023) · European Journal of Engineering Education · doi:10.1080/03043797.2023.2213169 · Cited by 336 · see citing works
Performance of GPT-3.5 and GPT-4 on the Japanese Medical Licensing Examination: Comparison Study
Takagi, S., Watari, T., Erabi, A., & Sakaguchi, K. (2023) · JMIR Medical Education · doi:10.2196/48002 · Cited by 271 · see citing works
Yu, P., Xu, H., Hu, X., & Deng, C. (2023) · Healthcare · doi:10.3390/healthcare11202776 · Cited by 267 · see citing works
Applications of Artificial Intelligence (AI) in Medical Education: A Scoping Review
Nagi, F., Salih, R., Alzubaidi, M., Shah, H., Alam, T., Shah, Z., & Househ, M. (2023) · Studies in Health Technology and Informatics · doi:10.3233/shti230581 · Cited by 135 · see citing works
Espejel, J. L., Ettifouri, E. H., Alassan, M. S. Y., Chouham, E. M., & Dahhane, W. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100032 · Cited by 131 · see citing works
Google DeepMind’s gemini AI versus ChatGPT: a comparative analysis in ophthalmology
Masalkhi, M., Ong, J., Waisberg, E., & Lee, A. G. (2024) · Eye · doi:10.1038/s41433-024-02958-w · Cited by 130 · see citing works
Evaluating language models for mathematics through interactions
Collins, K. M., Jiang, A. Q., Frieder, S., Wong, L., Zilka, M., Bhatt, U., Lukasiewicz, T., Wu, Y.-H., Tenenbaum, J., Hart, W., et al. (2023) · Proceedings of the National Academy of Sciences of the United States of America · doi:10.1073/pnas.2318124121 · Cited by 130 · see citing works
Is ChatGPT accurate and reliable in answering questions regarding head and neck cancer?
Kuşçu, O., Pamuk, E., Süslü, N., & Hosal, S. A. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1256459 · Cited by 124 · see citing works
Multi-Source and Multi-modal Deep Network Embedding for Cross-Network Node Classification
Yang, H., He, H. X., Zhang, W., Wang, Y., & Lin, J. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3653304 · Cited by 112 · see citing works
Rossettini, G., Rodeghiero, L., Corradi, F., Cook, C., Pillastrini, P., Turolla, A., Castellini, G., Chiappinotto, S., Gianola, S., & Palese, A. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05630-9 · Cited by 103 · see citing works
Roos, J., Kasapovic, A., Jansen, T., & Kaczmarczyk, R. (2023) · JMIR Medical Education · doi:10.2196/46482 · Cited by 98 · see citing works
Cheong, R. C. T., Unadkat, S. N., Mcneillis, V., Williamson, A., Joseph, J., Randhawa, P. S., Andrews, P. J., & Paleri, V. (2023) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-023-08319-9 · Cited by 94 · see citing works
Alhur, A. (2024) · Cureus · doi:10.7759/cureus.57795 · Cited by 82 · see citing works
Pharmacology education in the medical curriculum: Challenges and opportunities for improvement
Fasinu, P. S., & Wilborn, T. W. (2024) · Pharmacology Research & Perspectives · doi:10.1002/prp2.1178 · Cited by 80 · see citing works
Kaftan, A. N., Hussain, M. K., & Naser, F. H. (2024) · Scientific Reports · doi:10.1038/s41598-024-58964-1 · Cited by 47 · see citing works
Correctness Comparison of ChatGPT ‐4, Gemini, Claude‐3, and Copilot for Spatial Tasks
Hochmair, H. H., Juhász, L., & Kemp, T. (2024) · Transactions in GIS · doi:10.1111/tgis.13233 · Cited by 44 · see citing works
Tepe, M., & Emekli, E. (2024) · Cureus · doi:10.7759/cureus.59960 · Cited by 43 · see citing works
Chlorogiannis, D., Apostolos, A., Chlorogiannis, A., Palaiodimos, L., Giannakoulas, G. A., Pargaonkar, S., Xesfingi, S., & Kokkinidis, D. G. (2023) · Healthcare · doi:10.3390/healthcare11212906 · Cited by 41 · see citing works
Kaiser, K. N., Hughes, A. J., Yang, A. D., Turk, A. A., Mohanty, S., Gonzalez, A. A., Patzer, R. E., Bilimoria, K. Y., & Ellis, R. J. (2024) · Journal of Surgical Oncology · doi:10.1002/jso.27821 · Cited by 29 · see citing works
Comparison of the Audiological Knowledge of Three Chatbots: ChatGPT, Bing Chat, and Bard
Jędrzejczak, W. W., & Kochanek, K. (2024) · Audiology and Neurotology · doi:10.1159/000538983 · Cited by 23 · see citing works
A Quantitative Assessment of ChatGPT as a Neurosurgical Triaging Tool
Ward, M., Unadkat, P., Toscano, D., Kashanian, A., Lynch, D. G., Horn, A. C., D’Amico, R. S., Mittler, M. A., & Baum, G. R. (2024) · Neurosurgery · doi:10.1227/neu.0000000000002867 · Cited by 22 · see citing works
Naz, R., Akacı, O., Erdoğan, H., & Açıkgöz, A. (2024) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.14084 · Cited by 20 · see citing works
Fabijan, A., Zawadzka-Fabijan, A., Fabijan, R., Zakrzewski, K. M., Nowosławska, E., & Polis, B. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13144013 · Cited by 16 · see citing works
Leon, M., Ruaengsri, C., Pelletier, G., Bethencourt, D., Shibata, M., Flores, M. Q., & Shudo, Y. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13216543 · Cited by 12 · see citing works
Szczesniewski, J. J., Alba, A. R., Castro, P. M. R., Lorenzo-Gómez, M. F., González, J. P., & González, L. L. (2024) · Actas Urológicas Españolas (English Edition) · doi:10.1016/j.acuroe.2024.02.009 · Cited by 11 · see citing works
Assessing the Capability of Large Language Models in Naturopathy Consultation
Mondal, H., Komarraju, S., Sathyanath, D., & Muralidharan, S. (2024) · Cureus · doi:10.7759/cureus.59457 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Singal, A., & Goyal, S. (2025) · Surgical and Radiologic Anatomy · doi:10.1007/s00276-025-03707-8 · Cited by 14 · see citing works
Thesen, T., Lalchandani, R. R., Blumer, J., & Lee, M. W. (2025) · British Journal of Clinical Pharmacology · doi:10.1002/bcp.70119 · Cited by 8 · see citing works
Alzarea, A. I., Ishaqui, A. A., Maqsood, M. B., Alanazi, A. S., Alsaidan, A. A., Mallhi, T. H., Kumar, N., Imran, M., Alshahrani, S. M., Alhassan, H. H., et al. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1679153 · Cited by 6 · see citing works
Ordak, M., Adamczyk, J., Oskroba, A., Majewski, M., & Nasierowski, T. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14217563 · Cited by 4 · see citing works
A review on recent advancements of ChatGPT and datafication in healthcare applications
Jagatheesaperumal, S. K., Pandiyarajan, A., Boopathy, P., Deepa, N., Barreto, A. G., & de Albuquerque, V. D. (2025) · Comput. Biol. Medicine · doi:10.1016/j.compbiomed.2025.110885 · Cited by 4 · see citing works
Tang, Z., Su, Y., Hansen, P., & Zhang, P. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.104668 · Cited by 4 · see citing works
Keyif, B., Yurtçu, E., Başbuğ, A., & Göynümer, G. (2026) · Scientific Reports · doi:10.1038/s41598-026-40609-0 · Cited by 3 · see citing works
ZainAlAbdin, S., Kendakji, S., Alrahbi, F., Alazeezi, K., Jawas, M., Al-Shamsi, S. R., Almessabi, T., Zaki, N., & Aburuz, S. (2025) · Pharmacotherapy The Journal of Human Pharmacology and Drug Therapy · doi:10.1002/phar.70089 · Cited by 3 · see citing works
Diagnostic performance of four AI tools in pharmacology MCQs: Accuracy, sensitivity, and specificity
Al-Rahahleh, A. J., Rizik, M., Al-Ashwal, F. Y., & Abu-Farha, R. K. (2025) · PLoS ONE · doi:10.1371/journal.pone.0337688 · Cited by 2 · see citing works
Carou‐Senra, P., Rodríguez‐Pombo, L., Alvarez‐Lorenzo, C., & Goyanes, Á. (2026) · International Journal of Pharmaceutics · doi:10.1016/j.ijpharm.2026.126587 · Cited by 2 · see citing works
Dastani, M., Sajjadi, M. S., Yamout, B. I., Bafrani, M. A., & Nasirzadeh, A. (2026) · PLoS ONE · doi:10.1371/journal.pone.0346445 · Cited by 2 · see citing works
HOW AI TOOLS (LIKE CHATGPT/GEMINI) AFFECT CRITICAL THINKING IN NURSING STUDENTS. A SYSTEMATIC REVIEW
Din, A. L. (2026) · Journal of Medical & Health Sciences Review · doi:10.65035/650ytk03 · Cited by 1 · see citing works
Habib, D. R. S., Depala, K. S., Lin, J. Y., Le, S. D., McFall, N., Dewan, S., Huang, J. T., Habib, M. W. S., Bishay, A., Siebor, K., et al. (2026) · American Journal of Audiology · doi:10.1044/2025_aja-25-00216
Karakoyun, Z. N., Yörük, M. D., Özdemir, M. E., & Koşar, M. İ. (2026) · BMC Medical Education · doi:10.1186/s12909-026-09173-z · Cited by 1 · see citing works
Mensah, E. A., Swortzel, K. A., & Osman, N. (2026) · Journal of Agricultural Education · doi:10.5032/jae.v67i2.3302 · Cited by 1 · see citing works
Khosravi, M., Zamaninasab, Z., Khosravi, F., Attar, M., & Arab‐Zozani, M. (2026) · Health Science Reports · doi:10.1002/hsr2.72634 · Cited by 1 · see citing works
Aşan, H. (2025) · Uluslararası Sosyal Siyasal ve Mali Araştırmalar Dergisi · doi:10.70101/ussmad.1758962
Çoşkun, A., Elibol, V., İpek, O. M., Dinçer, E., Can, M., Bali, M., Oğulcan, A. L. A. N., & Can, U. (2025) · doi:10.2196/preprints.86687
Application of Generative AI in Health Care: Systematic Review
Perera, C. N., Herath, H. M. K. K. M. B., & Mittal, M. (2025) · Studies in computational intelligence · doi:10.1007/978-981-95-2129-6_10
Veitaitė, I. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-11651-2_3
Wong, A. V.-K., Roslan, N. L., Ho, S., Tan, R. A., Noor, J. M., Via, G., & Corradi, F. (2026) · The Ultrasound Journal · doi:10.5826/tuj.2026.18495
Assessment of Stage Two Hypertension Treatment Plans Written by Generative AI
Metzger, T., Hossain, Z., Park, K., Vu, S., Dixon, S., & Taylor, T. A. H. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15083103
Aksoy, H. B., Günaydın, S., Şimşek, Ş. M., & Cörüt, R. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1821639
Aljaafari, D., Aljumah, H. K., Alzuwayr, M. A., Al-Essa, Y. M., Alradhi, H. A., Alabdali, M. M., Almuslim, N. I., AlQarni, M., & Alesefir, W. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15103908
Predicted impact of Artificial Intelligence on Personalized Learning in Higher Education
Islam, S., Rezaul, M., Bably, S. A., Chowdhury, A. I., Leo, L. A., & Kabir, M. P. (2026) · IEEE International Conference on Quantum Photonics, Artificial Intelligence & Networking (QPAIN) · doi:10.1109/qpain69676.2026.11545546
Nadeem, F., Ibrahim, S. M., Taylor, S., Rawall, S., Mohammad, Z., Pirzadah, H., Ayala-Ortiz, J. L., & Rajaram, S. M. (2026) · Southern Medical Journal · doi:10.14423/smj.0000000000001989
Cohen, S., Tailor, P. D., Au, A., Vu, J. T., Marin, A. I., Prasad, P. S., Kwon, J., Hosseini, H., & Sridhar, J. (2026) · Journal of VitreoRetinal Diseases · doi:10.1177/24741264261469005
Sayed, M., & Mohamed, H. (2026) · Journal of Data Analysis and Information Processing · doi:10.4236/jdaip.2026.143018
Salleh, W. M. S. W., & Rahim, N. E. H. A. (2026) · INTERNATIONAL JOURNAL OF CARE SCHOLARS · doi:10.31436/ijcs.v9i3.606
Coşkun, S., Karaman, F. N., Uytun, H. A., & Akar, G. C. (2026) · Healthcare · doi:10.3390/healthcare14172797
Kağızman, S. Ç., Yazla, E., Aydınoğlu, Ü., Kızılaslan, O., Tunçkal, M., & Kahraman, Y. (2026) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-026-00702-7
Quinlan, R., Pacey, V., Thomas, R., Gray, K., & Davies, L. M. (2026) · Journal of Paediatrics and Child Health · doi:10.1111/jpc.70578
Evaluating AI Chatbots in Prosthodontics Education: A Quantitative MCQ‐Based Assessment
Patil, S., Bybee, S., Venkataiah, V. S., Amor, S., Majidi, A., Bhandi, S., & Licari, F. W. (2026) · International Journal of Dentistry · doi:10.1155/ijod/5985234
Küçükaydın, M. A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8740-6.ch006
Regulation of the Use of AI in the Health Sector From a Human Rights Perspective
Singh, N., Rattan, T., Pal, S., Khan, M. F., & Yadav, G. (2026) · IGI Global eBooks · doi:10.4018/979-8-2600-4881-8.ch008
Kraft, C. A., Luhanga, E. T., Xiao, M., Mani, G., & Gopalakrishnan, V. (2026) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-83742-484-920261010
Barr, L., Bakeer, S., Bemowski, B., Orio, S., & Zaveri, N. (2026) · Frontiers in Pharmacology · doi:10.3389/fphar.2026.1930362
- Authors
- Salman, I. M., Ameer, O. Z., Khanfar, M. A., & Hsieh, Y.-H.
- Affiliations
- Alfaisal University (SA)
- Case Western Reserve University (US)
- Corresponding author
- Salman, Ibrahim M.
- Venue
- Frontiers in Medicine
- Publisher
- Frontiers Media SA
- Published
- 2025-02-19
- Volume
- 12
- Pages
- 1495378-1495378
- ISSN
- 2296-858X (ISSN-L)
- PMC
- PMC11879995
- OpenAlex
- W4407719631
- Cited by
- 36
- References
- 32
- Citations by year
- 2026: 26 · 2025: 10
- FWCI
- 8.67
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Keywords
- generative AI, ChatGPT-4, Copilot, Google Gemini, cardiovascular pharmacology, medical education, AI accuracy evaluation, multiple-choice questions, short-answer questions, question difficulty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
