ChatGPT goes to the operating room: evaluating GPT-4 performance and its potential in surgical education and training in the era of large language models
Open accessPublished version of a preprint
Abstract
Purpose: This study aimed to assess the performance of ChatGPT, specifically the GPT-3.5 and GPT-4 models, in understanding complex surgical clinical information and its potential implications for surgical education and training. Methods: The dataset comprised 280 questions from the Korean general surgery board exams conducted between 2020 and 2022. Both GPT-3.5 and GPT-4 models were evaluated, and their performances were compared using McNemar test. Results: GPT-3.5 achieved an overall accuracy of 46.8%, while GPT-4 demonstrated a significant improvement with an overall accuracy of 76.4%, indicating a notable difference in performance between the models (P < 0.001). GPT-4 also exhibited consistent performance across all subspecialties, with accuracy rates ranging from 63.6% to 83.3%. Conclusion: ChatGPT, particularly GPT-4, demonstrates a remarkable ability to understand complex surgical clinical information, achieving an accuracy rate of 76.4% on the Korean general surgery board exam. However, it is important to recognize the limitations of large language models and ensure that they are used in conjunction with human expertise and judgment.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
GAI demonstrates potential in multidisciplinary knowledge acquisition within medical education by providing high-quality knowledge across various medical subfields.
“GAI demonstrates potential in multidisciplinary knowledge acquisition within medical education by providing high-quality knowledge across various medical subfields [87-94].”
From Applications, Challenges, and Prospects of Generative Artificial Intelligence Empowering Medical Education: Scoping Review (2025) · cites this paper as [88]
The Results report that GPT-4 answered 76.4% of Korean general surgery board-exam questions correctly, with results across ten surgical subspecialties. The Discussion connects this performance to potential uses in surgical education. This supports knowledge acquisition across surgical subspecialties, but does not establish high-quality knowledge across medical subfields more broadly. (LLM, conf=0.84)
Experimental — accuracy not yet measuredOnly a handful of studies on generative AI performance on standardized tests delved into potential implications; those that did posited that this technology could be useful for self-directed learning or exam preparation, but none provided an explicit exploration of this process.
“Only a handful of these studies delved into the potential implications of generative AI's performance on these tests [8,33]. Those that did posited that this technology could be useful for self-directed learning or exam preparation [8,11,34]. However, none of these studies provided an explicit exploration of this process.”
From Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review (2023) · cites this paper as [11]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredGPT-4 consistently performs well across different subspecialties.
“Notably, GPT-4 consistently performs well across different subspecialties, suggesting its capability to comprehend complex clinical information and enhance surgical education and training.”
From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [143]
GPT-3.5 achieved 46.8% overall accuracy and GPT-4 achieved 76.4% on understanding surgical clinical information, showing a significant performance difference.
“The results indicate an overall accuracy of 46.8% for GPT-3.5 and 76.4% for GPT-4, demonstrating a significant performance difference between the two models.”
From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [143]
Several studies focused on the role of generative AI models in tests of medical knowledge.
“Several studies focused on the role of generative AI models in tests of medical knowledge [8-11,13,26,27,31-39].”
From Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review (2023) · cites this paper as [11]
The precision and trustworthiness of generative AI are fundamental concerns elaborated in many publications.
“The precision and trustworthiness of generative AI are fundamental concerns thoroughly elaborated in many publications [8,11-15,20,22,24,32,35,41,42].”
From Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review (2023) · cites this paper as [11]
A few studies outline potential scenarios in which educators or learners might interact with AI systems, such as in self-directed learning, simulation environments, and writing assistance.
“A few studies outline potential scenarios in which educators or learners might interact with AI systems, such as in self-directed learning, simulation environments, and writing assistance [8,11,12,14,19,22,23,25,28,34,35,38,40,64].”
From Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review (2023) · cites this paper as [11]
Generative AI may produce biased content or lack representation of all populations.
“Moreover, concerns have arisen that generative AI may produce biased content or lack representation of all populations [8,11-15,19,20,22,24,32,35,41,42].”
From Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review (2023) · cites this paper as [11]
The Discussion says LLMs can generate inaccurate responses, but it does not address biased content or whether their outputs represent all populations. Searches for bias, diversity, and population representation found no passage making either point. (LLM, conf=0.73)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1× · see citing works
“4% accuracy rate on the Korean surgical board exam, it is important to remember that LLMs are generative models, often referred to as “stochastic parrots” [11].”
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · see citing works
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · see citing works
BioGPT: generative pre-trained transformer for biomedical text generation and mining
Luo, R., Sun, L., Xia, Y., Qin, T., Zhang, S., Poon, H., & Liu, T.-Y. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac409 · Cited by 1,150 · see citing works
Medical Education Must Move From the Information Age to the Age of Artificial Intelligence
Wartman, S. A., & Combs, C. D. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002044 · Cited by 585 · see citing works
ChatGPT Goes to Law SchoolPreprint
Choi, J. H., Hickman, K. E., Monahan, A., & Schwarcz, D. B. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4335905 · Cited by 522 · see citing works
Han, E.-R., Yeo, S., Kim, M.-J., Lee, Y.-H., Park, K.-H., & Roh, H. (2019) · BMC Medical Education · doi:10.1186/s12909-019-1891-5 · Cited by 393 · see citing works
American Surgical Association Blue Ribbon Committee Report on Surgical Education: 2004
Debas, H. T., Bass, B. L., Brennan, M. F., Flynn, T. C., Folse, J. R., Freischlag, J. A., Friedmann, P., Greenfield, L. J., Jones, R. S., Lewis, F. R., et al. (2005) · Annals of Surgery · doi:10.1097/01.sla.0000150066.83563.52 · Cited by 319 · see citing works
ChatGPT passing USMLE shines a spotlight on the flaws of medical education
Mbakwe, A. B., Lourentzou, I., Celi, L. A., Mechanic, O. J., & Dagan, A. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000205 · Cited by 273 · see citing works
Martelli, G., Boracchi, P., Palo, M. D., Pilotti, S., Oriana, S., Zucali, R., Daidone, M. G., & Palo, G. D. (2005) · Annals of Surgery · doi:10.1097/01.sla.0000167759.15670.14 · Cited by 199 · see citing works
GPT Takes the Bar ExamPreprint
Bommarito, M., & Katz, D. (2022) · Social Science Research Network · doi:10.48550/arxiv.2212.14402 · Cited by 195 · see citing works
Bommarito, M. J., & Katz, D. M. (2022) · doi:10.2139/ssrn.4314839 · Cited by 181 · see citing works
Being Affable, Available, and Able Is Not Enough
Kapadia, M. R., & Kieran, K. (2020) · JAMA Surgery · doi:10.1001/jamasurg.2019.5884 · Cited by 26 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 2,842 · see citing works
Toward expert-level medical question answering with large language models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03423-7 · Cited by 1,734 · see citing works
Bahroun, Z., Anane, C., Ahmed, V., & Zacca, A. (2023) · Sustainability · doi:10.3390/su151712983 · Cited by 784 · see citing works
A scoping review of artificial intelligence in medical education: BEME Guide No. 84
Gordon, M., Daniel, M., Ajiboye, A., Uraiby, H., Xu, N. Y., Bartlett, R., Hanson, J., Haas, M., Spadafore, M., Grafton-Clarke, C., et al. (2024) · Medical Teacher · doi:10.1080/0142159x.2024.2314198 · Cited by 543 · see citing works
Preiksaitis, C., & Rose, C. (2023) · JMIR Medical Education · doi:10.2196/48785 · Cited by 468 · see citing works
“The precision and trustworthiness of generative AI are fundamental concerns thoroughly elaborated in many publications [8,11-15, 20,22, 24, 32, The propensity of these systems to generate and propagate misinformation is a notable risk.”
BackgroundInfluentialA systematic review of large language models and their implications in medical education
Lucas, H. C., Upperman, J. S., & Robinson, J. R. (2024) · Medical Education · doi:10.1111/medu.15402 · Cited by 287 · see citing works
Moulaei, K., Yadegari, A., Baharestani, M., Farzanbakhsh, S., Sabet, B., & Reza Afrash, M. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105474 · Cited by 244 · see citing works
Younis, H. A., Eisa, T. A. E., Nasser, M., Sahib, T. M., Noor, A. A., Alyasiri, O. M., Salisu, S., Hayder, I. M., & Younis, H. A. (2024) · Diagnostics · doi:10.3390/diagnostics14010109 · Cited by 242 · see citing works
Evaluation of ChatGPT-generated medical responses: A systematic review and meta-analysis
Wei, Q., Yao, Z., Cui, Y., Wei, B., Jin, Z., & Xu, X. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104620 · Cited by 215 · see citing works
Garg, R. K., Urs, V. L., Agrawal, A. A., Chaudhary, S. K., Paliwal, V. K., & Kar, S. K. (2023) · Health Promotion Perspectives · doi:10.34172/hpp.2023.22 · Cited by 233 · see citing works
Miao, J., Thongprayoon, C., Suppadungsuk, S., Valencia, O. A. G., & Cheungpasitporn, W. (2024) · Medicina · doi:10.3390/medicina60030445 · Cited by 176 · see citing works
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102963 · Cited by 147 · see citing works
ChatGPT in Teaching and Learning: A Systematic Review
Ali, D., Fatemi, Y., Boskabadi, E., Nikfar, M., Ugwuoke, J., & Ali, H. (2024) · Education Sciences · doi:10.3390/educsci14060643 · Cited by 146 · see citing works
Fine-Tuning Large Language Models for Specialized Use Cases
Anisuzzaman, D., Malins, J. G., Friedman, P. A., & Attia, Z. I. (2024) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2024.11.005 · Cited by 155 · see citing works
Akpan, I. J., Kobara, Y. M., Owolabi, J., Akpan, A. A., & Offodile, O. F. (2024) · International Transactions in Operational Research · doi:10.1111/itor.13522 · Cited by 171 · see citing works
ChatGPT: promise and challenges for deployment in low- and middle-income countries
Wang, X., Sanders, H. M., Liu, Y., Seang, K., Tran, B. X., Atanasov, A. G., Qiu, Y., Tang, S., Car, J., Wang, Y. X., et al. (2023) · The Lancet Regional Health - Western Pacific · doi:10.1016/j.lanwpc.2023.100905 · Cited by 143 · see citing works
Seth, I., Cox, A., Xie, Y., Bulloch, G., Hunter‐Smith, D. J., Rozen, W. M., & Ross, R. J. (2023) · Aesthetic Surgery Journal · doi:10.1093/asj/sjad140 · Cited by 125 · see citing works
ChatGPT performance in prosthodontics: Assessment of accuracy and repeatability in answer generation
Freire, Y., Santamaría-Laorden, A., Pérez, J. O., Sánchez, M. G., García, V. D., & Suárez, A. (2024) · Journal of Prosthetic Dentistry · doi:10.1016/j.prosdent.2024.01.018 · Cited by 122 · see citing works
Miao, J., Thongprayoon, C., Suppadungsuk, S., Garcia Valencia, O. A., Qureshi, F., & Cheungpasitporn, W. (2023) · Clinics and Practice · doi:10.3390/clinpract14010008 · Cited by 111 · see citing works
Pressman, S. M., Borna, S., Gomez-Cabello, C. A., Haider, S. A., Haider, C. R., & Forte, A. J. (2024) · Healthcare · doi:10.3390/healthcare12080825 · Cited by 111 · see citing works
People Overtrust AI-Generated Medical Advice despite Low Accuracy
Shekar, S., Pataranutaporn, P., Sarabu, C. R., Cecchi, G., & Maes, P. (2025) · NEJM AI · doi:10.1056/aioa2300015 · Cited by 103 · see citing works
Fatima, A., Shafique, M. A., Alam, K., Ahmad, T. K. F., & Mustafa, M. S. (2024) · Medicine · doi:10.1097/md.0000000000039250 · Cited by 94 · see citing works
Yildirim, N., Richardson, H., Wetscherek, M. T., Bajwa, J., Jacob, J., Pinnock, M. A., Harris, S., Coelho De Castro, D., Bannur, S., Hyland, S., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642013 · Cited by 82 · see citing works
Benefits, limits, and risks of ChatGPT in medicine
Tangsrivimol, J. A., Darzidehkalani, E., Virk, H. U. H., Wang, Z., Egger, J., Wang, M., Hacking, S. M., Glicksberg, B. S., Strauß, M., & Krittanawong, C. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1518049 · Cited by 85 · see citing works
Performance of ChatGPT on Nephrology Test Questions
Miao, J., Thongprayoon, C., Valencia, O. A. G., Krisanapan, P., Sheikh, M. S., Davis, P. W., Mekraksakit, P., Suarez, M. L. G., Craici, I. M., & Cheungpasitporn, W. (2023) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.0000000000000330 · Cited by 71 · see citing works
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2024) · doi:10.2139/ssrn.4809363 · Cited by 69 · see citing works
Danesh, A., Pazouki, H., Danesh, F., Danesh, A., & Vardar‐Şengül, S. (2024) · Journal of Periodontology · doi:10.1002/jper.23-0514 · Cited by 67 · see citing works
GPT-4 can pass the Korean National Licensing Examination for Korean Medicine Doctors
Jang, D., Yun, T.-R., Lee, C.-Y., Kwon, Y. K., & Kim, C.-E. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000416 · Cited by 59 · see citing works
Gomez-Cabello, C. A., Borna, S., Pressman, S. M., Haider, S. A., & Forte, A. J. (2024) · Medicina · doi:10.3390/medicina60060957 · Cited by 59 · see citing works
Abbas, A., Rehman, M. S., & Rehman, S. S. (2024) · Cureus · doi:10.7759/cureus.55991 · Cited by 58 · see citing works
ChatGPT and Other Large Language Models in Medical Education — Scoping Literature Review
Aster, A., Laupichler, M. C., Rockwell-Kollmann, T., Masala, G., Bala, E., & Raupach, T. (2024) · Medical Science Educator · doi:10.1007/s40670-024-02206-6 · Cited by 59 · see citing works
ChatGPT in Plastic and Reconstructive Surgery
Sharma, S. C., Ramchandani, J. P., Thakker, A., & Lahiri, A. (2023) · Indian Journal of Plastic Surgery · doi:10.1055/s-0043-1771514 · Cited by 53 · see citing works
Analyzing evaluation methods for large language models in the medical field: a scoping review
Lee, J., Park, S., Shin, J., & Cho, B. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02709-7 · Cited by 52 · see citing works
Abouammoh, N. A., Alhasan, K. A., Aljamaan, F., Raina, R., Malki, K. H., Altamimi, I., Muaygil, R. A., Wahabi, H., Jamal, A. A., Alhaboob, A. A. N., et al. (2025) · JMIR Medical Education · doi:10.2196/63400 · Cited by 101 · see citing works
Kim, J. H., Kim, S. K., Choi, J., & Lee, Y. (2024) · Digital Health · doi:10.1177/20552076241227132 · Cited by 54 · see citing works
Waldock, W. J., Zhang, J., Guni, A., Nabeel, A., Darzi, A. W., & Ashrafian, H. (2024) · Journal of Medical Internet Research · doi:10.2196/56532 · Cited by 51 · see citing works
Applications of Artificial Intelligence in Medical Education: A Systematic Review
Hallquist, E., Gupta, I., Montalbano, M. J., & Loukas, M. (2025) · Cureus · doi:10.7759/cureus.79878 · Cited by 47 · see citing works
Can ChatGPT pass the thoracic surgery exam?
Gencer, A., & Aydın, S. (2023) · The American Journal of the Medical Sciences · doi:10.1016/j.amjms.2023.08.001 · Cited by 46 · see citing works
Abouammoh, N., Alhasan, K., Raina, R., Malki, K. A., Aljamaan, F., Tamimi, I., Muaygil, R., Wahabi, H., Jamal, A., Al-Tawfiq, J. A., et al. (2023) · doi:10.1101/2023.07.13.23292624 · Cited by 44 · see citing works
A descriptive study based on the comparison of ChatGPT and evidence-based neurosurgeons
Liu, J., Zheng, J., Cai, X., Wu, D., & Yin, C. (2023) · iScience · doi:10.1016/j.isci.2023.107590 · Cited by 44 · see citing works
Kunitsu, Y. (2023) · JMIR Medical Education · doi:10.2196/48452 · Cited by 45 · see citing works
Below average ChatGPT performance in medical microbiology exam compared to university students
Sallam, M., & Al-Salahat, K. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1333415 · Cited by 39 · see citing works
Weisman, D., Sugarman, A., Huang, Y. M., Gelberg, L., Ganz, P. A., & Comulada, W. S. (2025) · JMIR Formative Research · doi:10.2196/65670 · Cited by 37 · see citing works
Morreel, S., Verhoeven, V., & Mathysen, D. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000349 · Cited by 37 · see citing works
Lee, Y., Tessier, L., Brar, K., Malone, S., Jin, D., McKechnie, T., Jung, J. J., Kroh, M., & Dang, J. T. (2024) · Surgery for Obesity and Related Diseases · doi:10.1016/j.soard.2024.04.014 · Cited by 37 · see citing works
Garg, R. K., Urs, V. L., Agrawal, A. A., Chaudhary, S. K., Paliwal, V. K., & Kar, S. K. (2023) · medRxiv · doi:10.1101/2023.06.13.23291311 · Cited by 36 · see citing works
Assessing the Current Limitations of Large Language Models in Advancing Health Care Education
Kim, J., & Vajravelu, B. N. (2024) · JMIR Formative Research · doi:10.2196/51319 · Cited by 35 · see citing works
Chang, P. W., Amini, M. M., Davis, R. O., Nguyen, D. D., Dodge, J. L., Lee, H., Sheibani, S., Phan, J., Buxbaum, J. L., & Sahakian, A. B. (2024) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2024.04.022 · Cited by 34 · see citing works
Kollitsch, L., Eredics, K., Martin Marszalek, A., Rauchenwald, M., Brookman-May, S. D., Burger, M., Körner-Riffard, K., & May, M. (2024) · World Journal of Urology · doi:10.1007/s00345-023-04749-6 · Cited by 34 · see citing works
Understanding natural language: Potential application of large language models to ophthalmology
Yang, Z., Wang, D., Zhou, F., Song, D., Zhang, Y., Jiang, J., Kong, K., Liu, X., Qiao, Y., Chang, R. T., et al. (2024) · Asia-Pacific Journal of Ophthalmology · doi:10.1016/j.apjo.2024.100085 · Cited by 35 · see citing works
ChatGPT vs. Bard: A Comparative StudyPreprint
Ahmed, I., Roy, A., Kajol, M. A., Hasan, U., Datta, P. P., & Reza, M. R. (2023) · doi:10.22541/au.168923529.98827844/v1 · Cited by 33 · see citing works
Lin, Y., Luo, Z., Ye, Z., Zhong, N., Zhao, L., Zhang, L., Li, X., Chen, Z., & Chen, Y. (2025) · JMIR Medical Education · doi:10.2196/71125 · Cited by 31 · see citing works
“This lack of transparency makes it hard to understand how GAI reaches specific conclusions, especially when results are erroneous or biased, complicating efforts to trace and correct mistakes [86,88,148].”
Survey of Multimodal Medical Question Answering
Demirhan, H., & Zadrozny, W. (2023) · BioMedInformatics · doi:10.3390/biomedinformatics4010004 · Cited by 32 · see citing works
Wu, Z., Gan, W., Xue, Z., Ni, Z., Zheng, X., & Zhang, Y. (2024) · JMIR Medical Education · doi:10.2196/52746 · Cited by 29 · see citing works
Surgeons vs ChatGPT: Assessment and Feedback Performance Based on Real Surgical Scenarios
Jarry, C., Vela, J., Vivas, G. E., Escobar, E. A. G., Villagrán, I., Tirado, P. A., & Cohen, J. V. (2024) · Journal of surgical education · doi:10.1016/j.jsurg.2024.03.012 · Cited by 27 · see citing works
Tran, C. G., Chang, J., Sherman, S. K., & Andrade, J. P. D. (2024) · Journal of Surgical Research · doi:10.1016/j.jss.2024.04.060 · Cited by 26 · see citing works
RetractedYudovich, M., Makarova, E., Hague, C. M., & Raman, J. D. (2024) · Journal of Educational Evaluation for Health Professions · doi:10.3352/jeehp.2024.21.17 · Cited by 26 · see citing works
Jaleel, A., Aziz, U., Farid, G., Zahid Bashir, M., Mirza, T. R., Khizar Abbas, S. M., Aslam, S., & Sikander, R. M. H. (2025) · JMIR Medical Education · doi:10.2196/68070 · Cited by 25 · see citing works
ChatGPT versus Bard: A comparative study
Ahmed, I., Kajol, M. A., Hasan, U., Datta, P. P., Roy, A., & Reza, M. R. (2024) · Engineering Reports · doi:10.1002/eng2.12890 · Cited by 25 · see citing works
Applications of natural language processing tools in the surgical journey
Le, K. D. R., Tay, S. B. P., Choy, K. T., Verjans, J. W. H., Sasanelli, N., & Kong, J. C. H. (2024) · Frontiers in Surgery · doi:10.3389/fsurg.2024.1403540 · Cited by 25 · see citing works
ChatGPT and generative AI in urology and surgery—A narrative review
Qin, S., Chislett, B., Ischia, J. J., Ranasinghe, W. K. B., Silva, D. D., Coles‐Black, J., Woon, D. T. S., & Bolton, D. M. (2024) · BJUI Compass · doi:10.1002/bco2.390 · Cited by 24 · see citing works
ChatGPT in Medical Education: Bibliometric and Visual Analysis
Zhang, Y., Xie, X., & Xu, Q. (2025) · JMIR Medical Education · doi:10.2196/72356 · Cited by 29 · see citing works
Evaluating the performance of AI chatbots in responding to dental implant FAQs: A comparative study
Tuzlalı, M., Baki, N., Aral, K., Cüneyt Asım, A. R. A. L., & Bahçe, E. (2025) · BMC Oral Health · doi:10.1186/s12903-025-06863-w · Cited by 24 · see citing works
Moglia, A., Georgiou, K. E., Cerveri, P., Mainardi, L. T., Satava, R. M., & Cuschieri, A. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10849-5 · Cited by 24 · see citing works
Ming, S., Guo, Q., Cheng, W., & Lei, B. (2024) · JMIR Medical Education · doi:10.2196/52784 · Cited by 25 · see citing works
ChatGPT’s Impact on Education and Healthcare: Insights, Challenges, and Ethical Consideration
Hamad, Z. T., Jamil, N., & Belkacem, A. N. (2024) · IEEE Access · doi:10.1109/access.2024.3437374 · Cited by 25 · see citing works
Kim, J., Lee, S., Jeon, H., Lee, K., Bae, H., Kim, B., & Seo, J. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456215 · Cited by 25 · see citing works
Davies, N., Wilson, R., Winder, M. S., Tunster, S. J., McVicar, K., Thakrar, S., Williams, J., & Reid, A. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05042-9 · Cited by 24 · see citing works
Potential applications of ChatGPT in obstetrics and gynecology in Korea: a review article
Lee, Y., & Kim, S. Y. (2024) · Obstetrics & Gynecology Science · doi:10.5468/ogs.23231 · Cited by 22 · see citing works
Opportunities and challenges in the application of large artificial intelligence models in radiology
Pan, L., Zhao, Z., Lu, Y., Tang, K., Fu, L., Liang, Q., & Peng, S. (2024) · Meta-Radiology · doi:10.1016/j.metrad.2024.100080 · Cited by 22 · see citing works
Kankanamge, D., Wijeweera, C., Ong, Z., Preda, T. C., Carney, T., Wilson, M. J., & Preda, V. A. (2024) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2024.116074 · Cited by 20 · see citing works
One year in the classroom with ChatGPT: empirical insights and transformative impacts
Guo, F., Li, T., & Cunningham, C. J. L. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1574477 · Cited by 32 · see citing works
Expanding horizons and navigating challenges for enhanced clinical workflows: ChatGPT in urology
Talyshinskii, A., Naik, N., Hameed, B. M. Z., Zhanbyrbekuly, U., Khairli, G., Гулиев, Б. Г., Juilebø-Jones, P., Tzelves, L. I., & Somani, B. K. (2023) · Frontiers in Surgery · doi:10.3389/fsurg.2023.1257191 · Cited by 20 · see citing works
A review of ChatGPT in medical education: exploring advantages and limitations
Cheng, Y., & Zhu, L. (2025) · International Journal of Surgery · doi:10.1097/js9.0000000000002505 · Cited by 24 · see citing works
Adversarial prompt and fine-tuning attacks threaten medical large language models
Yang, Y., Jin, Q., Huang, F., & Lu, Z. (2025) · Nature Communications · doi:10.1038/s41467-025-64062-1 · Cited by 31 · see citing works
Application of Large Language Models in Stroke Rehabilitation Health Education: 2-Phase Study
Qiang, S., Liao, Y., Gu, Y., Gu, Y., Zhang, Y., Wang, Y., Xu, Z., Xu, Z., Han, N., Zhang, H., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/73226 · Cited by 21 · see citing works
DiDonna, N., Shetty, P. N., Khan, K., & Damitz, L. (2024) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000005929 · Cited by 18 · see citing works
Khalpey, Z. I., Kumar, U. A., King, N. C., Abraham, A., & Khalpey, A. (2024) · Cureus · doi:10.7759/cureus.65083 · Cited by 19 · see citing works
Kim, S. H., Tae, J. H., Chang, I. H., Kim, T., Myung, S. C., Nguyen, T. T., Choi, J., Kim, J. H., Kim, J. W., Lee, Y. S., et al. (2023) · Digital Health · doi:10.1177/20552076231203940 · Cited by 16 · see citing works
ChatGPT’s Performance on the Hand Surgery Self-Assessment Exam: A Critical Analysis
Han, Y., Choudhry, H. S., Simon, M., & Katt, B. M. (2024) · Journal of Hand Surgery Global Online · doi:10.1016/j.jhsg.2023.11.014 · Cited by 17 · see citing works
Hsieh, C., Hsieh, H.-Y., & Lin, H. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e34851 · Cited by 16 · see citing works
Omar, M., Nassar, S., Sharif, Κ., Glicksberg, B. S., Nadkarni, G. N., & Klang, E. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2024.1512824 · Cited by 15 · see citing works
Enhancing Surgical Education Through Artificial Intelligence in the Era of Digital Surgery
Aurello, P., Pace, M., Goglia, M., Pavone, M., Petrucciani, N., Carrano, F. M., Cicolani, A., Chiarini, L., Silecchia, G., & D’Andrea, V. (2025) · The American Surgeon · doi:10.1177/00031348251346539 · Cited by 15 · see citing works
Wuisang, M. C., Kurniawan, M., Santosa, K. A. W., Gunawan, A. A. S., & Saputra, K. E. (2023) · International Seminar on Application for Technology of Information and Communication (iSemantic) · doi:10.1109/isemantic59612.2023.10295323 · Cited by 14 · see citing works
Goglia, M., Pace, M., Yusef, M., Gallo, G., Pavone, M., Petrucciani, N., & Aurello, P. (2024) · In Vivo · doi:10.21873/invivo.13534 · Cited by 14 · see citing works
The accuracy of AI-assisted chatbots on the annual assessment test for emergency medicine residents
Arslan, B., Eyüpoğlu, G., Korkut, S., Türkdoğan, K. A., & Altınbilek, E. (2024) · Journal of Medicine Surgery and Public Health · doi:10.1016/j.glmedi.2024.100070 · Cited by 16 · see citing works
Large Language Model Architectures in Health Care: Scoping Review of Research Perspectives
Leiser, F., Guse, R., & Sunyaev, A. (2025) · Journal of Medical Internet Research · doi:10.2196/70315 · Cited by 12 · see citing works
Day, T., Gonzalez, M., Kim, J., McDaniel, P. N., Redican, K., & Zhu, T. (2025) · The Professional Geographer · doi:10.1080/00330124.2025.2478075 · Cited by 13 · see citing works
ChatGPT in surgery: a revolutionary innovation?
Bektaş, M., Pereira, J. K., Daams, F. D., & van der Peet, D. L. (2024) · Surgery Today · doi:10.1007/s00595-024-02800-6 · Cited by 13 · see citing works
Oh, N., Cha, W. C., Seo, J., Choi, S.-G., Kim, J. M., Chung, C. R., Suh, G. Y., Lee, S. Y., Oh, D. K., Park, M. H., et al. (2024) · Healthcare Informatics Research · doi:10.4258/hir.2024.30.3.266 · Cited by 12 · see citing works
Self-citationAbhari, S., Afshari, Y., Fatehi, F., Salmani, H., Garavand, A., Chumachenko, D., Zakerabasali, S., & Morita, P. P. (2024) · Annals of Medicine and Surgery · doi:10.1097/ms9.0000000000002716 · Cited by 13 · see citing works
The use of generative artificial intelligence in surgical education: a narrative review
Rao, L., Yang, E., Dissanayake, S., Cuomo, R., Seth, I., & Rozen, W. M. (2024) · Plastic and Aesthetic Research · doi:10.20517/2347-9264.2024.102 · Cited by 12 · see citing works
Quantum leap in medical mentorship: exploring ChatGPT’s transition from textbooks to terabytes
Chokkakula, S., Chong, S., Yang, B., Jiang, H., Yu, J., Han, R., Attitalla, I. H., Yin, C., & Zhang, S. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1517981 · Cited by 12 · see citing works
Annan, A., Eiden, A. L., Wang, D., Du, J., Rastegar-Mojarad, M., Nomula, V. K., & Wang, X. (2025) · JMIR Formative Research · doi:10.2196/64723 · Cited by 12 · see citing works
Sheikhalishahi, S., Haddadi, A., Sadeghipour, S., Rafiei, F., & Soltani, H. (2025) · Scientific Reports · doi:10.1038/s41598-025-31251-3 · Cited by 11 · see citing works
Feridooni, T., Javidan, A. P., Mahmood, D. N., Gomes, Z., Dueck, A. D., Wheatcroft, M., & Szalay, D. (2024) · JVS-Vascular Insights · doi:10.1016/j.jvsvi.2024.100137 · Cited by 11 · see citing works
Pears, M., Wadhwa, K., Payne, S. R., Hanchanale, V. S., Elmamoun, M. H., Jain, S., Konstantinidis, S. T., Rochester, M. A., Doherty, R., Spearpoint, K., et al. (2024) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-024-00180-7 · Cited by 11 · see citing works
Maruyama, H., Toyama, Y., Takanami, K., Takase, K., & Kamei, T. (2025) · JMIR Medical Education · doi:10.2196/69313 · Cited by 11 · see citing works
The role of ChatGPT in cardiothoracic surgery
Ahmed, S. K., Hussein, S. H., & Essa, R. A. (2023) · Indian Journal of Thoracic and Cardiovascular Surgery · doi:10.1007/s12055-023-01568-7 · Cited by 10 · see citing works
Mahajan, A., Tran, A. H., Tseng, E. S., Como, J. J., El-Hayek, K. M., Ladha, P., & Ho, V. P. (2024) · Surgery · doi:10.1016/j.surg.2024.08.026 · Cited by 10 · see citing works
- Authors
- Oh, N., Choi, G.-S., & Lee, W. Y.
- Affiliations
- Samsung Medical Center (KR)
- Corresponding author
- Lee, Woo Yong
- Venue
- Annals of Surgical Treatment and Research
- Publisher
- XMLink
- Published
- 2023
- Volume
- 104
- Issue
- 5
- Pages
- 269
- ISSN
- 2288-6575 (ISSN-L)
- 2288-6796
- PMC
- PMC10172028
- OpenAlex
- W4372047097
- Cited by
- 201
- References
- 12
- Citations by year
- 2026: 16 · 2025: 62 · 2024: 89 · 2023: 34
- FWCI
- 8.07
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Annals of Surgical Treatment and Research (publishedVersion, cc-by-nc)
- medRxiv (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- Samsung
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Surgery › Surgical Simulation and Training
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Keywords
- ChatGPT, GPT-4, large language models, surgical training, model performance comparison, GPT-3.5, artificial intelligence in healthcare
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
