The DOI matches its Crossref record; 12 of the 13 references it cites were found in a registry or an index, and 1 are not checked yet.
ChatGPT goes to the operating room: evaluating GPT-4 performance and its potential in surgical education and training in the era of large language models
Open accessPublished version of a preprint
Abstract
Purpose: This study aimed to assess the performance of ChatGPT, specifically the GPT-3.5 and GPT-4 models, in understanding complex surgical clinical information and its potential implications for surgical education and training. Methods: The dataset comprised 280 questions from the Korean general surgery board exams conducted between 2020 and 2022. Both GPT-3.5 and GPT-4 models were evaluated, and their performances were compared using McNemar test. Results: GPT-3.5 achieved an overall accuracy of 46.8%, while GPT-4 demonstrated a significant improvement with an overall accuracy of 76.4%, indicating a notable difference in performance between the models (P < 0.001). GPT-4 also exhibited consistent performance across all subspecialties, with accuracy rates ranging from 63.6% to 83.3%. Conclusion: ChatGPT, particularly GPT-4, demonstrates a remarkable ability to understand complex surgical clinical information, achieving an accuracy rate of 76.4% on the Korean general surgery board exam. However, it is important to recognize the limitations of large language models and ensure that they are used in conjunction with human expertise and judgment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“GAI demonstrates potential in multidisciplinary knowledge acquisition within medical education by providing high-quality knowledge across various medical subfields [87-94].”
From Applications, Challenges, and Prospects of Generative Artificial Intelligence Empowering Medical Education: Scoping Review (2025) · cites this paper as [88]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Only a handful of these studies delved into the potential implications of generative AI's performance on these tests [8,33]. Those that did posited that this technology could be useful for self-directed learning or exam preparation [8,11,34]. However, none of these studies provided an explicit exploration of this process.”
From Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review (2023) · cites this paper as [11]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Notably, GPT-4 consistently performs well across different subspecialties, suggesting its capability to comprehend complex clinical information and enhance surgical education and training.”
From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [143]
The cited source says what the claim attributes to it.
In the citing paper
“The results indicate an overall accuracy of 46.8% for GPT-3.5 and 76.4% for GPT-4, demonstrating a significant performance difference between the two models.”
From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [143]
The cited source says what the claim attributes to it.
In the citing paper
“Several studies focused on the role of generative AI models in tests of medical knowledge [8-11,13,26,27,31-39].”
From Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review (2023) · cites this paper as [11]
The cited source says what the claim attributes to it.
In the citing paper
“The precision and trustworthiness of generative AI are fundamental concerns thoroughly elaborated in many publications [8,11-15,20,22,24,32,35,41,42].”
From Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review (2023) · cites this paper as [11]
The cited source says what the claim attributes to it.
In the citing paper
“A few studies outline potential scenarios in which educators or learners might interact with AI systems, such as in self-directed learning, simulation environments, and writing assistance [8,11,12,14,19,22,23,25,28,34,35,38,40,64].”
From Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review (2023) · cites this paper as [11]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Moreover, concerns have arisen that generative AI may produce biased content or lack representation of all populations [8,11-15,19,20,22,24,32,35,41,42].”
From Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review (2023) · cites this paper as [11]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“4% accuracy rate on the Korean surgical board exam, it is important to remember that LLMs are generative models, often referred to as “stochastic parrots” [11].”
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115
BioGPT: generative pre-trained transformer for biomedical text generation and mining
Luo, R., Sun, L., Xia, Y., Qin, T., Zhang, S., Poon, H., & Liu, T.-Y. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac409 · Cited by 1,150
Medical Education Must Move From the Information Age to the Age of Artificial Intelligence
Wartman, S. A., & Combs, C. D. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002044 · Cited by 585
ChatGPT Goes to Law SchoolPreprint
Choi, J. H., Hickman, K. E., Monahan, A., & Schwarcz, D. B. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4335905 · Cited by 565
Han, E.-R., Yeo, S., Kim, M.-J., Lee, Y.-H., Park, K.-H., & Roh, H. (2019) · BMC Medical Education · doi:10.1186/s12909-019-1891-5 · Cited by 393
American Surgical Association Blue Ribbon Committee Report on Surgical Education: 2004
Debas, H. T., Bass, B. L., Brennan, M. F., Flynn, T. C., Folse, J. R., Freischlag, J. A., Friedmann, P., Greenfield, L. J., Jones, R. S., Lewis, F. R., et al. (2005) · Annals of Surgery · doi:10.1097/01.sla.0000150066.83563.52 · Cited by 319
ChatGPT passing USMLE shines a spotlight on the flaws of medical education
Mbakwe, A. B., Lourentzou, I., Celi, L. A., Mechanic, O. J., & Dagan, A. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000205 · Cited by 273
Martelli, G., Boracchi, P., Palo, M. D., Pilotti, S., Oriana, S., Zucali, R., Daidone, M. G., & Palo, G. D. (2005) · Annals of Surgery · doi:10.1097/01.sla.0000167759.15670.14 · Cited by 199
Research works printed but not linked: 1
- Touvron, 2023, arXiv [Preprint]
Cited by
Sorted by citations
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 4,254
Toward expert-level medical question answering with large language models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03423-7 · Cited by 1,734
Bahroun, Z., Anane, C., Ahmed, V., & Zacca, A. (2023) · Sustainability · doi:10.3390/su151712983 · Cited by 784
A scoping review of artificial intelligence in medical education: BEME Guide No. 84
Gordon, M., Daniel, M., Ajiboye, A., Uraiby, H., Xu, N. Y., Bartlett, R., Hanson, J., Haas, M., Spadafore, M., Grafton-Clarke, C., et al. (2024) · Medical Teacher · doi:10.1080/0142159x.2024.2314198 · Cited by 679
Preiksaitis, C., & Rose, C. (2023) · JMIR Medical Education · doi:10.2196/48785 · Cited by 552
“The precision and trustworthiness of generative AI are fundamental concerns thoroughly elaborated in many publications [8,11-15, 20,22, 24, 32, The propensity of these systems to generate and propagate misinformation is a notable risk.”
BackgroundInfluentialA systematic review of large language models and their implications in medical education
Lucas, H. C., Upperman, J. S., & Robinson, J. R. (2024) · Medical Education · doi:10.1111/medu.15402 · Cited by 287
Moulaei, K., Yadegari, A., Baharestani, M., Farzanbakhsh, S., Sabet, B., & Reza Afrash, M. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105474 · Cited by 244
Younis, H. A., Eisa, T. A. E., Nasser, M., Sahib, T. M., Noor, A. A., Alyasiri, O. M., Salisu, S., Hayder, I. M., & Younis, H. A. (2024) · Diagnostics · doi:10.3390/diagnostics14010109 · Cited by 242
Evaluation of ChatGPT-generated medical responses: A systematic review and meta-analysis
Wei, Q., Yao, Z., Cui, Y., Wei, B., Jin, Z., & Xu, X. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104620 · Cited by 233
Garg, R. K., Urs, V. L., Agrawal, A. A., Chaudhary, S. K., Paliwal, V. K., & Kar, S. K. (2023) · Health Promotion Perspectives · doi:10.34172/hpp.2023.22 · Cited by 247
Miao, J., Thongprayoon, C., Suppadungsuk, S., Valencia, O. A. G., & Cheungpasitporn, W. (2024) · Medicina · doi:10.3390/medicina60030445 · Cited by 198
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102963 · Cited by 147
ChatGPT in Teaching and Learning: A Systematic Review
Ali, D., Fatemi, Y., Boskabadi, E., Nikfar, M., Ugwuoke, J., & Ali, H. (2024) · Education Sciences · doi:10.3390/educsci14060643 · Cited by 146
Fine-Tuning Large Language Models for Specialized Use Cases
Anisuzzaman, D., Malins, J. G., Friedman, P. A., & Attia, Z. I. (2024) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2024.11.005 · Cited by 177
Akpan, I. J., Kobara, Y. M., Owolabi, J., Akpan, A. A., & Offodile, O. F. (2024) · International Transactions in Operational Research · doi:10.1111/itor.13522 · Cited by 187
ChatGPT: promise and challenges for deployment in low- and middle-income countries
Wang, X., Sanders, H. M., Liu, Y., Seang, K., Tran, B. X., Atanasov, A. G., Qiu, Y., Tang, S., Car, J., Wang, Y. X., et al. (2023) · The Lancet Regional Health - Western Pacific · doi:10.1016/j.lanwpc.2023.100905 · Cited by 151
Seth, I., Cox, A., Xie, Y., Bulloch, G., Hunter‐Smith, D. J., Rozen, W. M., & Ross, R. J. (2023) · Aesthetic Surgery Journal · doi:10.1093/asj/sjad140 · Cited by 125
ChatGPT performance in prosthodontics: Assessment of accuracy and repeatability in answer generation
Freire, Y., Santamaría-Laorden, A., Pérez, J. O., Sánchez, M. G., García, V. D., & Suárez, A. (2024) · Journal of Prosthetic Dentistry · doi:10.1016/j.prosdent.2024.01.018 · Cited by 132
Miao, J., Thongprayoon, C., Suppadungsuk, S., Garcia Valencia, O. A., Qureshi, F., & Cheungpasitporn, W. (2023) · Clinics and Practice · doi:10.3390/clinpract14010008 · Cited by 111
Pressman, S. M., Borna, S., Gomez-Cabello, C. A., Haider, S. A., Haider, C. R., & Forte, A. J. (2024) · Healthcare · doi:10.3390/healthcare12080825 · Cited by 111
People Overtrust AI-Generated Medical Advice despite Low Accuracy
Shekar, S., Pataranutaporn, P., Sarabu, C. R., Cecchi, G., & Maes, P. (2025) · NEJM AI · doi:10.1056/aioa2300015 · Cited by 112
Fatima, A., Shafique, M. A., Alam, K., Ahmad, T. K. F., & Mustafa, M. S. (2024) · Medicine · doi:10.1097/md.0000000000039250 · Cited by 95
Yildirim, N., Richardson, H., Wetscherek, M. T., Bajwa, J., Jacob, J., Pinnock, M. A., Harris, S., Coelho De Castro, D., Bannur, S., Hyland, S., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642013 · Cited by 82
Benefits, limits, and risks of ChatGPT in medicine
Tangsrivimol, J. A., Darzidehkalani, E., Virk, H. U. H., Wang, Z., Egger, J., Wang, M., Hacking, S. M., Glicksberg, B. S., Strauß, M., & Krittanawong, C. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1518049 · Cited by 88
Performance of ChatGPT on Nephrology Test Questions
Miao, J., Thongprayoon, C., Valencia, O. A. G., Krisanapan, P., Sheikh, M. S., Davis, P. W., Mekraksakit, P., Suarez, M. L. G., Craici, I. M., & Cheungpasitporn, W. (2023) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.0000000000000330 · Cited by 72
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2024) · doi:10.2139/ssrn.4809363 · Cited by 69
Danesh, A., Pazouki, H., Danesh, F., Danesh, A., & Vardar‐Şengül, S. (2024) · Journal of Periodontology · doi:10.1002/jper.23-0514 · Cited by 67
GPT-4 can pass the Korean National Licensing Examination for Korean Medicine Doctors
Jang, D., Yun, T.-R., Lee, C.-Y., Kwon, Y. K., & Kim, C.-E. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000416 · Cited by 63
Gomez-Cabello, C. A., Borna, S., Pressman, S. M., Haider, S. A., & Forte, A. J. (2024) · Medicina · doi:10.3390/medicina60060957 · Cited by 59
Abbas, A., Rehman, M. S., & Rehman, S. S. (2024) · Cureus · doi:10.7759/cureus.55991 · Cited by 58
ChatGPT and Other Large Language Models in Medical Education — Scoping Literature Review
Aster, A., Laupichler, M. C., Rockwell-Kollmann, T., Masala, G., Bala, E., & Raupach, T. (2024) · Medical Science Educator · doi:10.1007/s40670-024-02206-6 · Cited by 67
ChatGPT in Plastic and Reconstructive Surgery
Sharma, S. C., Ramchandani, J. P., Thakker, A., & Lahiri, A. (2023) · Indian Journal of Plastic Surgery · doi:10.1055/s-0043-1771514 · Cited by 54
Analyzing evaluation methods for large language models in the medical field: a scoping review
Lee, J., Park, S., Shin, J., & Cho, B. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02709-7 · Cited by 52
Abouammoh, N. A., Alhasan, K. A., Aljamaan, F., Raina, R., Malki, K. H., Altamimi, I., Muaygil, R. A., Wahabi, H., Jamal, A. A., Alhaboob, A. A. N., et al. (2025) · JMIR Medical Education · doi:10.2196/63400 · Cited by 112
Kim, J. H., Kim, S. K., Choi, J., & Lee, Y. (2024) · Digital Health · doi:10.1177/20552076241227132 · Cited by 56
Waldock, W. J., Zhang, J., Guni, A., Nabeel, A., Darzi, A. W., & Ashrafian, H. (2024) · Journal of Medical Internet Research · doi:10.2196/56532 · Cited by 57
Applications of Artificial Intelligence in Medical Education: A Systematic Review
Hallquist, E., Gupta, I., Montalbano, M. J., & Loukas, M. (2025) · Cureus · doi:10.7759/cureus.79878 · Cited by 58
Can ChatGPT pass the thoracic surgery exam?
Gencer, A., & Aydın, S. (2023) · The American Journal of the Medical Sciences · doi:10.1016/j.amjms.2023.08.001 · Cited by 44
Abouammoh, N., Alhasan, K., Raina, R., Malki, K. A., Aljamaan, F., Tamimi, I., Muaygil, R., Wahabi, H., Jamal, A., Al-Tawfiq, J. A., et al. (2023) · doi:10.1101/2023.07.13.23292624 · Cited by 44
A descriptive study based on the comparison of ChatGPT and evidence-based neurosurgeons
Liu, J., Zheng, J., Cai, X., Wu, D., & Yin, C. (2023) · iScience · doi:10.1016/j.isci.2023.107590 · Cited by 44
Kunitsu, Y. (2023) · JMIR Medical Education · doi:10.2196/48452 · Cited by 48
Below average ChatGPT performance in medical microbiology exam compared to university students
Sallam, M., & Al-Salahat, K. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1333415 · Cited by 39
Weisman, D., Sugarman, A., Huang, Y. M., Gelberg, L., Ganz, P. A., & Comulada, W. S. (2025) · JMIR Formative Research · doi:10.2196/65670 · Cited by 37
Morreel, S., Verhoeven, V., & Mathysen, D. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000349 · Cited by 37
Lee, Y., Tessier, L., Brar, K., Malone, S., Jin, D., McKechnie, T., Jung, J. J., Kroh, M., & Dang, J. T. (2024) · Surgery for Obesity and Related Diseases · doi:10.1016/j.soard.2024.04.014 · Cited by 40
Garg, R. K., Urs, V. L., Agrawal, A. A., Chaudhary, S. K., Paliwal, V. K., & Kar, S. K. (2023) · medRxiv · doi:10.1101/2023.06.13.23291311 · Cited by 36
Assessing the Current Limitations of Large Language Models in Advancing Health Care Education
Kim, J., & Vajravelu, B. N. (2024) · JMIR Formative Research · doi:10.2196/51319 · Cited by 39
Chang, P. W., Amini, M. M., Davis, R. O., Nguyen, D. D., Dodge, J. L., Lee, H., Sheibani, S., Phan, J., Buxbaum, J. L., & Sahakian, A. B. (2024) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2024.04.022 · Cited by 34
Kollitsch, L., Eredics, K., Marszalek, M., Rauchenwald, M., Brookman-May, S. D., Burger, M., Körner-Riffard, K., & May, M. (2024) · World Journal of Urology · doi:10.1007/s00345-023-04749-6 · Cited by 35
Understanding natural language: Potential application of large language models to ophthalmology
Yang, Z., Wang, D., Zhou, F., Song, D., Zhang, Y., Jiang, J., Kong, K., Liu, X., Qiao, Y., Chang, R. T., et al. (2024) · Asia-Pacific Journal of Ophthalmology · doi:10.1016/j.apjo.2024.100085 · Cited by 38
ChatGPT vs. Bard: A Comparative StudyPreprint
Ahmed, I., Roy, A., Kajol, M. A., Hasan, U., Datta, P. P., & Reza, M. R. (2023) · doi:10.22541/au.168923529.98827844/v1 · Cited by 33
Lin, Y., Luo, Z., Ye, Z., Zhong, N., Zhao, L., Zhang, L., Li, X., Chen, Z., & Chen, Y. (2025) · JMIR Medical Education · doi:10.2196/71125 · Cited by 31
“This lack of transparency makes it hard to understand how GAI reaches specific conclusions, especially when results are erroneous or biased, complicating efforts to trace and correct mistakes [86,88,148].”
Survey of Multimodal Medical Question Answering
Demirhan, H., & Zadrozny, W. (2023) · BioMedInformatics · doi:10.3390/biomedinformatics4010004 · Cited by 32
Wu, Z., Gan, W., Xue, Z., Ni, Z., Zheng, X., & Zhang, Y. (2024) · JMIR Medical Education · doi:10.2196/52746 · Cited by 29
Surgeons vs ChatGPT: Assessment and Feedback Performance Based on Real Surgical Scenarios
Jarry, C., Vela, J., Vivas, G. E., Escobar, E. A. G., Villagrán, I., Tirado, P. A., & Cohen, J. V. (2024) · Journal of surgical education · doi:10.1016/j.jsurg.2024.03.012 · Cited by 29
Tran, C. G., Chang, J., Sherman, S. K., & Andrade, J. P. D. (2024) · Journal of Surgical Research · doi:10.1016/j.jss.2024.04.060 · Cited by 26
RetractedYudovich, M., Makarova, E., Hague, C. M., & Raman, J. D. (2024) · Journal of Educational Evaluation for Health Professions · doi:10.3352/jeehp.2024.21.17 · Cited by 26
Jaleel, A., Aziz, U., Farid, G., Zahid Bashir, M., Mirza, T. R., Khizar Abbas, S. M., Aslam, S., & Sikander, R. M. H. (2025) · JMIR Medical Education · doi:10.2196/68070 · Cited by 25
ChatGPT versus Bard: A comparative study
Ahmed, I., Kajol, M. A., Hasan, U., Datta, P. P., Roy, A., & Reza, M. R. (2024) · Engineering Reports · doi:10.1002/eng2.12890 · Cited by 65
Applications of natural language processing tools in the surgical journey
Le, K. D. R., Tay, S. B. P., Choy, K. T., Verjans, J. W. H., Sasanelli, N., & Kong, J. C. H. (2024) · Frontiers in Surgery · doi:10.3389/fsurg.2024.1403540 · Cited by 26
ChatGPT and generative AI in urology and surgery—A narrative review
Qin, S., Chislett, B., Ischia, J. J., Ranasinghe, W. K. B., Silva, D. D., Coles‐Black, J., Woon, D. T. S., & Bolton, D. M. (2024) · BJUI Compass · doi:10.1002/bco2.390 · Cited by 25
ChatGPT in Medical Education: Bibliometric and Visual Analysis
Zhang, Y., Xie, X., & Xu, Q. (2025) · JMIR Medical Education · doi:10.2196/72356 · Cited by 35
Evaluating the performance of AI chatbots in responding to dental implant FAQs: A comparative study
Tuzlalı, M., Baki, N., Aral, K., Cüneyt Asım, A. R. A. L., & Bahçe, E. (2025) · BMC Oral Health · doi:10.1186/s12903-025-06863-w · Cited by 24
Moglia, A., Georgiou, K. E., Cerveri, P., Mainardi, L. T., Satava, R. M., & Cuschieri, A. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10849-5 · Cited by 27
Ming, S., Guo, Q., Cheng, W., & Lei, B. (2024) · JMIR Medical Education · doi:10.2196/52784 · Cited by 25
ChatGPT’s Impact on Education and Healthcare: Insights, Challenges, and Ethical Consideration
Hamad, Z. T., Jamil, N., & Belkacem, A. N. (2024) · IEEE Access · doi:10.1109/access.2024.3437374 · Cited by 27
Kim, J., Lee, S., Jeon, H., Lee, K., Bae, H., Kim, B., & Seo, J. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456215 · Cited by 27
Davies, N., Wilson, R., Winder, M. S., Tunster, S. J., McVicar, K., Thakrar, S., Williams, J., & Reid, A. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05042-9 · Cited by 26
Potential applications of ChatGPT in obstetrics and gynecology in Korea: a review article
Lee, Y., & Kim, S. Y. (2024) · Obstetrics & Gynecology Science · doi:10.5468/ogs.23231 · Cited by 24
Opportunities and challenges in the application of large artificial intelligence models in radiology
Pan, L., Zhao, Z., Lu, Y., Tang, K., Fu, L., Liang, Q., & Peng, S. (2024) · Meta-Radiology · doi:10.1016/j.metrad.2024.100080 · Cited by 22
Kankanamge, D., Wijeweera, C., Ong, Z., Preda, T. C., Carney, T., Wilson, M. J., & Preda, V. A. (2024) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2024.116074 · Cited by 20
One year in the classroom with ChatGPT: empirical insights and transformative impacts
Guo, F., Li, T., & Cunningham, C. J. L. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1574477 · Cited by 40
Expanding horizons and navigating challenges for enhanced clinical workflows: ChatGPT in urology
Talyshinskii, A., Naik, N., Hameed, B. M. Z., Zhanbyrbekuly, U., Khairli, G., Гулиев, Б. Г., Juilebø-Jones, P., Tzelves, L. I., & Somani, B. K. (2023) · Frontiers in Surgery · doi:10.3389/fsurg.2023.1257191 · Cited by 21
A review of ChatGPT in medical education: exploring advantages and limitations
Cheng, Y., & Zhu, L. (2025) · International Journal of Surgery · doi:10.1097/js9.0000000000002505 · Cited by 26
Adversarial prompt and fine-tuning attacks threaten medical large language models
Yang, Y., Jin, Q., Huang, F., & Lu, Z. (2025) · Nature Communications · doi:10.1038/s41467-025-64062-1 · Cited by 38
Application of Large Language Models in Stroke Rehabilitation Health Education: 2-Phase Study
Qiang, S., Liao, Y., Gu, Y., Gu, Y., Zhang, Y., Wang, Y., Xu, Z., Xu, Z., Han, N., Zhang, H., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/73226 · Cited by 21
DiDonna, N., Shetty, P. N., Khan, K., & Damitz, L. (2024) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000005929 · Cited by 18
Khalpey, Z. I., Kumar, U. A., King, N. C., Abraham, A., & Khalpey, A. (2024) · Cureus · doi:10.7759/cureus.65083 · Cited by 19
Kim, S. H., Tae, J. H., Chang, I. H., Kim, T., Myung, S. C., Nguyen, T. T., Choi, J., Kim, J. H., Kim, J. W., Lee, Y. S., et al. (2023) · Digital Health · doi:10.1177/20552076231203940 · Cited by 16
ChatGPT’s Performance on the Hand Surgery Self-Assessment Exam: A Critical Analysis
Han, Y., Choudhry, H. S., Simon, M., & Katt, B. M. (2024) · Journal of Hand Surgery Global Online · doi:10.1016/j.jhsg.2023.11.014 · Cited by 17
Hsieh, C., Hsieh, H.-Y., & Lin, H. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e34851 · Cited by 18
Omar, M., Nassar, S., Sharif, Κ., Glicksberg, B. S., Nadkarni, G. N., & Klang, E. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2024.1512824 · Cited by 20
Enhancing Surgical Education Through Artificial Intelligence in the Era of Digital Surgery
Aurello, P., Pace, M., Goglia, M., Pavone, M., Petrucciani, N., Carrano, F. M., Cicolani, A., Chiarini, L., Silecchia, G., & D’Andrea, V. (2025) · The American Surgeon · doi:10.1177/00031348251346539 · Cited by 15
Wuisang, M. C., Kurniawan, M., Santosa, K. A. W., Gunawan, A. A. S., & Saputra, K. E. (2023) · International Seminar on Application for Technology of Information and Communication (iSemantic) · doi:10.1109/isemantic59612.2023.10295323 · Cited by 14
Goglia, M., Pace, M., Yusef, M., Gallo, G., Pavone, M., Petrucciani, N., & Aurello, P. (2024) · In Vivo · doi:10.21873/invivo.13534 · Cited by 14
The accuracy of AI-assisted chatbots on the annual assessment test for emergency medicine residents
Arslan, B., Eyüpoğlu, G., Korkut, S., Türkdoğan, K. A., & Altınbilek, E. (2024) · Journal of Medicine Surgery and Public Health · doi:10.1016/j.glmedi.2024.100070 · Cited by 17
Large Language Model Architectures in Health Care: Scoping Review of Research Perspectives
Leiser, F., Guse, R., & Sunyaev, A. (2025) · Journal of Medical Internet Research · doi:10.2196/70315 · Cited by 16
Day, T., Gonzalez, M., Kim, J., McDaniel, P. N., Redican, K., & Zhu, T. (2025) · The Professional Geographer · doi:10.1080/00330124.2025.2478075 · Cited by 13
ChatGPT in surgery: a revolutionary innovation?
Bektaş, M., Pereira, J. K., Daams, F. D., & van der Peet, D. L. (2024) · Surgery Today · doi:10.1007/s00595-024-02800-6 · Cited by 14
Oh, N., Cha, W. C., Seo, J., Choi, S.-G., Kim, J. M., Chung, C. R., Suh, G. Y., Lee, S. Y., Oh, D. K., Park, M. H., et al. (2024) · Healthcare Informatics Research · doi:10.4258/hir.2024.30.3.266 · Cited by 18
Self-citationAbhari, S., Afshari, Y., Fatehi, F., Salmani, H., Garavand, A., Chumachenko, D., Zakerabasali, S., & Morita, P. P. (2024) · Annals of Medicine and Surgery · doi:10.1097/ms9.0000000000002716 · Cited by 13
The use of generative artificial intelligence in surgical education: a narrative review
Rao, L., Yang, E., Dissanayake, S., Cuomo, R., Seth, I., & Rozen, W. M. (2024) · Plastic and Aesthetic Research · doi:10.20517/2347-9264.2024.102 · Cited by 13
Quantum leap in medical mentorship: exploring ChatGPT’s transition from textbooks to terabytes
Chokkakula, S., Chong, S., Yang, B., Jiang, H., Yu, J., Han, R., Attitalla, I. H., Yin, C., & Zhang, S. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1517981 · Cited by 12
Annan, A., Eiden, A. L., Wang, D., Du, J., Rastegar-Mojarad, M., Nomula, V. K., & Wang, X. (2025) · JMIR Formative Research · doi:10.2196/64723 · Cited by 14
Sheikhalishahi, S., Haddadi, A., Sadeghipour, S., Rafiei, F., & Soltani, H. (2025) · Scientific Reports · doi:10.1038/s41598-025-31251-3 · Cited by 11
Feridooni, T., Javidan, A. P., Mahmood, D. N., Gomes, Z., Dueck, A. D., Wheatcroft, M., & Szalay, D. (2024) · JVS-Vascular Insights · doi:10.1016/j.jvsvi.2024.100137 · Cited by 11
Pears, M., Wadhwa, K., Payne, S. R., Hanchanale, V. S., Elmamoun, M. H., Jain, S., Konstantinidis, S. T., Rochester, M. A., Doherty, R., Spearpoint, K., et al. (2024) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-024-00180-7 · Cited by 11
Maruyama, H., Toyama, Y., Takanami, K., Takase, K., & Kamei, T. (2025) · JMIR Medical Education · doi:10.2196/69313 · Cited by 11
The role of ChatGPT in cardiothoracic surgery
Ahmed, S. K., Hussein, S. H., & Essa, R. A. (2023) · Indian Journal of Thoracic and Cardiovascular Surgery · doi:10.1007/s12055-023-01568-7 · Cited by 10
Mahajan, A., Tran, A. H., Tseng, E. S., Como, J. J., El-Hayek, K. M., Ladha, P., & Ho, V. P. (2024) · Surgery · doi:10.1016/j.surg.2024.08.026 · Cited by 10
Metadata
- Authors
- Oh, N., Choi, G.-S., & Lee, W. Y.
- Affiliations
- Samsung Medical Center (KR)
- Corresponding author
- Lee, Woo Yong
- Venue
- Annals of Surgical Treatment and Research
- Publisher
- XMLink
- Published
- 2023
- Volume
- 104
- Issue
- 5
- Pages
- 269
- ISSN
- 2288-6575 (ISSN-L)
- 2288-6796
- PMC
- PMC10172028
- OpenAlex
- W4372047097
- Cited by
- 201
- References
- 12
- Citations by year
- 2026: 16 · 2025: 62 · 2024: 89 · 2023: 34
- FWCI
- 8.07
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Annals of Surgical Treatment and Research (publishedVersion, cc-by-nc)
- medRxiv (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- Samsung
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Surgery › Surgical Simulation and Training
- Social Sciences › Education › Innovations in Medical Education
- Keywords
- ChatGPT, GPT-4, large language models, surgical training, model performance comparison, GPT-3.5, artificial intelligence in healthcare
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
