Using ChatGPT for Clinical Practice and Medical Education: Cross-Sectional Survey of Medical Students’ and Physicians’ Perceptions
Open accessPublished version of a preprint
The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
Background: ChatGPT is a well-known large language model–based chatbot. It could be used in the medical field in many aspects. However, some physicians are still unfamiliar with ChatGPT and are concerned about its benefits and risks. Objective: We aim to evaluate the perception of physicians and medical students toward using ChatGPT in the medical field. Methods: A web-based questionnaire was sent to medical students, interns, residents, and attending staff with questions regarding their perception toward using ChatGPT in clinical practice and medical education. Participants were also asked to rate their perception of ChatGPT’s generated response about knee osteoarthritis. Results: Participants included 124 medical students, 46 interns, 37 residents, and 32 attending staff. After reading ChatGPT’s response, 132 of the 239 (55.2%) participants had a positive rating about using ChatGPT for clinical practice. The proportion of positive answers was significantly lower in graduated physicians (48/115, 42%) compared with medical students (84/124, 68%; P<.001). Participants listed a lack of a patient-specific treatment plan, updated evidence, and a language barrier as ChatGPT’s pitfalls. Regarding using ChatGPT for medical education, the proportion of positive responses was also significantly lower in graduate physicians (71/115, 62%) compared to medical students (103/124, 83.1%; P<.001). Participants were concerned that ChatGPT’s response was too superficial, might lack scientific evidence, and might need expert verification. Conclusions: Medical students generally had a positive perception of using ChatGPT for guiding treatment and medical education, whereas graduated doctors were more cautious in this regard. Nonetheless, both medical students and graduated doctors positively perceived using ChatGPT for creating patient educational materials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In a medical environment, GAI-generated treatment plans, although generally reasonable, often fail to consider individual patient characteristics, such as the severity of the disease, lifestyle, and personal preferences [105].”
From Applications, Challenges, and Prospects of Generative Artificial Intelligence Empowering Medical Education: Scoping Review (2025) · cites this paper as [105]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The content generated by GAI lacks personalization tailored to individual needs.”
From Applications, Challenges, and Prospects of Generative Artificial Intelligence Empowering Medical Education: Scoping Review (2025) · cites this paper as [105]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“First, although the number of respondents exceeded the required sample size, the response rate was lower compared with similar studies [19, 21, 22, 24, 25, 29, 31, 34, 37, 40, 62], which may limit the generalizability of the findings.”
The cited source says what the claim attributes to it.
In the citing paper
“Eight studies examined the knowledge, attitudes, and skills of medical students and educators regarding LLMs, specifically ChatGPT [24, 26–29, 79–81]. Students generally expressed a more positive attitude toward LLMs than practicing physicians, showing optimism about their potential to support learners and educators [81]. Those who were more knowledgeable about LLMs tended to hold a more positive”
From Artificial intelligence in undergraduate medical education: an updated scoping review (2025) · cites this paper as [81]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Huynh, L. M., Bonebrake, B. T., Schultis, K., Quach, A., & Deibert, C. M. (2023) · Urology Practice · doi:10.1097/upj.0000000000000406 · Cited by 46 · cited 1×
“For more advanced examination levels, such as resident-level examinations, ChatGPT performed more poorly [7,34,35].”
BackgroundRetractedKung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · cited 1×
“However, the results of using ChatGPT were quite varied, from passing the exam to failing some [7,8,10,30-33].”
ResultKung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023) · JMIR Medical Education · doi:10.2196/45312 · Cited by 2,078
Dergaa, I., Chamari, K., Zmijewski, P., & Ben Saad, H. (2023) · Biology of Sport · doi:10.5114/biolsport.2023.125623 · Cited by 796 · cited 1×
“In the medical field, it has been used to help write manuscripts [3-5].”
BackgroundLarge Language Models in Medical Education: Opportunities, Challenges, and Future Directions
Abd-alrazaq, A., AlSaad, R., Alhuwail, D., Ahmed, A., Healy, P. M., Latifi, S., Aziz, S., Damseh, R., Alabed Alrazak, S., & Sheikh, J. (2023) · JMIR Medical Education · doi:10.2196/48291 · Cited by 722 · cited 1×
“In medical education, ChatGPT could be used in various aspects [27].”
MethodMájovský, M., Černý, M., Kasal, M., Komarc, M., & Netuka, D. (2023) · Journal of Medical Internet Research · doi:10.2196/46924 · Cited by 301 · cited 1×
“However, researchers were still concerned about the contents’ethical consideration and validity [6].”
BackgroundSamaan, J. S., Yeo, Y. H., Rajeev, N., Hawley, L., Abel, S., Ng, W. H., Srinivasan, N., Park, J., Burch, M., Watson, R., et al. (2023) · Obesity Surgery · doi:10.1007/s11695-023-06603-5 · Cited by 282 · cited 1×
“For example, bariatric surgeons rated responses of ChatGPT as “comprehensive” for 86.8% of the questions asked [18].”
BackgroundInfluentialHuh, S. (2023) · Journal of Educational Evaluation for Health Professions · doi:10.3352/jeehp.2023.20.1 · Cited by 278 · cited 1×
“However, the results of using ChatGPT were quite varied, from passing the exam to failing some [7,8,10,30-33].”
ResultPerformance of GPT-3.5 and GPT-4 on the Japanese Medical Licensing Examination: Comparison Study
Takagi, S., Watari, T., Erabi, A., & Sakaguchi, K. (2023) · JMIR Medical Education · doi:10.2196/48002 · Cited by 271
Can ChatGPT pass the life support exams without entering the American heart association course?
Fijačko, N., Gosak, L., Štiglic, G., Picard, C. T., & John Douma, M. (2023) · Resuscitation · doi:10.1016/j.resuscitation.2023.109732 · Cited by 206
Thirunavukarasu, A. J., Hassan, R., Mahmood, S., Sanghera, R., Barzangi, K., El Mukashfi, M., & Shah, S. (2023) · JMIR Medical Education · doi:10.2196/46599 · Cited by 187 · cited 1×
“However, the results of using ChatGPT were quite varied, from passing the exam to failing some [7,8,10,30-33].”
ResultMomenaei, B., Wakabayashi, T., Shahlaee, A., Durrani, A. F., Pandit, S. A., Wang, K., Mansour, H. A., Abishek, R. M., Xu, D., Sridhar, J., et al. (2023) · Ophthalmology Retina · doi:10.1016/j.oret.2023.05.022 · Cited by 178 · cited 1×
“The responses to common questions about retinal detachments were rated appropriate in 80%-90% of the questions asked [20].”
BackgroundAnalysis of large-language model versus human performance for genetics questions
Duong, D., & Solomon, B. D. (2024) · European Journal of Human Genetics · doi:10.1038/s41431-023-01396-8 · Cited by 159 · cited 1×
“However, the results of using ChatGPT were quite varied, from passing the exam to failing some [7,8,10,30-33].”
ResultHumar, P., Asaad, M., Bengur, F. B., & Nguyen, V. (2023) · Aesthetic Surgery Journal · doi:10.1093/asj/sjad130 · Cited by 155 · cited 1×
“For example, ChatGPT’s score in the plastic surgery in-training examination was ranked at the 49th percentile compared with first-year residents but significantly worse than fifth-and sixth-year residents at the zeroth percentile [9].”
MethodHoch, C. C., Wollenberg, B., Lüers, J.-C., Knoedler, S., Knoedler, L., Frank, K., Cotofana, S., & Alfertshofer, M. (2023) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-023-08051-4 · Cited by 154 · cited 3×
“However, the results of using ChatGPT were quite varied, from passing the exam to failing some [7,8,10,30-33].”
BackgroundResultLum, Z. C. (2023) · Clinical Orthopaedics & Related Research · doi:10.1097/corr.0000000000002704 · Cited by 152 · cited 1×
“ChatGPT itself has been tested with the American Board of Orthopaedic Surgery Examination, but it cannot pass the exam [17].”
MethodThe credibility of dietary advice formulated by ChatGPT: Robo-diets for people with food allergies
Niszczota, P., & Rybicka, I. (2023) · Nutrition · doi:10.1016/j.nut.2023.112076 · Cited by 138
Seth, I., Cox, A., Xie, Y., Bulloch, G., Hunter-Smith, D. J., Rozen, W. M., & Ross, R. J. (2023) · Aesthetic Surgery Journal · doi:10.1093/asj/sjad140 · Cited by 122 · cited 1×
“It could provide a well-structured and comprehensive response to common breast augmentation surgery questions [19].”
BackgroundLi, S. W., Kemp, M. W., Logan, S. J. S., Dimri, P. S., Singh, N., Mattar, C. N. Z., Dashraath, P., Ramlal, H., Mahyuddin, A. P., Kanayan, S., et al. (2023) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2023.04.020 · Cited by 116
Evaluating Artificial Intelligence Responses to Public Health Questions
Ayers, J. W., Zhu, Z., Poliak, A., Leas, E. C., Dredze, M., Hogarth, M., & Smith, D. M. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.17517 · Cited by 115 · cited 1×
“For example, in responding to public health questions, 91% of the answers given were determined to be based on evidence [21].”
BackgroundWang, H., Gupta, S., Singhal, A., Muttreja, P., Singh, S., Sharma, P., & Piterova, A. (2022) · Journal of Medical Internet Research · doi:10.2196/29969 · Cited by 111 · cited 1×
“For example, the SnehAI chatbot was developed to educate adolescents in India about sexual health and showed promising results [25].”
BackgroundPractical Applications of ChatGPT in Undergraduate Medical Education
Tsang, R. (2023) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205231178449 · Cited by 104 · cited 1×
“Using ChatGPT for preparing for OSCE and other exams was mentioned by participants and in the literature [28].”
MethodChatGPT failed Taiwan’s Family Medicine Board Exam
Weng, T.-L., Wang, Y.-M., Chang, S., Chen, T.-J., & Hwang, S.-J. (2023) · Journal of the Chinese Medical Association · doi:10.1097/jcma.0000000000000946 · Cited by 100 · cited 1×
“For more advanced examination levels, such as resident-level examinations, ChatGPT performed more poorly [7,34,35].”
BackgroundRoos, J., Kasapovic, A., Jansen, T., & Kaczmarczyk, R. (2023) · JMIR Medical Education · doi:10.2196/46482 · Cited by 98
Lahat, A., Shachar, E., Avidan, B., Glicksberg, B., & Klang, E. (2023) · Diagnostics · doi:10.3390/diagnostics13111950 · Cited by 92
Performance of ChatGPT on the Peruvian National Licensing Medical Examination: Cross-Sectional Study
Flores-Cohaila, J. A., García-Vicente, A., Vizcarra-Jiménez, S. F., De la Cruz-Galán, J. P., Gutiérrez-Arratia, J. D., Quiroga Torres, B. G., & Taype-Rondan, A. (2023) · JMIR Medical Education · doi:10.2196/48039 · Cited by 82 · cited 2×
“Many researchers have also evaluated ChatGPT for medical education, such as taking examinations and comparing the results to medical students [7-11].”
BackgroundResultA Critical Review of ChatGPT as a Potential Substitute for Diabetes Educators
Sharma, S., Pajai, S., Prasad, R., Wanjari, M. B., Munjewar, P. K., Sharma, R., & Pathade, A. (2023) · Cureus · doi:10.7759/cureus.38380 · Cited by 64 · cited 1×
“ChatGPT had the potential to be used as a diabetic educator [23].”
BackgroundHuang, R. S., Lu, K. J. Q., Meaney, C., Kemppainen, J., Punnett, A., & Leung, F.-H. (2023) · JMIR Medical Education · doi:10.2196/50514 · Cited by 56 · cited 1×
“GPT-4 was able to pass the Peruvian National Licensing Examination, the Japanese Medical Licensing Examination, German medical state examinations, and the Family Medicine Residency Progress Test with exceptional scores [11,36-38].”
BackgroundHsu, H.-Y., Hsu, K.-C., Hou, S.-Y., Wu, C.-L., Hsieh, Y.-W., & Cheng, Y.-D. (2023) · JMIR Medical Education · doi:10.2196/48433 · Cited by 50 · cited 1×
“ChatGPT correctly answered 61% of basic public medical consultations, but only 39% of questions asked by health care personnel were correctly answered [24].”
BackgroundWeeks, R., Cooper, L., Sangha, P., Sedoc, J., White, S., Toledo, A., Gretz, S., Lahav, D., Martin, N., Michel, A. E., et al. (2022) · Journal of Medical Internet Research · doi:10.2196/38418 · Cited by 33 · cited 1×
“Another chatbot, “VIRA,” was created XSL • FO RenderX to communicate and ensure COVID-19 vaccine safety with young adults and minority populations [26].”
BackgroundSalimi, A., & Saheb, H. (2023) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2023.06.004 · Cited by 32 · cited 1×
“In the medical field, it has been used to help write manuscripts [3-5].”
BackgroundNew Challenges in Scientific Publications: Referencing, Artificial Intelligence and ChatGPT
Švab, I., Klemenc-Ketiš, Z., & Zupanič, S. (2023) · Slovenian Journal of Public Health · doi:10.2478/sjph-2023-0015 · Cited by 29
Machine Learning for Detecting Total Knee Arthroplasty Implant Loosening on Plain Radiographs
Kim, M. S., Cho, R.-K., Yang, S.-C., Hur, J. H., & In, Y. (2023) · Bioengineering · doi:10.3390/bioengineering10060632 · Cited by 24 · cited 1×
“For example, deep learning AI has been used for detecting and classifying many orthopedic conditions, such as degenerative spinal conditions, rotator cuff injury, and implant loosening [14-16].”
MethodLi, X., Xie, S., Ye, Z., Ma, S., & Yu, G. (2022) · Journal of Medical Internet Research · doi:10.2196/40681 · Cited by 20 · cited 1×
“However, it evolved and became involved in other aspects of the medical field, for example, helping physicians gather patient data before the visit [1].”
BackgroundHueber, A. J., & Kleyer, A. (2023) · RMD Open · doi:10.1136/rmdopen-2023-003248 · Cited by 16 · cited 1×
“However, there were reports of ChatGPT citing nonexistent publications when asked [22].”
BackgroundBenhenneda, R., Brouard, T., Charousset, C., & Berhouet, J. (2023) · Orthopaedics & Traumatology Surgery & Research · doi:10.1016/j.otsr.2023.103652 · Cited by 16
Coppock, J. A., Zimmer, N. E., Spritzer, C. E., Goode, A. P., & DeFrate, L. E. (2023) · Osteoarthritis and Cartilage Open · doi:10.1016/j.ocarto.2023.100378 · Cited by 11 · cited 1×
“For example, deep learning AI has been used for detecting and classifying many orthopedic conditions, such as degenerative spinal conditions, rotator cuff injury, and implant loosening [14-16].”
Method
Cited by
Large Language Models in Medicine: Applications, Challenges, and Future Directions
Yu, E., Chu, X., Zhang, W., Meng, X., Yang, Y., Ji, X., & Wu, C. (2025) · International Journal of Medical Sciences · doi:10.7150/ijms.111780 · Cited by 175
“An increasing amount of research indicates that medical students have positive attitudes toward the application of LLMs to assist medical education [56].”
Tao, W., Yang, J., & Qu, X. (2024) · JMIR Medical Education · doi:10.2196/57132 · Cited by 71
“While research on AI chatbot applications in medical students has emerged [7-10], the utilization of, perceptions on, and intention to use AI chatbots among Chinese medical students are still unknown.”
BackgroundAn automated framework for assessing how well LLMs cite relevant medical references
Wu, K., Wu, E., Wei, K., Zhang, A., Casasola, A., Nguyen, T., Riantawan, S., Shi, P., Ho, D., & Zou, J. (2025) · Nature Communications · doi:10.1038/s41467-025-58551-6 · Cited by 72
“A growing number of clinicians report using LLMs in their clinical practice or education 8,9 .”
ChatGPT’s Response Consistency: A Study on Repeated Queries of Medical Examination Questions
Funk, P. F., Hoch, C. C., Knoedler, S., Knoedler, L., Cotofana, S., Sofo, G., Bashiri Dezfouli, A., Wollenberg, B., Guntinas-Lichius, O., & Alfertshofer, M. (2024) · European Journal of Investigation in Health, Psychology and Education · doi:10.3390/ejihpe14030043 · Cited by 69
Artificial intelligence in undergraduate medical education: an updated scoping review
Simoni, J., Urtubia-Fernandez, J., Mengual, E., Simoni, D. A., Royo, M., Egaña-Yin, D., Hertog, O. L. A., López-Ortiz, L., Muñoz-Tomás, A., Santiago-Martínez, P., et al. (2025) · BMC Medical Education · doi:10.1186/s12909-025-08188-2 · Cited by 62
“Eight studies examined the knowledge, attitudes, and skills of medical students and educators regarding LLMs, specifically ChatGPT [24, 26–29 , 79– 81].”
Metadata
- Authors
- Tangadulrat, P., Sono, S., & Tangtrakulwanich, B.
- Affiliations
- Prince of Songkla University (TH)
- Corresponding author
- Tangtrakulwanich, Boonsin
- Venue
- JMIR Medical Education
- Publisher
- JMIR Publications Inc.
- Published
- 2023-12-22
- Volume
- 9
- Pages
- e50658
- ISSN
- 2369-3762 (ISSN-L)
- PMC
- PMC10770783
- OpenAlex
- W4390114552
- Cited by
- 99
- References
- 39
- Citations by year
- 2026: 36 · 2025: 31 · 2024: 32
- FWCI
- 3.95
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Medical Education (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Greater South Information System
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Technology and Human Factors in Education and Health
- Social Sciences › Education › Artificial Intelligence in Education
- Keywords
- ChatGPT, large language models, medical students, physicians, artificial intelligence in medical education, knee osteoarthritis, patient education materials
- MeSH terms
- Cross-Sectional Studies, Education, Medical, Humans, Language, Physicians, Students, Medical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.