Ethical Considerations and Fundamental Principles of Large Language Models in Medical Education: Viewpoint
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
This viewpoint article first explores the ethical challenges associated with the future application of large language models (LLMs) in the context of medical education. These challenges include not only ethical concerns related to the development of LLMs, such as artificial intelligence (AI) hallucinations, information bias, privacy and data risks, and deficiencies in terms of transparency and interpretability but also issues concerning the application of LLMs, including deficiencies in emotional intelligence, educational inequities, problems with academic integrity, and questions of responsibility and copyright ownership. This paper then analyzes existing AI-related legal and ethical frameworks and highlights their limitations with regard to the application of LLMs in the context of medical education. To ensure that LLMs are integrated in a responsible and safe manner, the authors recommend the development of a unified ethical framework that is specifically tailored for LLMs in this field. This framework should be based on 8 fundamental principles: quality control and supervision mechanisms; privacy and data protection; transparency and interpretability; fairness and equal treatment; academic integrity and moral norms; accountability and traceability; protection and respect for intellectual property; and the promotion of educational research and innovation. The authors further discuss specific measures that can be taken to implement these principles, thereby laying a solid foundation for the development of a comprehensive and actionable ethical framework. Such a unified ethical framework based on these 8 fundamental principles can provide clear guidance and support for the application of LLMs in the context of medical education. This approach can help establish a balance between technological advancement and ethical safeguards, thereby ensuring that medical education can progress without compromising the principles of fairness, justice, or patient safety and establishing a more equitable, safer, and more efficient environment for medical education.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · cited 1×
“A well-known test of the accuracy of LLMs in the context of medical questions is ChatGPT's successful passing of the United States Medical Licensing Exam [40].”
Kung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 1×
“Failure to address training data bias and adjust algorithms perpetuates unfairness in terms of clinical practices [24].”
Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., et al. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1838 · Cited by 2,613 · cited 1×
“Although LLMs are trained to provide empathetic responses in the context of engaging with patients [28]—recent research has shown that ChatGPT even exhibits higher levels of empathy when consulting on systemic sclerosis than do neurologists [29]—the artificial empathy expressed by AI generally…”
Artificial hallucinations in ChatGPT: implications in scientific writing
Alkaissi, H., & McFarlane, S. I. (2023) · Cureus · doi:10.7759/cureus.35179 · Cited by 1,540 · cited 1×
“In the field of medicine, AI hallucinations occur when LLMs generate responses that appear to be logical but are actually incorrect, inconsistent, or fabricated, including those that rely on fabricated data and forged references [14].”
Dave, T., Athaluri, S. A., & Singh, S. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1169595 · Cited by 1,204
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00819-6 · Cited by 1,099
ChatGPT and other large language models are double-edged swords
Shen, Y., Heacock, L., Elias, J., Hentel, K. D., Reig, B., Shih, G., & Moy, L. (2023) · Radiology · doi:10.1148/radiol.230163 · Cited by 1,008
What ChatGPT and generative AI mean for science
Stokel-Walker, C., & Van Noorden, R. (2023) · Nature · doi:10.1038/d41586-023-00340-6 · Cited by 828 · cited 1×
“This issue is known as the “black box” effect, in which context transparency and interpretability are lacking [26].”
Estimating the success of re-identifications in incomplete datasets using generative models
Rocher, L., Hendrickx, J. M., & de Montjoye, Y.-A. (2019) · Nature Communications · doi:10.1038/s41467-019-10933-3 · Cited by 816 · cited 1×
“Rocher et al [11] reported that LLMs could use as few as 15 demographic attributes to reidentify 99.98% of personal data in any anonymized data set.”
InfluentialLarge Language Models in Medical Education: Opportunities, Challenges, and Future Directions
Abd-alrazaq, A., AlSaad, R., Alhuwail, D., Ahmed, A., Healy, P. M., Latifi, S., Aziz, S., Damseh, R., Alabed Alrazak, S., & Sheikh, J. (2023) · JMIR Medical Education · doi:10.2196/48291 · Cited by 722
A Conversation on Artificial Intelligence, Chatbots, and Plagiarism in Higher Education
King, M. R., & chatGPT (2023) · Cellular and Molecular Bioengineering · doi:10.1007/s12195-022-00754-8 · Cited by 602
Generating scholarly content with ChatGPT: Ethical challenges for medical publishing
Liebrenz, M., Schleifer, R., Buadze, A., Bhugra, D., & Smith, A. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00019-5 · Cited by 594 · cited 1×
“To protect copyrights and maintain academic integrity more effectively, current guidelines issued by the International Committee of Medical Journal Editors and the Committee on Public Ethics suggest that LLMs such as ChatGPT should not be listed as authors of papers [47].”
ChatGPT and the future of medical writing
Biswas, S. (2023) · Radiology · doi:10.1148/radiol.223312 · Cited by 571 · cited 1×
“They also offer relatively equal learning opportunities to regions located in remote geographical areas and enable nonnative speakers to access and understand medical knowledge in other languages through cross-language support [33].”
Ethical Considerations of Using ChatGPT in Health Care
Wang, C., Liu, S., Yang, H., Guo, J., Wu, Y., & Liu, J. (2023) · Journal of Medical Internet Research · doi:10.2196/48009 · Cited by 541
Zack, T., Lehman, E., Suzgun, M., Rodriguez, J. A., Celi, L. A., Gichoya, J., Jurafsky, D., Szolovits, P., Bates, D. W., Abdulnour, R.-E. E., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00225-x · Cited by 510 · cited 2×
“Additionally, treatment suggestions have been shown to be linked to demographic factors, to favor costlier procedures and to exhibit variations in terms of patient perceptions [21].”
Ethics and governance of trustworthy medical artificial intelligence
Zhang, J., & Zhang, Z.-M. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02103-9 · Cited by 507 · cited 1×
“From a legal perspective, AI lacks the legal standing of a human being, thus leaving humans as the ultimate accountable parties [45].”
Principles of Biomedical Ethics : Marking Its Fortieth Anniversary
Beauchamp, T., & Childress, J. (2019) · The American Journal of Bioethics · doi:10.1080/15265161.2019.1665402 · Cited by 493
The future of medical education and research: Is ChatGPT a blessing or blight in disguise?
Arif, T. B., Munaf, U., & Ul-Haque, I. (2023) · Medical Education Online · doi:10.1080/10872981.2023.2181052 · Cited by 317
CheXclusion: Fairness gaps in deep chest X-ray classifiers
Seyyed-Kalantari, L., Liu, G., McDermott, M., Chen, I. Y., & Ghassemi, M. (2020) · Biocomputing 2021 · doi:10.1142/9789811232701_0022 · Cited by 275 · cited 1×
“A study on a chest x-ray classifier that had been trained mainly on White patients revealed that it lacked recognition capability in cases involving non-White patients [22].”
Rapamycin in the context of Pascal's Wager: generative pre-trained transformer perspective
Generative Pre-trained Transformer, C., & Zhavoronkov, A. (2022) · Oncoscience · doi:10.18632/oncoscience.571 · Cited by 203
Artificial intelligence in medical education curriculum: An e-Delphi study for competencies
Çalışkan, S. A., Demir, K., & Karaca, O. (2022) · PLOS ONE · doi:10.1371/journal.pone.0271872 · Cited by 115 · cited 1×
“While previous research on education-related LLMs has emphasized the tasks of enhancing key competencies or exploring educational LLM programs and related concepts, it has failed to account for ethical principles specifically in the development and application of such technologies [7,8].”
Zong, H., Li, J., Wu, E., Wu, R., Lu, J., & Shen, B. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05125-7 · Cited by 114
Ong, J. C. L., Seng, B. J. J., Law, J. Z. F., Low, L. L., Kwa, A. L. H., Giacomini, K. M., & Ting, D. S. W. (2024) · Cell Reports Medicine · doi:10.1016/j.xcrm.2023.101356 · Cited by 111
Marchandot, B., Matsushita, K., Carmona, A., Trimaille, A., & Morel, O. (2023) · European Heart Journal Open · doi:10.1093/ehjopen/oead007 · Cited by 99 · cited 1×
“However, this approach is not universally accepted, and the role of LLMs in authorship declarations remains the subject of ongoing discussion by various publishing institutions [48,49].”
Busch, F., Adams, L. C., & Bressem, K. K. (2023) · Medical Science Educator · doi:10.1007/s40670-023-01815-x · Cited by 88 · cited 1×
“Medical education should not only aim to cultivate these principles among students but also continuously evolve and improve under their guidance [6].”
AI in Medicine—JAMA’s Focus on Clinical Outcomes, Patient-Centered Care, Quality, and Equity
Khera, R., Butte, A. J., Berkwits, M., Hswen, Y., Flanagin, A., Park, H., Curfman, G., & Bibbins-Domingo, K. (2023) · JAMA · doi:10.1001/jama.2023.15481 · Cited by 78
ChatGPT in Medical Education and Research: A Boon or a Bane?
Jeyaraman, M., K, S. P., Jeyaraman, N., Nallakumarasamy, A., Yadav, S., & Bondili, S. K. (2023) · Cureus · doi:10.7759/cureus.44316 · Cited by 58 · cited 1×
“Excessive reliance on LLMs may decrease students' research ability, critical thinking, innovation, and motivation to engage in proper learning [44].”
Patino, G. A., Amiel, J. M., Brown, M., Lypson, M. L., & Chan, T. M. (2024) · Academic Medicine · doi:10.1097/acm.0000000000005636 · Cited by 43 · cited 1×
“In the context of medical education, the lack of transparency and interpretability in LLMs can prevent teachers and students from assessing the accuracy of information effectively [27].”
Cited by
Large Language Models in Healthcare and Medical Applications: A Review
Maity, S., & Saikia, M. J. (2025) · Bioengineering · doi:10.3390/bioengineering12060631 · Cited by 249
“By analyzing the latest research literature and clinical trials, these models can provide concise summaries of emerging evidence, facilitating the integration of new findings into clinical practice [5,28,33].”
The Importance of AI Data Governance in Large Language Models
(2025) · doi:10.3390/bdcc9060147 · Cited by 58
The Importance of AI Data Governance in Large Language Models
Pahune, S., Akhtar, Z., Mandapati, V., & Siddique, K. (2025) · doi:10.20944/preprints202504.0219.v1 · Cited by 58
Miao, Y., Zhao, Y., Luo, Y., Wang, H., & Wu, Y. (2025) · Journal of Medical Internet Research · doi:10.2196/80557 · Cited by 38
Esmaeilzadeh, P. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03182-6 · Cited by 26
Metadata
- Authors
- Zhui, L., Fenghe, L., Xuehu, W., Qining, F., & Wei, R.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2024-08-01
- Volume
- 26
- Pages
- e60083
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W4400392484
- Cited by
- 91
- References
- 30
- FWCI
- 7.30
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.