The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Ethical and regulatory challenges of large language models in medicine
Open accessAbstract
With the rapid growth of interest in and use of large language models (LLMs) across various industries, we are facing some crucial and profound ethical concerns, especially in the medical field. The unique technical architecture and purported emergent abilities of LLMs differentiate them substantially from other artificial intelligence (AI) models and natural language processing techniques used, necessitating a nuanced understanding of LLM ethics. In this Viewpoint, we highlight ethical concerns stemming from the perspectives of users, developers, and regulators, notably focusing on data privacy and rights of use, data provenance, intellectual property contamination, and broad applications and plasticity of LLMs. A comprehensive framework and mitigating strategies will be imperative for the responsible integration of LLMs into medical practice, ensuring alignment with ethical principles and safeguarding against potential societal risks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Second, it may erode academic credibility as the boundary between authentic scholarship and machine-generated output becomes increasingly blurred (Ong et al., 2024).”
From The Attention Mismatch: Mapping the Structural Academic Governance Deficit in the Age of Generative AI (2026) · cites this paper as Ong et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Specifically, in the health care domain, data often exhibit bias due to the uneven distribution of demographic attributes, preconceptions held by health care professionals involved in data collection and interpretation, and the varied academic and experiential backgrounds that influence their perspectives [175-177].”
From Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook. (2024) · cites this paper as [175]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“If M-LLMs are trained on patient data that contain biases related to gender, ethnicity, socioeconomic status, or geographic location, they may inadvertently cause biases in their predictions or recommendations [175,178,179].”
From Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook. (2024) · cites this paper as [175]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, data pre-processing decisions vary drastically across research teams, which can lead to integrating sensitive or protected health information (PHI) into LLMs, creating security, privacy, and regulatory risks.3”
From Development of secure infrastructure for advancing generative artificial intelligence research in healthcare at an academic medical center (2025) · cites this paper as [3]
Experimental — accuracy not yet measuredIn the citing paper
“There is a potential for misuse of patient data, medicolegal implications, and loss of physician-patient trust.181,192”
From Large Language Models for Chatbot Health Advice Studies: A Systematic Review (2025) · cites this paper as [181]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944
Foundation models for generalist medical artificial intelligence
Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., Topol, E. J., & Rajpurkar, P. (2023) · Nature · doi:10.1038/s41586-023-05881-4 · Cited by 2,028
The imperative for regulatory oversight of large language models (or generative AI) in healthcare
Meskó, B., & Topol, E. J. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00873-0 · Cited by 1,047
A governance model for the application of AI in health care
Reddy, S., Allan, S., Coghlan, S., & Cooper, P. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz192 · Cited by 979
Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist.
Norgeot, B., Quer, G., Beaulieu-Jones, B. K., Torkamani, A., Dias, R., Gianfrancesco, M., Arnaout, R., Kohane, I. S., Saria, S., Topol, E., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1041-y · Cited by 665
Feng, J., Phillips, R. V., Malenica, I., Bishara, A., Hubbard, A. E., Celi, L. A., & Pirracchio, R. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00611-y · Cited by 423
Collingridge and the dilemma of control: Towards responsible and accountable innovation
Genus, A., & Stirling, A. (2018) · Research Policy · doi:10.1016/j.respol.2017.09.012 · Cited by 301
Auditing Large Language Models: A Three-Layered Approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2023) · doi:10.2139/ssrn.4361607 · Cited by 292
Auditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
Liu, X., Glocker, B., McCradden, M. M., Ghassemi, M., Denniston, A. K., & Oakden-Rayner, L. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(22)00003-6 · Cited by 266
Cited by
Sorted by citations
Items 301–342 of 342
Tan, M., Tang, S., Kan, S., Wu, B., Ni, Z., Zhang, H., & Ding, J. (2026) · Journal of Medical Internet Research · doi:10.2196/87288 · Cited by 1
Multimodal natural language processing in ophthalmology: bridging clinical text and medical imaging
Qin, S., Zhao, N., Shen, L., Zhang, Y., & Ge, S. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1762586
“The integration of multimodal NLP systems in ophthalmology encounters significant regulatory challenges, particularly in meeting the rigorous standards set by agencies like the FDA and NMPA (96, 97).”
Exploring Bias in Medical Applications of Large Language Models: Protocol for a Systematic Review
Rao, S. G., Boutrous, C., Kormilitzin, A., Tai, X. Y., Hotchkiss, L., Cong, C., Meinert, E., Liang, H., Dong, H., & Harrison, J. R. (2026) · JMIR Research Protocols · doi:10.2196/91348
Fan, T., Gao, T., Tang, W., Xu, Q., Wang, X., Su, Q., & Lyu, Q. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1899528
“Clear responsibility should be assigned for reviewing, approving, correcting, and reporting AI-generated outputs, because accountability can become ambiguous when model-generated content is incorporated into teaching materials, feedback, or documentation-related education (52, 53).”
InfluentialDescamps, J., Bouguennec, N., Porcher, R., Resche‐Rigon, M., & Bouché, P.-A. (2026) · Revue de Chirurgie Orthopédique et Traumatologique · doi:10.1016/j.rcot.2026.07.013
Koohsari, M. J., Loo, B. P. Y., Zhao, J., Li, J., Long, Y., Lu, Y., Oka, K., & Kaczynski, A. T. (2026) · Developments in the Built Environment · doi:10.1016/j.dibe.2026.101007
Bleuze, C., Fort, K., Martin, V., & Névéol, A. (2026) · JMIR AI · doi:10.2196/88082
“Moreover, existing evaluation settings for biomedical LLMs are limited [10,19] and real-world impact studies are still lacking in the NLP community [20], leaving many ethical and regulatory challenges unaddressed [21].”
Peng, J., Tuo, Y., Wang, G., 韩瑞琴, Wang, Q., Wang, Y., Hu, Z., & Hu, X. (2026) · Visual Computing for Industry Biomedicine and Art · doi:10.1186/s42492-026-00228-y
“However, their adoption has raised critical concerns regarding their fairness, toxicity, privacy, and safe deployment, particularly in regulated clinical environments [7].”
Large artificial intelligence models for coal mining: Development, applications and challenges
Wang, G., Luo, Y., Yang, L., Wang, D., & Cheng, J. (2026) · International Journal of Minerals Metallurgy and Materials · doi:10.1007/s12613-026-3492-8
The Diagnostic Mirage: Path Forward for Artificial Intelligence in Clinical Diagnosis
Mukku, L. (2026) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.70571
Shikino, K., Nishizaki, Y., Kataoka, K., Fukui, S., Sakamaki, K., Yamamoto, Y., Shimizu, T., Sakamoto, S., Morishima, R., Wakabayashi, T., et al. (2026) · doi:10.2196/preprints.110021
Kaur, A., & Siddharth, R. (2026) · Indian journal of clinical psychology · doi:10.4103/ijcp.ijcp_21_26
Xu, T., Yin, Y., Zhang, X., Wei, W., Yuan, J., Wang, H., Ding, G., Xue, W., Chen, Z., Xie, S., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/98118
Khormali, A., Soghli, N., & Miguez, P. A. (2026) · Periodontology 2000 · doi:10.1111/prd.70078 · Cited by 1
Foundation Models in Healthcare Systems: Architectural Integration and Oversight Considerations
Costa, R., Teixeira, B., & Santos, V. (2026) · Journal of Artificial Intelligence for Healthcare Systems · doi:10.68159/g765182791
Generative AI in Clinical Workflows: Documentation Utility, Failure Modes, and Oversight Mechanisms
Petrova, E., & Georgiev, I. (2024) · Journal of Health Informatics and Digital Systems · doi:10.68159/d070156303
Large Language Models in Clinical Contexts: Infrastructure, Oversight, and Risk Dynamics
Müller, A., Weber, S., Hoffmann, J., Schneider, L., & Klein, T. (2025) · Journal of Artificial Intelligence for Healthcare Systems · doi:10.68159/j510298368
Prabha, S., Collaço, B. G., Malena, C. R., Haider, S. A., Wood, N. G., & Forte, A. J. (2026) · Bioengineering · doi:10.3390/bioengineering13101097
“We outline several categories of evidence required before these architectures can be safely introduced into patient care [46,49,52].”
The Role of Generative AI in Cybersecurity Defensive Operations for Critical Infrastructure
Ha, S., & Wang, P. (2026) · Advances in intelligent systems and computing · doi:10.1007/978-3-032-27214-0_7
Sun, Y., Yi, Z., Mao, J., Tang, M., Ma, X., Chen, R., Guo, Z., Tong, X., & Gu, N. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3817002
“Third, our work remains subject to broader limitations inherent to GenAI-powered NPIs, particularly those associated with GenAI, such as black-box characteristic-caused trust and accountability [23, 86], privacy and safety [5, 62], and data governance [70, 94].”
Fróes, L. T.-C., & Klein, A. Z. (2026) · The International Journal of Entrepreneurship and Innovation · doi:10.1177/14657503261492574
Arslan, M., & Ümit, T. B. (2026) · Journal of Emergency Medicine · doi:10.1016/j.jemermed.2026.03.038
Rovis, M., Smajić, A., Miličević, M., Šegota, S. B., Mrzljak, V., Gršković, A., Ahel, J., Smolić, K., Markić, D., & Lorencin, I. (2026) · Bioengineering · doi:10.3390/bioengineering13101123
Sung, E. (2026) · Kosin Medical Journal · doi:10.7180/kmj.26.192
“Le-gal Le-gal and product-liability frameworks will need to be updated to define accountability clearly according to the level of autonomy the system was exercising at the time of the inci-dent—protecting patients without discouraging continued innovation [20].”
Liu, L., Du, X., Yang, G., Xiang, S., Guo, H., Huang, L., Li, Y., & Yu, H. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1862627
Hu, G., Lai, Q., Liu, S., Xie, B., Wu, Z., Zhuang, Q., Tang, H., & Xu, J. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1961597
Lim, E. Y. P., Chen, F., & Lim, C. E. D. (2026) · Journal of Mind and Medical Sciences · doi:10.3390/jmms13040024
Liang, J., Wang, Y., Ni, J., Cai, Y., Chen, K., Wang, J., Shi, Y., Chen, B., Dong, C., Guo, T., et al. (2026) · Nature Health · doi:10.1038/s44360-026-00189-9
Macharia, P. M., Hartley, A., Ding, J., Kan, S., Currie, S., Wamalwa, P., Ngugi, L., Nyongesa, T., Mukhwana, R., Mbwayo, A., et al. (2026) · Open Research Africa · doi:10.12688/openresafrica.16217.2
Weber, A., Müller, T., Nair, P., & Anderson, J. (2026) · Asian Journal of Ethics in Health and Medicine · doi:10.51847/y4wmkd6kz5
Liu, L.-X., Wang, H., Lu, S., Yu, G., Li, J.-F., Lai, Q.-H., Guo, B., Dong, M., Zhang, L., Ye, M.-X., et al. (2026) · Digital Health · doi:10.1177/20552076261493266
Ambient Digital Scribes as Educational Mirrors: Closing the Feedback Gap in Physician Communication
Park, J. J., & Park, A. S. (2026) · The American Journal of Bioethics · doi:10.1080/15265161.2026.2741285
Sharing blame with me: How AI anthropomorphism encourages individual unethical behavior
Fei, X., Yu, Y., & Li, Y. (2026) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2026.103142
Qin, N., Shao, X., Wang, J., Lei, J., Wen, R., Wu, W., Liu, F., Wang, H., & Zhu, Y. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1966612
“…the need for guide-line-aligned verification, explicit safeguards against misinformation, and clinician oversight before LLM-generated HF information is used in patient-facing settings, particularly for medication use, symptom escalation, device therapy, and procedural decision-making (23, 24).”
Large Language Models in Hearing CarePreprint
Wang, S., Mo, C., Wu¹, J., Siu, A. K. Y., Zhao, F., Wasmann, J.-W. A., Swanepoel, D. W., & Manchaiah, V. K. C. (2026) · Research Square · doi:10.21203/rs.3.rs-11384026/v1
Beste, N. C., Bauer, F., Lopez, N., Schoemig, T., Schoenfeld, M., Zopfs, D., Goertz, L., ten Persigehl, T., Kabbasch, C., Schlamann, M., et al. (2026) · medRxiv · doi:10.64898/2026.10.17.26365809
ChikungunyaQA: A Clinical Dataset of Questions and Answers on Chikungunya
dos Santos, D. A., Moreira, R., da Costa, A. C. V., de Barros, G. M., Santos, B., Teixeira, S. S., & Teles, A. S. (2026) · Data · doi:10.3390/data11100279
Zhao, X., Zhang, X., & Liu, L. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1961647
Chuah, C. S., Gros, B., & Plevris, N. (2026) · BMJ Connections Digital Health & AI · doi:10.1136/bmjcdh-2026-000041
Oishi, S., Hiraide, M., Kawaguchi, T., Ogata, M., Masuyama, K., Ohta, T., Hori, Y., & Hanai, A. (2026) · doi:10.2196/preprints.115452
Servet, E., Unat, B., Bilgin, M. A., Karabulut, Ç., & Gönder, N. (2026) · Journal of Children s Orthopaedics · doi:10.1177/18632521261500230
“The completeness index was the one exception, being higher for Claude (85.0”
InfluentialThe opportunities and risks of large language models in mental health
Lawrence, H. R., Schneider, R. A., Rubin, S. B., Mataric, M. J., McDuff, D. J., & Bell, M. J. (2024) · arXiv · doi:10.48550/arxiv.2403.14814
Metadata
- Authors
- Ong, J. C. L., Chang, S. Y.-H., William, W., Butte, A. J., Shah, N. H., Chew, L. S. T., Liu, N., Doshi-Velez, F., Lu, W., Savulescu, J., & Ting, D. S. W.
- Affiliations
- National University of Singapore (SG)
- University of California, San Francisco (US)
- Stanford Health Care (US)
- SingHealth (SG)
- Corresponding author
- Ting, Daniel Shu Wei
- ORCID
- Venue
- The Lancet Digital Health
- Publisher
- Elsevier BV
- Published
- 2024-06
- Volume
- 6
- Issue
- 6
- Pages
- e428-e432
- ISSN
- 2589-7500 (ISSN-L)
- OpenAlex
- W4395026179
- Cited by
- 342
- References
- 23
- Citations by year
- 2026: 165 · 2025: 136 · 2024: 41
- FWCI
- 31.35
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- The Lancet Digital Health (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- A*STAR
- National University of Singapore (NUHSRO/2022/078/Startup/13, NUS Start-Up)
- Duke-NUS Medical School
- Wellcome Trust (WT203132/Z/16/Z)
- National Medical Research Council
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Law, AI, and Intellectual Property
- Keywords
- large language models, medical ethics, AI regulation, data privacy, data provenance, intellectual property, user rights, responsible AI, medical practice, societal risks of AI
- MeSH terms
- Artificial Intelligence, Humans, Natural Language Processing, Intellectual Property
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
