CoChat
Sign Up
Cited by 342Artificial Intelligence in Healthcare and EducationEthics and Social Impacts of AILaw, AI, and Intellectual Property

Ethical and regulatory challenges of large language models in medicine

Open access
Ong, Jasmine Chiat Ling, Chang, Shelley Yin-Hsi, William, Wasswa, Butte, Atul J, Shah, Nigam H, Chew, Lita Sui Tjien, Liu, Nan, Doshi-Velez, Finale +3 more·2024·The Lancet Digital Health
DOI 10.1016/s2589-7500(24)00061-x
Open this paper in CoChat — free accountOpen PDF

Abstract

With the rapid growth of interest in and use of large language models (LLMs) across various industries, we are facing some crucial and profound ethical concerns, especially in the medical field. The unique technical architecture and purported emergent abilities of LLMs differentiate them substantially from other artificial intelligence (AI) models and natural language processing techniques used, necessitating a nuanced understanding of LLM ethics. In this Viewpoint, we highlight ethical concerns stemming from the perspectives of users, developers, and regulators, notably focusing on data privacy and rights of use, data provenance, intellectual property contamination, and broad applications and plasticity of LLMs. A comprehensive framework and mitigating strategies will be imperative for the responsible integration of LLMs into medical practice, ensuring alignment with ethical principles and safeguarding against potential societal risks.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/s2589-7500(24)00061-x

Cites

24 of 24 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Synthetic contamination may erode academic credibility as the boundary between authentic scholarship and machine-generated output becomes increasingly blurred.

    “Second, it may erode academic credibility as the boundary between authentic scholarship and machine-generated output becomes increasingly blurred (Ong et al., 2024).”

    From The Attention Mismatch: Mapping the Structural Academic Governance Deficit in the Age of Generative AI (2026) · cites this paper as Ong et al., 2024

    The “Data provenance and intellectual property contamination” section (p. e430) says watermarking seeks to address originality and ownership concerns for AI-generated content. It also discusses a proposed platform for medical research publishing intended to preserve original work and minimise fraud. It does not say that synthetic contamination erodes academic credibility or that the boundary between authentic scholarship and machine-generated output is becoming blurred. (LLM, conf=0.72)

    Experimental — accuracy not yet measured

  • In the health care domain, data often exhibit bias due to the uneven distribution of demographic attributes, preconceptions held by health care professionals involved in data collection and interpretation, and the varied academic and experiential backgrounds that influence their perspectives.

    “Specifically, in the health care domain, data often exhibit bias due to the uneven distribution of demographic attributes, preconceptions held by health care professionals involved in data collection and interpretation, and the varied academic and experiential backgrounds that influence their perspectives [175-177].”

    From Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook. (2024) · cites this paper as [175]

    The ‘Data provenance and intellectual property contamination’ section (page not available) says that biases can be present in medical LLM training data and calls for transparency about data diversity. It does not identify uneven demographic distribution, health care professionals’ preconceptions, or their academic and experiential backgrounds as causes of those biases. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • If M-LLMs are trained on patient data that contain biases related to gender, ethnicity, socioeconomic status, or geographic location, they may inadvertently cause biases in their predictions or recommendations.

    “If M-LLMs are trained on patient data that contain biases related to gender, ethnicity, socioeconomic status, or geographic location, they may inadvertently cause biases in their predictions or recommendations [175,178,179].”

    From Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook. (2024) · cites this paper as [175]

    The Data provenance and intellectual property contamination section says biases in training data can affect LLM accuracy, and the Broad applications and plasticity section identifies propagation of bias or discrimination as a risk in medicine. Our copy does not specify bias related to gender, ethnicity, socioeconomic status, or geographic location in patient training data, so those details are not supported by this source. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Data pre-processing decisions vary drastically across research teams, which can lead to integrating sensitive or protected health information (PHI) into LLMs, creating security, privacy, and regulatory risks.

    “Furthermore, data pre-processing decisions vary drastically across research teams, which can lead to integrating sensitive or protected health information (PHI) into LLMs, creating security, privacy, and regulatory risks.3”

    From Development of secure infrastructure for advancing generative artificial intelligence research in healthcare at an academic medical center (2025) · cites this paper as [3]

    The ‘Data privacy and data rights of use’ section (page not provided) says that using identifiable patient data to train LLMs can lead to disclosure of sensitive information and raises privacy, security, and regulatory concerns. It does not say that data pre-processing decisions vary drastically across research teams or identify that variation as the cause of these risks. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • There is a potential for misuse of patient data, medicolegal implications, and loss of physician-patient trust.

    “There is a potential for misuse of patient data, medicolegal implications, and loss of physician-patient trust.181,192”

    From Large Language Models for Chatbot Health Advice Studies: A Systematic Review (2025) · cites this paper as [181]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Ong, J. C. L., Chang, S. Y.-H., William, W., Butte, A. J., Shah, N. H., Chew, L. S. T., Liu, N., Doshi-Velez, F., Lu, W., Savulescu, J., & Ting, D. S. W.
Affiliations
  • National University of Singapore (SG)
  • University of California, San Francisco (US)
  • Stanford Health Care (US)
  • SingHealth (SG)
Corresponding author
Ting, Daniel Shu Wei
ORCID
Venue
The Lancet Digital Health
Publisher
Elsevier BV
Published
2024-06
Volume
6
Issue
6
Pages
e428-e432
ISSN
2589-7500 (ISSN-L)
DOI
10.1016/s2589-7500(24)00061-x
OpenAlex
W4395026179
Cited by
342
References
23
Citations by year
2026: 165 · 2025: 136 · 2024: 41
FWCI
31.35
Type
journal article
Language
en
Open access
Licence
Locations
  • The Lancet Digital Health (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
Funders
  • A*STAR
  • National University of Singapore (NUHSRO/2022/078/Startup/13, NUS Start-Up)
  • Duke-NUS Medical School
  • Wellcome Trust (WT203132/Z/16/Z)
  • National Medical Research Council
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Law, AI, and Intellectual Property
Keywords
large language models, medical ethics, AI regulation, data privacy, data provenance, intellectual property, user rights, responsible AI, medical practice, societal risks of AI
MeSH terms
Artificial Intelligence, Humans, Natural Language Processing, Intellectual Property

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review