CoChat

Ethical Considerations of Using ChatGPT in Health Care

Open access
Wang, Changyu, Liu, Siru, Yang, Hao, Guo, Jiulin, Wu, Yuxuan, Liu, Jialin · 2023 · Journal of Medical Internet Research

Published version of a preprint

The DOI matches its Crossref record; 82 of the 82 references it cites were found in a registry or an index; 19 of the 50 sources its claims cite were checked for support.

Abstract

ChatGPT has promising applications in health care, but potential ethical issues need to be addressed proactively to prevent harm. ChatGPT presents potential ethical challenges from legal, humanistic, algorithmic, and informational perspectives. Legal ethics concerns arise from the unclear allocation of responsibility when patient harm occurs and from potential breaches of patient privacy due to data collection. Clear rules and legal boundaries are needed to properly allocate liability and protect users. Humanistic ethics concerns arise from the potential disruption of the physician-patient relationship, humanistic care, and issues of integrity. Overreliance on artificial intelligence (AI) can undermine compassion and erode trust. Transparency and disclosure of AI-generated content are critical to maintaining integrity. Algorithmic ethics raise concerns about algorithmic bias, responsibility, transparency and explainability, as well as validation and evaluation. Information ethics include data bias, validity, and effectiveness. Biased training data can lead to biased output, and overreliance on ChatGPT can reduce patient adherence and encourage self-diagnosis. Ensuring the accuracy, reliability, and validity of ChatGPT-generated content requires rigorous validation and ongoing updates based on clinical practice. To navigate the evolving ethical landscape of AI, AI in health care must adhere to the strictest ethical standards. Through comprehensive ethical guidelines, health care professionals can ensure the responsible use of ChatGPT, promote accurate and reliable information exchange, protect patient privacy, and empower patients to make informed decisions about their health care.

Overview

CoChat Summary

TL;DR

This paper examines ethical challenges of using ChatGPT in health care across legal, humanistic, algorithmic, and information ethics, proposing a regulatory framework involving society/government, health care departments, and AI companies. It argues that ChatGPT’s benefits must be balanced against risks such as unclear liability, privacy breaches, bias, and inaccurate or unvalidated content.

Key findings

  • The authors organize ethical concerns into four domains—legal, humanistic, algorithmic, and information ethics—and propose a regulatory framework for 3 primary entities (society and government, health care departments, and AI companies) encompassing 13 points.
  • Legal ethics concerns include unclear allocation of responsibility when ChatGPT advice harms patients, potential privacy breaches from collection and storage of sensitive patient data, and the need for licensing and regulatory requirements; OpenAI disclaims responsibility for generated texts, leaving the burden on users.
  • Humanistic ethics concerns include potential disruption of the physician-patient relationship, humanistic care, and integrity; overreliance on AI can undermine compassion and trust, and transparency/disclosure of AI involvement is critical.
  • Algorithmic and information ethics concerns include algorithmic bias, responsibility, transparency and explainability, validation and evaluation, data bias, validity, and effectiveness; ChatGPT may provide logically coherent but incorrect responses, and errors have been identified in discharge summaries and radiology reports.

Methods

  • Design: conceptual/ethical commentary and framework development, not an empirical study.
  • Data or sample: Not stated in the paper; no human participants, dataset, or sample is described.
  • Main analysis: narrative synthesis of ethical issues across legal, humanistic, algorithmic, and information ethics, with a proposed regulatory framework for 3 entities and 13 points.

Limitations

  • The paper states no formal limitations section; it acknowledges that ethical review standards for AI systems are not established, that OpenAI has not disclosed ChatGPT’s training set or underlying LLM, and that manual validation of ChatGPT output would be time- and resource-intensive.
  • Evident from the text: the work is conceptual and does not report empirical data, systematic review methods, or tested interventions; its recommendations are not empirically validated.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/48009

Cites

82 of 82 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 The regulatory framework of artificial intelligence (AI).
Figure 1 The regulatory framework of artificial intelligence (AI).CC BY · PubMed Central

Verification

Claims the paper makes

17 Valid · 0 Not valid · 20 Not checked · 17 Own results and interpretations

24 more claims · 11 with their cited sources checked

Create a free account to see all 54 claims and to verify the 31 references still open.

See all claims — free account
  • Probably not a substantial claim. 22
  • It may state two separate claims as one. 12
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 82 references

Research works printed but not linked: 2

  1. Best practice guidelines on research integrity and publishing ethicsWiley Author Services2023-07-01https://authorservices.wiley.com/ethics-guidelines/index.html
  2. Williams, R, 2022, Data & Policy

Cited by

Metadata

Authors
Wang, C., Liu, S., Yang, H., Guo, J., Wu, Y., & Liu, J.
Affiliations
  • Sichuan University (CN)
  • Vanderbilt University Medical Center (US)
Corresponding author
Liu, Jialin
Venue
Journal of Medical Internet Research
Published
2023-08-11
Volume
25
Pages
e48009
ISSN
1438-8871 (ISSN-L)
OpenAlex
W4385242971
Cited by
541
References
72
Citations by year
2026: 99 · 2025: 215 · 2024: 197 · 2023: 30
FWCI
21.60
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics and bioethics in healthcare
Keywords
ChatGPT, large language models, artificial intelligence in healthcare, medical ethics, legal liability, patient privacy, physician-patient relationship, algorithmic bias, transparency and explainability, overreliance on AI, patient trust, self-diagnosis, patient adherence, training data bias
MeSH terms
Artificial Intelligence, Data Collection, Humans, Patient Compliance, Reproducibility of Results, Disclosure

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.