Cited by 401Ethics and Social Impacts of AIMachine Learning in HealthcareArtificial Intelligence in Healthcare and Education

Can AI Help Reduce Disparities in General Medical and Mental Health Care?

Open access
Irene Y. Chen, Peter Szolovits B, Marzyeh Ghassemi·2019·AMA Journal of Ethics

Abstract

Background: As machine learning becomes increasingly common in health care applications, concerns have been raised about bias in these systems' data, algorithms, and recommendations.Simply put, as health care improves for some, it might not improve for all.Methods: Two case studies are examined using a machine learning algorithm on unstructured clinical and psychiatric notes to predict intensive care unit (ICU) mortality and 30-day psychiatric readmission with respect to race, gender, and insurance payer type as a proxy for socioeconomic status.Results: Clinical note topics and psychiatric note topics were heterogenous with respect to race, gender, and insurance payer type, which reflects known clinical findings.Differences in prediction accuracy and therefore machine bias are shown with respect to gender and insurance type for ICU mortality and with respect to insurance policy for psychiatric 30-day readmission.Conclusions: This analysis can provide a framework for assessing and identifying disparate impacts of artificial intelligence in health care. Bias in Machine Learning ModelsWhile health care is an inherently data-driven field, most clinicians operate with limited evidence guiding their decisions.Randomized trials estimate average treatment effects for a trial population, but participants in clinical trials often aren't representative of the patient population that ultimately receives the treatment with respect to race and gender.1,2 As a result, drugs and interventions are not tailored to historically mistreated groups; for example, women, minority groups, and obese patients tend to have generally poorer treatment options and longitudinal health outcomes.[3][4][5][6][7][8][9] Advances in artificial intelligence (AI) and machine learning offer the potential to provide personalized care by taking into account granular patient differences.Machine learning using images, clinical notes, and other electronic health record (EHR) data has been

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1001/amajethics.2019.167

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. AI versus doctors [news]. IEEE Spectr. 2017;54(10):13.

Cited by

Sorted by citations

Authors
Chen, I. Y., Peter Szolovits, B., & Ghassemi, M.
Affiliations
  • MIT Computer Science and Artificial Intelligence Laboratory (US)
  • University of Toronto (CA)
Venue
AMA Journal of Ethics
Published
2019-02-01
Volume
21
Issue
2
Pages
E167-179
ISSN
  • 1937-7010 (ISSN-L)
  • 2376-6980
OpenAlex
W2914514892
Cited by
401
References
53
Citations by year
2026: 58 · 2025: 76 · 2024: 66 · 2023: 55 · 2022: 59 · 2021: 42 · 2020: 39 · 2019: 5 · 2018: 1
FWCI
63.10
Type
journal article
Language
en
Open access
Locations
  • The AMA Journal of Ethic (publishedVersion)
  • PubMed (publishedVersion)
Funders
  • National Institute of Mental Health (R01MH106577)
  • Division of Graduate Education
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
algorithmic bias, health disparities, artificial intelligence in healthcare, machine learning prediction, ICU mortality prediction, psychiatric readmission, clinical notes, socioeconomic status, race and gender disparities, electronic health records, model prediction accuracy
MeSH terms
Adult, Aged, Aged, 80 and over, Artificial Intelligence, Delivery of Health Care, Female, Humans, Intensive Care Units, Male, Mental Health Services, Middle Aged, Mortality, Patient Readmission, Sex Factors, Healthcare Disparities

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review