CoChat

CheXclusion: Fairness gaps in deep chest X-ray classifiers

Open access

The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.

Abstract

Machine learning systems have received much attention recently for their ability to achieve expert-level performance on clinical tasks, particularly in medical imaging. Here, we examine the extent to which state-of-the-art deep learning classifiers trained to yield diagnostic labels from X-ray images are biased with respect to protected attributes. We train convolution neural networks to predict 14 diagnostic labels in 3 prominent public chest X-ray datasets: MIMIC-CXR, Chest-Xray8, CheXpert, as well as a multi-site aggregation of all those datasets. We evaluate the TPR disparity - the difference in true positive rates (TPR) - among different protected attributes such as patient sex, age, race, and insurance type as a proxy for socioeconomic status. We demonstrate that TPR disparities exist in the state-of-the-art classifiers in all datasets, for all clinical tasks, and all subgroups. A multi-source dataset corresponds to the smallest disparities, suggesting one way to reduce bias. We find that TPR disparities are not significantly correlated with a subgroup's proportional disease burden. As clinical models move from papers to products, we encourage clinical decision makers to carefully audit for algorithmic disparities prior to deployment. Our supplementary materials can be found at, http://www.marzyehghassemi.com/chexclusion-supp-3/.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1142/9789811232701_0022

Cites

55 of 55 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 55 references

Research works printed but not linked: 1

  1. [22]22. D. S. Char, N. H. Shah and D. Magnus, Implementing machine learning in health care addressing ethical challenges, New England Journal of Medicine 378 , 981 (2018).

Cited by

See all 275

Metadata

Authors
Seyyed-Kalantari, L., Liu, G., McDermott, M., Chen, I. Y., & Ghassemi, M.
Affiliations
  • University of Toronto (CA)
  • Massachusetts Institute of Technology (US)
Venue
Biocomputing 2021
Published
2020-11
Volume
26
Pages
232-243
OpenAlex
W3007679811
Cited by
275
References
47
Citations by year
2026: 21 · 2025: 58 · 2024: 45 · 2023: 66 · 2022: 45 · 2021: 29 · 2020: 11
FWCI
19.39
Type
conference paper
Language
en
Open access
Licence
cc-by-nc
Locations
  • Pacific Symposium on Biocomputing (PSB) (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
algorithmic fairness, algorithmic bias, chest X-ray classification, deep learning classifier, protected characteristics, socioeconomic status, insurance type, race, sex, algorithmic auditing, CheXpert, MIMIC-CXR, ChestX-ray14
MeSH terms
Machine Learning, Humans, X-Rays, Neural Networks, Computer, Computational Biology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: CheXclusion: Fairness gaps in deep chest X-ray classifiers. https://app.cochat.ai/papers/10.1142/9789811232701_0022
The paper
Seyyed-Kalantari, L., Liu, G., McDermott, M., Chen, I. Y., & Ghassemi, M. (2020). CheXclusion: Fairness gaps in deep chest X-ray classifiers. Biocomputing 2021. https://doi.org/10.1142/9789811232701_0022

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.