CoChat
Sign Up
Cited by 167Ethics and Social Impacts of AIMachine Learning in HealthcareArtificial Intelligence in Healthcare and Education

The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.

An adversarial training framework for mitigating algorithmic biases in clinical machine learning

Open access
Yang, Jenny, Soltan, Andrew A. S., Eyre, David W., Yang, Yang, Clifton, David A.·2023·npj Digital Medicine
DOI 10.1038/s41746-023-00805-yPMC PMC10050816
Open this paper in CoChat — free accountOpen PDF

Abstract

Machine learning is becoming increasingly prominent in healthcare. Although its benefits are clear, growing attention is being given to how these tools may exacerbate existing biases and disparities. In this study, we introduce an adversarial training framework that is capable of mitigating biases that may have been acquired through data collection. We demonstrate this proposed framework on the real-world task of rapidly predicting COVID-19, and focus on mitigating site-specific (hospital) and demographic (ethnicity) biases. Using the statistical definition of equalized odds, we show that adversarial training improves outcome fairness, while still achieving clinically-effective screening performances (negative predictive values >0.98). We compare our method to previous benchmarks, and perform prospective and external validation across four independent hospital cohorts. Our method can be generalized to any outcomes, models, and definitions of fairness.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41746-023-00805-y

Cites

44 of 44 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 COVID-19 diagnosis performance results across different UK NHS Trusts. Panels compare performance of basic (standalone) and adversarial models (blue and red, respectively) during prospective validation (a OUH) and external validation (b PUH, c UHB, d BH). Adversarial models were trained to mitigate ethnicity biases. All models were optimized during training to achieve sensitivities of 0.9. Error bars show 95% confidence intervals. Numerical results are shown in Supplementary Table 3.
Fig. 1 COVID-19 diagnosis performance results across different UK NHS Trusts. Panels compare performance of basic (standalone) and adversarial models (blue and red, respectively) during prospective validation (a OUH) and external validation (b PUH, c UHB, d BH). Adversarial models were trained to mitigate ethnicity biases. All models were optimized during training to achieve sensitivities of 0.9. Error bars show 95% confidence intervals. Numerical results are shown in Supplementary Table 3.CC BY · PubMed Central
Fig. 2 t-Stochastic Neighbor Embedding (t-SNE) visualization of data. Figure shows t-SNE representation of the training dataset used in training, including all positive COVID-19 cases across the four NHS trusts (OUH, PUH, UHB, BH).
Fig. 2 t-Stochastic Neighbor Embedding (t-SNE) visualization of data. Figure shows t-SNE representation of the training dataset used in training, including all positive COVID-19 cases across the four NHS trusts (OUH, PUH, UHB, BH).CC BY · PubMed Central
Fig. 3 COVID-19 diagnosis performance results on dataset combining all four NHS Trusts. Panel shows performance of basic and adversarial models during validation. Adversarial models were trained to mitigate site-specific (hospital) biases. All models were optimized during training to achieve a sensitivity of 0.9. Error bars show 95% confidence intervals. Numerical results are shown in Supplementary Table 4.
Fig. 3 COVID-19 diagnosis performance results on dataset combining all four NHS Trusts. Panel shows performance of basic and adversarial models during validation. Adversarial models were trained to mitigate site-specific (hospital) biases. All models were optimized during training to achieve a sensitivity of 0.9. Error bars show 95% confidence intervals. Numerical results are shown in Supplementary Table 4.CC BY · PubMed Central
Fig. 4 Adversarial Training Framework and Loss Functions. Figure shows the adversarial training framework used for bias mitigation. The framework consists of a predictor network (blue) and an adversary network (orange). Loss functions are also displayed alongside the respective network.
Fig. 4 Adversarial Training Framework and Loss Functions. Figure shows the adversarial training framework used for bias mitigation. The framework consists of a predictor network (blue) and an adversary network (orange). Loss functions are also displayed alongside the respective network.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 8

  1. Hardt, M., Price, E. & Srebro, N. Equality of opportunity in supervised learning. Adv. Neural Inf. Process. Syst. 29, 3315–3323 (2016).
  2. Iosifidis, V. & Ntoutsi, E. Dealing with bias via data augmentation in supervised learning scenarios. Jo Bates Paul D. Clough Robert Jäschke 24, 11 (2018).
  3. Wang, H., Ustun, B., Calmon, F. P. & Harvard, S. E. A. S. Avoiding disparate impact with counterfactual distributions. In NeurIPS Workshop on Ethical, Social and Governance Issues in AI. (2018).
  4. Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J. & Weinberger, K. Q. On fairness and calibration. Adv. Neural Inf. Process. Syst. 30, 5680–5689 (2017).
  5. Menon, A. K. & Williamson, R. C. The cost of fairness in binary classification. In Conference on Fairness, Accountability and Transparency (pp. 107–118) PMLR. (2018).
  6. Jung, C. et al. An algorithmic framework for fairness elicitation. arXiv. https://arxiv.org/abs/1905.10660 (2019). · arXiv
  7. Kim, M., Reingold, O. & Rothblum, G. Fairness through computationally-bounded awareness. Adv. Neural Inf. Process. Syst. 31, 4842–4852 (2018).
  8. Blazer, D. G. & Hernandez, L. M. (Eds.). Genes, behavior, and the social environment: Moving beyond the nature/nurture debate. (2006).

Cited by

Sorted by citations

See all 167

Related

See all

Metadata

Authors
Yang, J., Soltan, A. A. S., Eyre, D. W., Yang, Y., & Clifton, D. A.
Affiliations
  • University of Oxford (GB)
  • John Radcliffe Hospital (GB)
  • Shanghai Jiao Tong University (CN)
Corresponding author
Yang, Jenny
ORCID
Venue
npj Digital Medicine
Publisher
Springer Science and Business Media LLC
Published
2023-03-29
Volume
6
Issue
1
Pages
55-55
ISSN
2398-6352 (ISSN-L)
DOI
10.1038/s41746-023-00805-y
PMC
PMC10050816
OpenAlex
W4361285150
Cited by
159
References
40
Citations by year
2026: 44 · 2025: 66 · 2024: 38 · 2023: 11
FWCI
39.78
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Wellcome Trust (0009350)
  • Robertson Foundation
  • University Hospitals Birmingham NHS Foundation Trust
  • National Institute for Health Research Health Protection Research Unit
  • National Institute for Health and Care Research (ACF-2020-13-015)
  • Royal Academy of Engineering
  • University of Oxford
  • European Commission (HORIZON2020, 955681)
  • Shanghai Jiao Tong University
  • Oxford University Hospitals NHS Foundation Trust
  • Medical Research Council (MC_PC_19049)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
adversarial training, algorithmic bias, machine learning in healthcare, fairness, equalized odds, COVID-19 prediction, ethnic prejudice, health disparities, external validation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review