Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.

Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis

Open access
Larrazabal, Agostina J., Nieto, Nicolás, Peterson, Victoria, Milone, Diego H., Ferrante, Enzo · 2020 · Proceedings of the National Academy of Sciences

Abstract

Artificial intelligence (AI) systems for computer-aided diagnosis and image-based screening are being adopted worldwide by medical institutions. In such a context, generating fair and unbiased classifiers becomes of paramount importance. The research community of medical image computing is making great efforts in developing more accurate algorithms to assist medical doctors in the difficult task of disease diagnosis. However, little attention is paid to the way databases are collected and how this may influence the performance of AI systems. Our study sheds light on the importance of gender balance in medical imaging datasets used to train AI systems for computer-assisted diagnosis. We provide empirical evidence supported by a large-scale study, based on three deep neural network architectures and two well-known publicly available X-ray image datasets used to diagnose various thoracic diseases under different gender imbalance conditions. We found a consistent decrease in performance for underrepresented genders when a minimum balance is not fulfilled. This raises the alarm for national agencies in charge of regulating and approving computer-assisted diagnosis systems, which should include explicit gender balance and diversity recommendations. We also establish an open problem for the academic medical image computing community which needs to be addressed by novel algorithms endowed with robustness to gender imbalance.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1073/pnas.1919012117

Cites

36 of 36 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1. Experimental results for a DenseNet-121 (18) classifier trained with images from the NIH dataset (16, 19) for 14 thoracic diseases under different gender imbalance ratios. (A) The box plots aggregate the results for 20 folds, training with male-only (blue) and female-only (orange) patients. Both models are evaluated given male (Top) and female (Bottom) test folds. A consistent decrease in performance is observed when using male patients for training and female for testing (and vice versa). (B and C) AUC achieved for two exemplar diseases under a gradient of gender imbalance ratios, from 0% of female images in training data to 100%, with increments of 25%. In B, 1 and 2 show the results when testing on male patients, while, in C, 1 and 2 present the results when testing on female patients. Statistical significance according to Mann–Whitney U test is denoted by **** (P≤ 0.00001), *** (0.00001 <P≤ 0.0001), ** (0.0001 <P≤ 0.001), * (0.001 <P≤, 0.01), and not significant (ns) (P>0.01).
Fig. 1. Experimental results for a DenseNet-121 (18) classifier trained with images from the NIH dataset (16, 19) for 14 thoracic diseases under different gender imbalance ratios. (A) The box plots aggregate the results for 20 folds, training with male-only (blue) and female-only (orange) patients. Both models are evaluated given male (Top) and female (Bottom) test folds. A consistent decrease in performance is observed when using male patients for training and female for testing (and vice versa). (B and C) AUC achieved for two exemplar diseases under a gradient of gender imbalance ratios, from 0% of female images in training data to 100%, with increments of 25%. In B, 1 and 2 show the results when testing on male patients, while, in C, 1 and 2 present the results when testing on female patients. Statistical significance according to Mann–Whitney U test is denoted by **** (P≤ 0.00001), *** (0.00001 <P≤ 0.0001), ** (0.0001 <P≤ 0.001), * (0.001 <P≤, 0.01), and not significant (ns) (P>0.01).CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 36 references

Cited by

See all 898

Metadata

Authors
Larrazabal, A. J., Nieto, N., Peterson, V., Milone, D. H., & Ferrante, E.
Affiliations
Consejo Nacional de Investigaciones Científicas y Técnicas (AR)
Corresponding author
Ferrante, Enzo
Venue
Proceedings of the National Academy of Sciences
Published
2020-05-26
Volume
117
Issue
23
Pages
12592-12594
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
OpenAlex
W3030030520
Cited by
685
References
36
Citations by year
2026: 117 · 2025: 149 · 2024: 125 · 2023: 110 · 2022: 101 · 2021: 71 · 2020: 10
FWCI
21.48
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (submittedVersion, cc-by-nc-sa)
Funders
  • AXA Research Fund (2017-AXA-PDOC1-097)
  • MINCyT | Agencia Nacional de Promoción Científica y Tecnológica (PICT 2017-4596, PICT-2018-03907, PICT 2014-2627, PICT-2018-3384)
  • Universidad Nacional del Litoral (CAID-PIC-50220140100084L, 2016-082)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Sex and Gender in Healthcare
Keywords
gender imbalance, medical imaging datasets, algorithmic fairness, computer-aided diagnosis, X-ray images, thoracic disease diagnosis, deep neural network, underrepresented groups, dataset collection
MeSH terms
Deep Learning, Female, Humans, Male, Radiographic Image Interpretation, Computer-Assisted, Reference Standards, Sex Factors, Radiography, Thoracic, Bias, Datasets as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis. https://app.cochat.ai/papers/10.1073/pnas.1919012117
The paper
Larrazabal, A. J., Nieto, N., Peterson, V., Milone, D. H., & Ferrante, E. (2020). Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.1919012117

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.