Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
Research in machine learning fairness has historically considered a single binary demographic attribute; however, the reality is of course far more complicated. In this work, we grapple with questions that arise along three stages of the machine learning pipeline when incorporating intersectionality as multiple demographic attributes: (1) which demographic attributes to include as dataset labels, (2) how to handle the progressively smaller size of subgroups during model training, and (3) how to move beyond existing evaluation metrics when benchmarking model fairness for more subgroups. For each question, we provide thorough empirical evaluation on tabular datasets derived from the US Census, and present constructive recommendations for the machine learning community. First, we advocate for supplementing domain knowledge with empirical validation when choosing which demographic attribute labels to train on, while always evaluating on the full set of demographic attributes. Second, we warn against using data imbalance techniques without considering their normative implications and suggest an alternative using the structure in the data. Third, we introduce new evaluation metrics which are more appropriate for the intersectional setting. Overall, we provide substantive suggestions on three necessary (albeit not sufficient!) considerations when incorporating intersectionality into machine learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment.
Brewer, R. M., & Collins, P. H. (1992) · Contemporary Sociology · doi:10.2307/2074808 · Cited by 19,445
The Complexity of Intersectionality
McCall, L. (2005) · Signs: Journal of Women in Culture and Society · doi:10.1086/426800 · Cited by 6,682 · cited 2×
“1 There is a long history of considering intersectional harms in fields outside of computer science [9, 22–24, 52, 80, 90, 92, 98, 112, 115], and an urgent need to do so in machine learning 1 There are complex nuances to this conceptualization that are out of scope of this work [98].”
BackgroundResultDwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160 · cited 1×
“One field in which synthetic examples are being widely adopted, and where results are relatively accepted as indicative of real-world gains largely due to work on transfer learning (e.g., Sim2Real [66]), is reinforcement learning for robotics [82].”
BackgroundEquality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 3×
“However, for the scope of our work since we consider the positive labels to be more desirable, we choose a metric analogous to equal opportunity [58], i.e., equalizing the true positive rate (TPR).”
BackgroundResultFairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · cited 1×
“We also note that despite the language we employ, we do not suggest that fairness can be treated as a purely algorithmic problem that neglects the sociotechnical frame [12, 50, 51, 109].”
BackgroundBellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581
Gender: An Intersectionality Perspective
Shields, S. A. (2008) · Sex Roles · doi:10.1007/s11199-008-9501-8 · Cited by 1,509 · cited 1×
“1 There is a long history of considering intersectional harms in fields outside of computer science [9, 22–24, 52, 80, 90, 92, 98, 112, 115], and an urgent need to do so in machine learning 1 There are complex nuances to this conceptualization that are out of scope of this work [98].”
BackgroundStatistical Theories of Discrimination in Labor Markets
Aigner, D. J., & Cain, G. G. (1977) · ILR Review · doi:10.1177/001979397703000204 · Cited by 1,503
A Reductions Approach to Fair Classification
Agarwal, A., Beygelzimer, A., Dudík, M., Langford, J., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1803.02453 · Cited by 1,288
Cited by
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279
“…244], an intersectional approach draws attention to how harms play out for people whose lives are shaped by interlocking forms of oppression [46, 229] – for aim(s)tocapturethemor-phological sexual differences between male and female faces by comparing their shape differences to a defined face…”
Background"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters
Wan, Y., Pu, G., Sun, J., Garimella, A., Chang, K.-W., & Peng, N. (2023) · arXiv · doi:10.48550/arxiv.2310.09219 · Cited by 175
Palmer, Z. B., & Oswal, S. K. (2025) · Journal of Business and Technical Communication · doi:10.1177/10506519241280644 · Cited by 22
Data Agency Theory: A Precise Theory of Justice for AI Applications
Ajmani, L., Stapleton, L., Houtti, M., & Chancellor, S. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658930 · Cited by 11
“Recent work in AI has focused heavily on expanding our evaluation metrics beyond quantitative ones, such as accuracy, and towards human-centered concepts, such as bias, fairness, and equality [14, 87].”
BackgroundThe Ecological Fallacy in Annotation: Modeling Human Label Variation goes beyond Sociodemographics
Orlikowski, M., Röttger, P., Cimiano, P., & Hovy, D. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2023.acl-short.88 · Cited by 9
“In contrast, social scientists have long adopted the idea of intersectionality (Crenshaw, 1989), which also informs research on fairness in machine learning (Wang et al., 2022).”
BackgroundCo-Producing AI: Toward an Augmented, Participatory Lifecycle
Mushkani, R., Berard, H., Ammar, T., Chatonnier, C., & Koseki, S. (2025) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v8i2.36674 · Cited by 5
Metadata
- Authors
- Wang, A., Ramaswamy, V. V., & Russakovsky, O.
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 336-349
- OpenAlex
- W4283166422
- Cited by
- 87
- References
- 38
- FWCI
- 18.23
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.