Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Research in machine learning fairness has historically considered a single binary demographic attribute; however, the reality is of course far more complicated. In this work, we grapple with questions that arise along three stages of the machine learning pipeline when incorporating intersectionality as multiple demographic attributes: (1) which demographic attributes to include as dataset labels, (2) how to handle the progressively smaller size of subgroups during model training, and (3) how to move beyond existing evaluation metrics when benchmarking model fairness for more subgroups. For each question, we provide thorough empirical evaluation on tabular datasets derived from the US Census, and present constructive recommendations for the machine learning community. First, we advocate for supplementing domain knowledge with empirical validation when choosing which demographic attribute labels to train on, while always evaluating on the full set of demographic attributes. Second, we warn against using data imbalance techniques without considering their normative implications and suggest an alternative using the structure in the data. Third, we introduce new evaluation metrics which are more appropriate for the intersectional setting. Overall, we provide substantive suggestions on three necessary (albeit not sufficient!) considerations when incorporating intersectionality into machine learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment.
Brewer, R. M., & Collins, P. H. (1992) · Contemporary Sociology · doi:10.2307/2074808 · Cited by 19,445
The Complexity of Intersectionality
McCall, L. (2005) · Signs: Journal of Women in Culture and Society · doi:10.1086/426800 · Cited by 6,682 · cited 2×
“1 There is a long history of considering intersectional harms in fields outside of computer science [9, 22–24, 52, 80, 90, 92, 98, 112, 115], and an urgent need to do so in machine learning 1 There are complex nuances to this conceptualization that are out of scope of this work [98].”
BackgroundResultDwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 3×
“However, for the scope of our work since we consider the positive labels to be more desirable, we choose a metric analogous to equal opportunity [58], i.e., equalizing the true positive rate (TPR).”
BackgroundResultFairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581
Gender: An Intersectionality Perspective
Shields, S. A. (2008) · Sex Roles · doi:10.1007/s11199-008-9501-8 · Cited by 1,509 · cited 1×
“1 There is a long history of considering intersectional harms in fields outside of computer science [9, 22–24, 52, 80, 90, 92, 98, 112, 115], and an urgent need to do so in machine learning 1 There are complex nuances to this conceptualization that are out of scope of this work [98].”
BackgroundStatistical Theories of Discrimination in Labor Markets
Aigner, D. J., & Cain, G. G. (1977) · ILR Review · doi:10.1177/001979397703000204 · Cited by 1,503
“The Combahee River Collective Statement” (1977)
Collective, C. R. (2017) · Available Means · doi:10.2307/j.ctt5hjqnj.50 · Cited by 1,020
Cited by
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279
Palmer, Z. B., & Oswal, S. K. (2025) · Journal of Business and Technical Communication · doi:10.1177/10506519241280644 · Cited by 22
Data Agency Theory: A Precise Theory of Justice for AI Applications
Ajmani, L., Stapleton, L., Houtti, M., & Chancellor, S. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658930 · Cited by 11
“Recent work in AI has focused heavily on expanding our evaluation metrics beyond quantitative ones, such as accuracy, and towards human-centered concepts, such as bias, fairness, and equality [14, 87].”
BackgroundThe Ecological Fallacy in Annotation: Modeling Human Label Variation goes beyond Sociodemographics
Orlikowski, M., Röttger, P., Cimiano, P., & Hovy, D. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2023.acl-short.88 · Cited by 9
Co-Producing AI: Toward an Augmented, Participatory Lifecycle
Mushkani, R., Berard, H., Ammar, T., Chatonnier, C., & Koseki, S. (2025) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v8i2.36674 · Cited by 5
Metadata
- Authors
- Wang, A., Ramaswamy, V. V., & Russakovsky, O.
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 336-349
- OpenAlex
- W4283166422
- Cited by
- 87
- References
- 35
- FWCI
- 18.23
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation. https://app.cochat.ai/papers/10.1145/3531146.3533101
- The paper
- Wang, A., Ramaswamy, V. V., & Russakovsky, O. (2022). Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation. 2022 ACM Conference on Fairness Accountability and Transparency. https://doi.org/10.1145/3531146.3533101
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.