An Intersectional Definition of Fairness
Abstract
We propose differential fairness, a multi-attribute definition of fairness in machine learning which is informed by intersectionality, a critical lens arising from the humanities literature, leveraging connections between differential privacy and legal notions of fairness. We show that our criterion behaves sensibly for any subset of the set of protected attributes, and we prove economic, privacy, and generalization guarantees. We provide a learning algorithm which respects our differential fairness criterion. Experiments on the COMPAS criminal recidivism dataset and census data demonstrate the utility of our methods.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment.
Brewer, R. M., & Collins, P. H. (1992) · Contemporary Sociology · doi:10.2307/2074808 · Cited by 19,445 · see citing works
Crenshaw, K. (2018) · Feminist Legal Theory · doi:10.4324/9780429500480-5 · Cited by 9,510 · see citing works
Calibrating Noise to Sensitivity in Private Data Analysis
Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11681878_14 · Cited by 7,272 · see citing works
Crenshaw, K. W. (1998) · Oxford University Press eBooks · doi:10.1093/oso/9780198782063.003.0016 · Cited by 5,707 · see citing works
(2018) · Cited by 3,510 · see citing works
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · cited 1× · see citing works
“The AI community has responded by developing a broad array of mathematical formulations of fairness and learning algorithms which aim to satisfy them [3], [10], [13], [20], [24].”
BackgroundBarocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · see citing works
Algorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709 · see citing works
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · see citing works
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 1× · see citing works
“A number of studies have subsequently demonstrated that bias and fairness issues in AI are both harmful and pervasive [1], [4], [5].”
BackgroundHill Collins, P. (2002) · Routledge · doi:10.4324/9780203900055 · Cited by 1,532 · see citing works
Ain't I a Woman: Black Women and Feminism, by Bell Hooks
Perkins, L. M. (1983) · Political Science Quarterly · doi:10.2307/2150228 · Cited by 1,492 · see citing works
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1323 · Cited by 829 · see citing works
Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2018) · arXiv · doi:10.48550/arxiv.1811.07867 · Cited by 687 · cited 1× · see citing works
“The necessity has now become clear for interdisciplinary analyses of fairness in AI and its relationship to society, to civil rights, and to the social goals which are to be achieved by mathematical fairness definitions, which have not always been made explicit [18].”
BackgroundAge, Race, Class, and Sex: Women Redefining Difference
Lorde, A. (2021) · Campus Wars · doi:10.4324/9780429038556-22 · Cited by 408 · see citing works
Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
Kearns, M., Neel, S., Roth, A., & Wu, Z. S. (2017) · arXiv · doi:10.48550/arxiv.1711.05144 · Cited by 381 · see citing works
Kifer, D., & Machanavajjhala, A. (2014) · ACM Transactions on Database Systems · doi:10.1145/2514689 · Cited by 270 · see citing works
River, T. (2014) · Women's studies quarterly · doi:10.1353/wsq.2014.0052 · Cited by 211 · see citing works
A Convex Framework for Fair Regression
Berk, R., Heidari, H., Jabbari, S., Joseph, M., Kearns, M., Morgenstern, J., Neel, S., & Roth, A. (2017) · arXiv · doi:10.48550/arxiv.1706.02409 · Cited by 198 · cited 1× · see citing works
“The AI community has responded by developing a broad array of mathematical formulations of fairness and learning algorithms which aim to satisfy them [3], [10], [13], [20], [24].”
BackgroundThe problem of infra-marginality in outcome tests for discrimination
Simoiu, C., Corbett-Davies, S., & Goel, S. (2017) · The Annals of Applied Statistics · doi:10.1214/17-aoas1058 · Cited by 154 · cited 5× · see citing works
“ted with race [21]. On the other hand, attributes which are important for the algorithm may have different distributions depending on the protected attributes; a phenomenon known as infra-marginality [32]. Consequently, equity may not always correspond to fairness, as defined by [34]. It is also important to connect fa…”
BackgroundInfluentialInterventional Fairness: Causal Database Repair for Algorithmic Fairness
Salimi, B., Rodriguez, L., Howe, B., & Suciu, D. (2019) · Proceedings of the 2019 International Conference on Management of Data · doi:10.1145/3299869.3319901 · Cited by 145 · cited 1× · see citing works
“The AI community has responded by developing a broad array of mathematical formulations of fairness and learning algorithms which aim to satisfy them [3], [10], [13], [20], [24].”
BackgroundCristobal, N., & García, G. A. (2018) · NASPA Journal · doi:10.1080/19496591.2018.1470007 · Cited by 78 · cited 1× · see citing works
“ces, while acknowledged, are frequently due to systemic structural disadvantages such as racism, sexism, inter-generational poverty, 2 the school-to-prison pipeline, and the prison-industrial complex [11, 13, 20, 36, 14, 37]. Systems of oppression can lead individuals to perform below their potential, for instance by…”
Background
Flagged first, then by citations
10 of the 32 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- 1978, Guidelines on employee selection procedures, C F R, 29, 1607
- campolo, 2017, AI Now 2017 Symposium Report
- angwin, 2016, Machine bias: There’s software used across the country to predict future criminals. and it’s biased against blacks, ProPublica
- truth, 0, Ain’t I a woman? 1851
Cited by
Sorted by citations
Generative Adversarial Networks for face generation: A survey
Kammoun, A., Slama, R., Tabia, H., Ouni, T., & Abid, M. (2022) · ACM Computing Surveys · doi:10.1145/1122445.1122456 · Cited by 326 · see citing works
50 Years of Test (Un)fairness: Lessons for Machine Learning
Hutchinson, B., & Mitchell, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287600 · Cited by 297 · see citing works
“ality can be modeled. This might include, for example, how to quantify fairness along one dimension (e.g., age) conditioned on another dimension (e.g., skin tone), as recent work has begun to address [27, 39]. 6 CONCLUSIONS The spike in interest in test fairness in the 1960s arose during a time of social and political…”
BackgroundRethinking Fairness: An Interdisciplinary Survey of Critiques of Hegemonic ML Fairness Approaches
Weinberg, L. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13196 · Cited by 127 · see citing works
Rigotti, C., & Fosch‐Villaronga, E. (2024) · Computer law & security review · doi:10.1016/j.clsr.2024.105966 · Cited by 164 · see citing works
Algorithmic inclusion: Shaping the predictive algorithms of artificial intelligence in hiring
Kelan, E. K. (2024) · Human Resource Management Journal · doi:10.1111/1748-8583.12511 · Cited by 106 · see citing works
Koshiyama, A., Kazim, E., Treleaven, P., Rai, P., Szpruch, L., Pavey, G., Ahamat, G., Leutner, F., Goebel, R., Knight, A., et al. (2024) · Royal Society Open Science · doi:10.1098/rsos.230859 · Cited by 110 · see citing works
“…a component of robustness, may need to be traded for lowering any existing outcome metric of bias [14], making the model more explainable may affect a system’s performance and privacy [15,16], improving privacy may affect ways to assess adverse impacts of algorithmic systems [17] and so on.”
In-Processing Modeling Techniques for Machine Learning Fairness: A Survey
Wan, M., Zha, D., Liu, N., & Zou, N. (2023) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3551390 · Cited by 99 · see citing works
“— Differential fairness [34]: A model fθ is ε-differentially fair if e −ε ≤ log p (ŷ=0 |si ) p (ŷ=0 |sj ) ≤ e ε and e−ε ≤ log p (ŷ=1 |si ) p (ŷ=1 |sj ) ≤ e ε for all si and sj .”
BackgroundWang, A., Ramaswamy, V., & Russakovsky, O. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533101 · Cited by 90 · see citing works
“To generalize the equal opportunity measure to more than two groups, we adopt a method similar to prior work [40, 77, 121].”
BackgroundMethodInfluentialDebiasing Career Recommendations with Neural Fair Collaborative Filtering
Islam, R., Keya, K. N., Zeng, Z., Pan, S., & Foulds, J. R. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449904 · Cited by 81 · see citing works
“Following [22], our learning algorithm for fine-tuning uses the fairness cost as a regularizer to balance the trade-off between fairness and accuracy.”
InfluentialSelf-citationAre “Intersectionally Fair” AI Algorithms Really Fair to Women of Color? A Philosophical Analysis
Kong, Y. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533114 · Cited by 58 · see citing works
“Several studies suggested technical solutions to fairness gerrymandering [27, 32, 34, 59], and a growing number of articles directly employ the concept of intersectionality for analyzing unfairness and improving fairness in AI [23, 31, 40, 50, 56, 57].”
BackgroundInfluentialImproving Fairness in Deepfake Detection
(2024) · doi:10.1109/wacv57701.2024.00459 · Cited by 64 · see citing works
“To sum up, there are two types of fairness measures using demographics: group fairness [12] and intersectional fairness [14].”
BackgroundHow fair can we go in machine learning ? Assessing the boundaries of accuracy and fairness
Valdivia, A., Sánchez‐Monedero, J., & Casillas, J. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22354 · Cited by 58 · see citing works
“Differential fairness [28] is a growing concept highly related with this work, which addresses intersectionality.”
BackgroundThe Causal Fairness Field Guide: Perspectives From Social and Formal Sciences
Carey, A. N., & Wu, X. (2022) · AI and Ethics · doi:10.3389/fdata.2022.892837 · Cited by 90 · see citing works
“An avenue of fair machine learning research that would be a good start for solving this issue is the work on intersectional fairness [81, 82].”
BackgroundLooking for Trouble: Analyzing Classifier Behavior via Pattern Divergence
Pastor, E., de Alfaro, L., & Baralis, E. (2021) · SIGMOD Conference · doi:10.1145/3448016.3457284 · Cited by 68 · see citing works
“Several works [10, 11, 16, 19] consider fairness for an intersection of multiple sensitive attributes, known as inter-sectional fairness.”
BackgroundExplanation-guided fairness testing through genetic algorithm
Fan, M. Y., Wei, W., Jin, W., Yang, Z., & Liu, T. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510137 · Cited by 51 · see citing works
Is explainable artificial intelligence intrinsically valuable?
Colaner, N. R. (2021) · AI & Society · doi:10.1007/s00146-021-01184-2 · Cited by 43 · see citing works
“2020; Foulds et. al.”
Bogen, M., Rieke, A., & Ahmed, S. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372877 · Cited by 49 · see citing works
“[17, 32, 39, 40, 48, 50, 65]. When labeled data are not available, researchers have made powerful discoveries by augmenting existing data through the inference or construction of labeled data de novo [5, 10, 27, 46]. At times, this work has insufficiently acknowledged the full range of challenges to generating or obta…”
BackgroundFairness Improvement with Multiple Protected Attributes: How Far Are We?
Chen, Z., Zhang, J. M., Sarro, F., & Harman, M. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639083 · Cited by 46 · see citing works
“To consider multiple protected attributes and their intersectionality, researchers divide a population into subgroups based on the combination of different protected attributes [32, 33, 65].”
BackgroundImplementing equitable and intersectionality‐aware ML in education: A practical guide
Mangal, M., & Pardos, Z. A. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13484 · Cited by 52 · see citing works
Explainable and Fair AI: Balancing Performance in Financial and Real Estate Machine Learning Models
Acharya, D. B., Divya, B., & Kuppan, K. (2024) · IEEE Access · doi:10.1109/access.2024.3484409 · Cited by 41 · see citing works
“Reference [14] extensively reviewed existing and proposed bias mitigation strategies in the machine learning domain, articulating ways to achieve fairness [15] in various contexts.”
BackgroundA Survey on Intersectional Fairness in Machine Learning: Notions, Mitigation, and Challenges
Gohar, U., & Cheng, L. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/742 · Cited by 40 · see citing works
“Finally, [ Foulds et al. , 2020 ] also proposes DF-bias amplification , which measures the discrimination of a predictor by taking the difference of the DF of the dataset ( (cid:15) 1 ) and the predictor ( (cid:15) 2 ).”
BackgroundLogothetis, S. B., Green, D., Holland, M., & Moubayed, N. A. (2023) · Scientific Reports · doi:10.1038/s41598-023-40661-0 · Cited by 37 · see citing works
“Supplemental Observations & Phenotype Age (years) 69 (50-82) 69 (50-82) 68 (50-81) 64 (44-79)”
Zimmermann, A., & Lee‐Stronach, C. (2021) · Canadian Journal of Philosophy · doi:10.1017/can.2021.17 · Cited by 42 · see citing works
What-is and How-to for Fairness in Machine Learning: A Survey, Reflection, and Perspective
Tang, Z., Zhang, J., & Zhang, K. (2023) · ACM Computing Surveys · doi:10.1145/3597199 · Cited by 33 · see citing works
“There might be additional technical considerations for certain fairness notions to be able to apply in different practical scenarios, for instance, the phenomenon of fairness gerrymandering [99, 100] and the quantification of differential fairness [65] when considering subgroups formed by structured combinations of pr…”
BackgroundFarayola, M. M., Tal, I., Connolly, R., Saber, T., & Bendechache, M. (2023) · Information · doi:10.3390/info14080426 · Cited by 30 · see citing works
MithraCoverage: A System for Investigating Population Bias for Intersectional Fairness
Jin, Z., Xu, M., Sun, C., et al. (2020) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/3318464.3384689 · Cited by 29 · see citing works
“This is known as intersectional (aka. subgroup) (un)fairness [6–8, We must ensure that there are enough entries in the dataset for each subgroup, defined as the intersection of multiple attributes, to prevent population bias and its consequences.”
BackgroundThrough the Data Management Lens: Experimental Analysis and Evaluation of Fair Classification
Islam, M. T., Fariha, A., Meliou, A., & Salimi, B. (2022) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3514221.3517841 · Cited by 30 · see citing works
“intersectional fairness [29] differential fairness ✓ ✓ ✓”
When Fair Ranking Meets Uncertain Inference
Ghosh, A., Dutt, R., & Wilson, C. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462850 · Cited by 33 · see citing works
Can We Obtain Fairness For Free?
Islam, R., Pan, S., & Foulds, J. R. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462614 · Cited by 28 · see citing works
“In [19], the fairness/accuracy trade-off hyper-parameter is selected for fair models based on the validation set to minimize unfairness, such that accuracy is allowed to be degraded by at most 5% from the typical model.”
BackgroundMethodInfluentialSelf-citationEquity and Access in Algorithms, Mechanisms, and Optimization
(2021) · doi:10.1145/3465416 · Cited by 28 · see citing works
Differential Fairness: An Intersectional Framework for Fair AI
Islam, R., Keya, K. N., Pan, S., Sarwate, A. D., & Foulds, J. R. (2023) · Entropy · doi:10.3390/e25040660 · Cited by 27 · see citing works
“Countering Structural Oppression: Recently, Kong [30] critiqued fairness definitions based on multiple tuples of attributes [17,18,31] regarding the extent to which they implement intersectionality.”
BackgroundSelf-citationMhasawade, V., & Chunara, R. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462587 · Cited by 27 · see citing works
“In illustrating the relationship between AP and AI (green arrow) we make distinction between the multi-level fairness setting and literature on intersectionality in fairness, which has considered multiple sensitive attributes that are independent of each other (Figure 1b) [16, 37, 53, 54].”
BackgroundKlumbytė, G., Draude, C., & Taylor, A. S. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533207 · Cited by 28 · see citing works
“Calls for intersectional and feminist approaches to ML/AI research and systems design exemplify this need for inter- and trans-disciplinary openness and collaboration [24, 32, 36, 66, 67].”
BackgroundDo Humans Prefer Debiased AI Algorithms? A Case Study in Career Recommendation
Wang, C., Wang, K., Bian, A., Islam, R., Keya, K. N., Foulds, J. R., & Pan, S. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511108 · Cited by 28 · see citing works
“…outcome should be equal for both protected and unprotected group members”) [35] and Differential Fairness (“the probabilities of the outcomes will be similar with respect to different subgroups defined by the intersections of multiple protected attributes such as race, gender and race”) [28].”
BackgroundSelf-citationIntersectional analysis for science and technology
Nielsen, M. W., Gissi, E., Heidari, S., Horton, R. C., Nadeau, K. C., Ngila, D., Noble, S., Paik, H. Y., Tadesse, G. A., Zeng, E. Y., et al. (2025) · Nature · doi:10.1038/s41586-025-08774-w · Cited by 29 · see citing works
“We note that such metrics rely on representational models and data as well as competing theories of what constitutes fairness, which have not been settled 156 .”
Equitable Allocation of Healthcare Resources with Fair Survival Models
Keya, K. N., Islam, R., Pan, S., Stockwell, I., & Foulds, J. R. (2021) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611976700.22 · Cited by 25 · see citing works
Self-citationGardner, J., Yu, R., Nguyen, Q., Brooks, C. A., & Kizilcec, R. F. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594107 · Cited by 29 · see citing works
“[31] James Foulds, Rashidul Islam, Kamrun Naher Keya, and Shimei Pan.”
BackgroundChizari, N., Shoeibi, N., & Garcı́a, M. N. M. (2022) · Electronics · doi:10.3390/electronics11203301 · Cited by 21 · see citing works
“DF specifically focuses on both intersectionality [81,82], which involves fairness for each of the protected attributes individually, and behavior toward minorities, which is related to the previously mentioned anti-discrimination laws.”
Fairness seen as global sensitivity analysis
Bénesse, C., Gamboa, F., Loubes, J.-M., & Boissin, T. (2024) · Machine Learning · doi:10.1007/s10994-022-06202-y · Cited by 19 · see citing works
InfoFair: Information-Theoretic Intersectional Fairness
Kang, J. L., Xie, T., Wu, X., Maciejewski, R., & Tong, H. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020588 · Cited by 31 · see citing works
“Foulds et al. [31] propose (cid:15) -differential fairness, which ensures pairwise equal acceptance rate.”
BackgroundMulti-dimensional Discrimination in Law and Machine Learning - A Comparative Overview
Roy, A. G., Horstmann, J., & Ntoutsi, E. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593979 · Cited by 26 · see citing works
“Unlike previous works[23, 44], they defined discrimination over the entire collection of subgroups SG as a min-max ratio of prediction probability over any subgroup.”
BackgroundInfluentialFairness amidst non‐IID graph data: A literature review
Zhang, W., Zhou, S., Walsh, T., & Weiss, J. C. (2025) · AI Magazine · doi:10.1002/aaai.12212 · Cited by 29 · see citing works
“For example, the idea of preserving private information in privacy can help with mitigate bias so that the sensitive information is non-inferrable [Foulds et al. , 2020] leading to fair graph learning.”
Intersectional Two-sided Fairness in Recommendation
Wang, Y., Sun, P., Ma, W., et al. (2024) · ACM Web Conference (WWW) · doi:10.1145/3589334.3645518 · Cited by 19 · see citing works
“There have been several studies [9, 10, 18–20, 36, 43, 53] on inter-sectional fairness in machine learning (ML), which focuses on the intersectional groups of different attributes, such as race & gender (e.g., black females).”
Optimizing Equity: Working towards Fair Machine Learning Algorithms in Laboratory Medicine
Azimi, V., & Zaydman, M. A. (2023) · The Journal of Applied Laboratory Medicine · doi:10.1093/jalm/jfac085 · Cited by 20 · see citing works
MinMax fairness: from Rawlsian Theory of Justice to solution for algorithmic bias
Barsotti, F., & Koçer, R. G. (2022) · AI & Society · doi:10.1007/s00146-022-01577-x · Cited by 18 · see citing works
“Moreover, due to intersectionality issues (Ghosh et al. 2021; Foulds et al. 2020), it might be that ensuring fairness on the basis of a specific attribute (e.”
Farayola, M. M., Tal, I., Bendechache, M., Saber, T., & Connolly, R. (2023) · International Conference on Availability, Reliability and Security (ARES) · doi:10.1145/3600160.3605033 · Cited by 18 · see citing works
Solar, G. V. (2022) · Applied Sciences · doi:10.3390/app12125813 · Cited by 18 · see citing works
“A thorough understanding of the way the workforce accesses the AI and DS opportunities in industry [29] and academia [30,31] is essential for building fair and inclusive societies [32].”
Measuring User Experience Inclusivity in Human-AI Interaction via Five User Problem-Solving Styles
Anderson, A. W., Guevara, J. N., Moussaoui, F., Li, T., Vorvoreanu, M., & Burnett, M. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3663740 · Cited by 17 · see citing works
“Armed with this new information, HAI practitioners could gain actionable insights in use-cases like the following: HAI Practice Use-Case 1 : To see which problem-solving groups of users are being left behind on an AI product with a problematic HAI-UX, measure equity state .”
BackgroundCausal Intersectionality and Fair RankingPreprint
Yang, K., Loftus, J. R., & Stoyanovich, J. (2020) · arXiv (Cornell University) · doi:10.4230/lipics.forc.2021.7 · Cited by 50 · see citing works
“Other recent technical work in this area focuses on achieving guarantees across intersectional subgroups [20, 24, 27], including on computer vision tasks [7], or makes connections to privacy [18].”
BackgroundMANI-Rank: Multiple Attribute and Intersectional Group Fairness for Consensus Ranking
Cachel, K., Rundensteiner, E. A., & Harrison, L. (2022) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00089 · Cited by 14 · see citing works
Fairness Implications of Encoding Protected Categorical Attributes
Mougan, C., Álvarez, J. M., Ruggieri, S., & Staab, S. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604657 · Cited by 14 · see citing works
“, [1, 24, 72, 76, 77]), most works assume the single binary protected attribute or disregard intersectionality entirely when handling multiple protected attributes [72], which is unrealistic and reductive.”
BackgroundA Brief Tour of Deep Learning from a Statistical Perspective
Nalisnick, E., Smyth, P., & Tran, D. (2023) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-032921-013738 · Cited by 16 · see citing works
Yi, F., Singh, A., & Tao, Z. (2024) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000101 · Cited by 12 · see citing works
Fair and Interpretable Models for Survival Analysis
Rahman, M. M., & Purushotham, S. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539259 · Cited by 13 · see citing works
“[20], where they used existing fairness definitions [11, 12] as fairness constraints in training Cox-based survival models, CPH [9], and DeepSurv [19].”
BackgroundMethodSuk, Y., & Han, K. T. (2023) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986231171711 · Cited by 11 · see citing works
“First, we did not consider intersectionality, that is, systematic disadvantages along intersecting dimensions , which contain not only gender, but also race, ethnicity, or disability status (Foulds et al., 2020).”
BackgroundThe Use of Artificial Intelligence in Residency Application Evaluation—A Scoping Review
Sumner, M. D., Howell, T. C., Soto, A. L., Kaplan, S. J., Tracy, E. T., Zaas, A. K., Migaly, J., Kirk, A. D., & Shah, K. N. (2025) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-24-00604.1 · Cited by 13 · see citing works
Fairness Beyond the Algorithmic Frame: Actionable Recommendations for an Intersectional Approach
Vethman, S., Smit, Q. T. S., van Liebergen, N. M., & Veenman, C. J. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732210 · Cited by 11 · see citing works
“Many technical oriented papers cite this study in relation to their intersectional bias measurement and consequently continue to pro-pose a bias mitigation techniques aimed at addressing intersection-ality within the algorithmic frame by minimizing unequal errors [44, 71].”
Artificial Intelligence and Human Rights
(2023) · Oxford University Press eBooks · doi:10.1093/law/9780192882486.001.0001 · Cited by 10 · see citing works
Amiri, S., Belloum, A. S. Z., Nalisnick, E., Klous, S., & Gommans, L. (2022) · International Conference on Dependable Systems and Networks workshops · doi:10.1109/dsn-w54100.2022.00018 · Cited by 13 · see citing works
Bias Assessment Approaches for Addressing User-Centered Fairness in GNN-Based Recommender Systems
Chizari, N., Tajfar, K., & Garcı́a, M. N. M. (2023) · Information · doi:10.3390/info14020131 · Cited by 10 · see citing works
When Biased Humans Meet Debiased AI: A Case Study in College Major Recommendation
Wang, C., Wang, K., Bian, A., Islam, R., Keya, K. N., Foulds, J. R., & Pan, S. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3611313 · Cited by 14 · see citing works
“The mission of the field has primarily been to develop (a) new metrics that define and quantify fairness [Dwork et al. 2012; Foulds et al. 2020a; Kusner et al. 2017], and (b) new machine learning techniques that remove bias [Dev and Phillips 2019; Foulds et al. 2020b; Zhang et al. 2018].”
BackgroundSelf-citationUnmasking Gender Bias in Recommendation Systems and Enhancing Category-Aware Fairness
Kheya, T. A., Bouadjenek, M. R., & Aryal, S. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714528 · Cited by 10 · see citing works
An intersectional framework for counterfactual fairness in risk prediction
Wastvedt, S., Huling, J. D., & Wolfson, J. A. (2023) · Biostatistics · doi:10.1093/biostatistics/kxad021 · Cited by 14 · see citing works
“Work such as [16], which outlines the core tenets of an intersectional approach to fairness measurement, may help bridge the gap between technical methods and the wider fairness picture.”
BackgroundCharacterizing Intersectional Group Fairness with Worst-Case ComparisonsPreprint
Ghosh, A., Genuit, L., & Reagan, M. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.01673 · Cited by 27 · see citing works
“Group Benefit Equality, introduced by Gartner (2020) aims to be useful in the domain of healthcare.”
BackgroundModeling Techniques for Machine Learning Fairness: A SurveyPreprint
Wan, M., Zha, D., Liu, N., & Zou, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.03015 · Cited by 10 · see citing works
“This problem is often caused by the algorithm and cannot be solely attributed to the data [33, 78].”
BackgroundRevisiting Group Fairness Metrics: The Effect of Networks
Mehrotra, A., Sachs, J., & Celis, L. E. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555100 · Cited by 8 · see citing works
“Intersectionality considers how individuals situated within intersections of more than one protected groups can face multiple and more extreme biases [37, 47].”
BackgroundNeural Embedding Allocation: Distributed Representations of Topic Models
Keya, K. N., Papanikolaou, Y., & Foulds, J. R. (2022) · Computational Linguistics · doi:10.1162/coli_a_00457 · Cited by 8 · see citing works
“To evaluate the classifier models in terms of fairness regarding the category label, we additionally considered several fairness metrics including differential fairness -DF (Foulds et al. 2020), which handles multi-class classification; demographic parity δ-DP (Dwork et al.”
MethodSelf-citationFairness in Forecasting and Learning Linear Dynamical SystemsPreprint
Zhou, Q. Z., Mareček, J., & Shorten, R. N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07315 · Cited by 5 · see citing works
“High accuracy need not be the primary goal of the system, especially when we consider that “accuracy” is measured on unfair data (Foulds et al. 2020).”
BackgroundBiasTrap: Runtime Detection of Biased Prediction in Machine Learning Systems
Mamman, H., Basri, S. B., Balogun, A. O., Imam, A. A., Kumar, G., & Capretz, L. F. (2024) · Journal of Advanced Research in Applied Sciences and Engineering Technology · doi:10.37934/araset.40.2.127139 · Cited by 7 · see citing works
Fairness-Aware Data Preparation for Entity Matching
Shahbazi, N., Wang, J., Miao, Z., & Bhutani, N. (2024) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00268 · Cited by 6 · see citing works
Trust, Privacy and Security Aspects of Bias and Fairness in Machine Learning
Atabek, A., Eralp, E., & Gürsoy, M. E. (2023) · IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) · doi:10.1109/tps-isa58951.2023.00023 · Cited by 6 · see citing works
Artificial Intelligence and Fair Trial Rights
Molbæk-Steensig, H., & Quemy, A. (2023) · Oxford University Press eBooks · doi:10.1093/law/9780192882486.003.0018 · Cited by 6 · see citing works
Fairness without Demographics through Shared Latent Space-Based Debiasing
Islam, R., Chen, H., & Cai, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i11.29167 · Cited by 6 · see citing works
Self-citationArtificial Intelligence and Freedom of Expression
Gregorio, G. D., & Dunn, P. (2023) · Oxford University Press eBooks · doi:10.1093/law/9780192882486.003.0005 · Cited by 6 · see citing works
mdfa: Multi-Differential Fairness Auditor for Black Box Classifiers
Gitiaux, X., & Rangwala, H. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/814 · Cited by 6 · see citing works
“Reinterpretations of fairness as a privacy problem can be found in [Jagielski et al., 2018; Foulds and Pan, 2018], but none of those contributions make a connection to disparate treatment.”
BackgroundA structured regression approach for evaluating model performance across intersectional subgroups
Herlihy, C. P., Truong, K., Chouldechova, A., & Dudı́k, M. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658908 · Cited by 5 · see citing works
Intersectionality and Testimonial Injustice in Medical Records
Andrews, K. S., Shah, B., & Cheng, L. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.39 · Cited by 7 · see citing works
“For intersectionality, we first look at ε-Differential fairness (Foulds et al., 2020), which requires that the difference between groups, regardless of their combination of sensitive attributes, not be treated differently within a range.”
BackgroundBeiting‐Parrish, M., & Whitmer, J. (2023) · Chinese/English Journal of Educational Measurement and Evaluation · doi:10.59863/nkcj9608 · Cited by 6 · see citing works
Data Engineering: An Overview from a Future Perspective
Jain, A., Sumit, S., Monga, C., et al. (2023) · International Conference on Contemporary Computing and Informatics (IC3I) · doi:10.1109/ic3i59117.2023.10398128 · Cited by 5 · see citing works
“As a result, big data analytics now involve issues with information security [32].”
BackgroundArtificial Intelligence and Human Rights at Work
Atkinson, J., & Collins, P. (2023) · Oxford University Press eBooks · doi:10.1093/law/9780192882486.003.0025 · Cited by 6 · see citing works
From Bias to Balance: Navigating Gender Inclusion in AI
Monko, G. J., & Mjahidi, M. M. (2024) · Artificial intelligence · doi:10.5772/intechopen.1007449 · Cited by 5 · see citing works
Fair and Interpretable Algorithmic Hiring using Evolutionary Many Objective Optimization
Geden, M., & Andrews, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i17.17737 · Cited by 6 · see citing works
“We addressed this by using the smoothed empirical differential fairness objective (Foulds et al. 2019; 2020).”
MethodDisaggregated Interventions to Reduce Inequality
Bynum, L., Loftus, J. R., & Stoyanovich, J. (2021) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3465416.3483286 · Cited by 5 · see citing works
“Another area where the impact remediation framework affords future exploration is in modeling intersectionality, and considering in-group fairness with multi-level sensitive attributes that potentially influence each other in unknownways [8, 27, 36, 37].”
BackgroundLuo, L. K., Nakao, Y., Chollet, M., Inakoshi, H., & Stumpf, S. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710908 · Cited by 7 · see citing works
“Thus researchers have introduced subgroup fairness, applying group fairness metrics on multiple protected features to ensure fairness over subgroups [20, 35].”
Ocak, C., & Caskurlu, S. (2025) · Computers and Education Open · doi:10.1016/j.caeo.2025.100321 · Cited by 6 · see citing works
Mirror, Mirror, on the Wall, Who's the Fairest of Them All?
Xiang, A. (2024) · Daedalus · doi:10.1162/daed_a_02058 · Cited by 3 · see citing works
Data distribution tailoring revisited: cost-efficient integration of representative data
Chang, J., Cui, B., Nargesian, F., Asudeh, A., & Jagadish, H. V. (2024) · The VLDB Journal · doi:10.1007/s00778-024-00849-w · Cited by 4 · see citing works
“Subgroups could also be useful for preventing fairness gerryman-dering, where fairness requirements are met for top-level groups but not subgroups [39–41].”
Multi-fairness Under Class-Imbalance
Roy, A. G., Iosifidis, V., & Ntoutsi, E. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-18840-4_21 · Cited by 4 · see citing works
“Notions built around intersectional discrimination [5, 13] is the most common practice to measure multi-discrimination.”
Mecati, M., Torchiano, M., Vetrò, A., & Martin, J. C. D. (2023) · IEEE Access · doi:10.1109/access.2023.3252370 · Cited by 4 · see citing works
“measures [39] and in causal models [40]: however, up to our knowledge, none of these and other studies in the AI/ML fairness literature constructed and applied synthetic measures of (im)balance to intersectional protected attributes.”
BackgroundFairness in Forecasting of Observations of Linear Dynamical Systems
Zhou, Q. Z., Mareček, J., & Shorten, R. N. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14050 · Cited by 4 · see citing works
“Somewhat removed from the mainstream literature, there are some excellent works in [38, 107], who give the definition of independent subgroups, intersec-”
BackgroundArtificial Intelligence Human Rights Impact Assessment
Ortalda, A., & Hert, P. D. (2023) · Oxford University Press eBooks · doi:10.1093/law/9780192882486.003.0035 · Cited by 3 · see citing works
An Algorithmic Framework for Positive Action
Thomas, O., Zilka, M., Weller, A., & Quadrianto, N. (2021) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3465416.3483303 · Cited by 4 · see citing works
“Analysing each protected attribute separately does not capture any compounding effects that might be experienced by a specific under-represented group, for example, unmarried, not-white women [6, 10].”
BackgroundKobayashi, K., & Nakao, Y. (2021) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-87687-6_5 · Cited by 4 · see citing works
Fair Selection through Kernel Density Estimation
Jiang, X. S., Dai, Y., & Wu, Y. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191616 · Cited by 4 · see citing works
“It is worth pointing out that our method can be readily extended to multi-class and multi-sensitive feature cases, motivated by the definition of intersectional fairness [15].”
MethodArtificial Intelligence and the Right to a Healthy Environment
Quintavalla, A. (2023) · Oxford University Press eBooks · doi:10.1093/law/9780192882486.003.0028 · Cited by 4 · see citing works
Identifying Biased Subgroups in Ranking and ClassificationPreprint
Pastor, E., de Alfaro, L., & Baralis, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.07450 · Cited by 4 · see citing works
“Several approaches evaluate if different treatment or performance occur on groups determined by some sensitive or protected attributes [9, 10, 14, 16].”
BackgroundAI Life Cycle and Human Rights
Martina, Š., Luboš, K., & Drchal, J. (2023) · Oxford University Press eBooks · doi:10.1093/law/9780192882486.003.0002 · Cited by 4 · see citing works
An Empirical Study on Group Fairness Metrics of Judicial Data
Li, Y., Huang, H., Guo, X., & Yuan, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3122443 · Cited by 3 · see citing works
“From the perspective of the individual and the group, fairness is classified as individual fairness [10], [11] and group fairness [8], [12].”
BackgroundFair and Private Data Preprocessing through Microaggregation
González-Zelaya, V., Salas, J., Megías, D., & Missier, P. (2023) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3617377 · Cited by 3 · see citing works
“[22] define a multi-attribute fairness metric, differential fairness , inspired by the use of ε in differential privacy, and extends the notion of the “80% rule” [4]: in order to satisfy ε -differential fairness, the quotient of any two PA subgroups’ positive ratio ( PR ) should be greater than e −…”
BackgroundMulti-Group Fairness Evaluation via Conditional Value-at-Risk Testing
Paes, L. M., Suresh, A. T., Beutel, A., Calmon, F. P., & Beirami, A. (2024) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2024.3397741 · Cited by 3 · see citing works
“A fine-grained characterization of groups is critical for discovering performance disparities that may lead to systematic disadvantages across intersecting dimensions [23].”
- Authors
- Foulds, J. R., Islam, R., Keya, K. N., & Pan, S.
- Affiliations
- University of Maryland, Baltimore County (US)
- Venue
- 2020 IEEE 36th International Conference on Data Engineering (ICDE)
- Publisher
- IEEE
- Published
- 2020-04
- Pages
- 1918-1921
- ISSN
- 1084-4627 (ISSN-L)
- OpenAlex
- W3032340379
- Cited by
- 201
- References
- 32
- Citations by year
- 2026: 20 · 2025: 18 · 2024: 30 · 2023: 79 · 2022: 25 · 2021: 21 · 2020: 6 · 2019: 2
- FWCI
- 34.52
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- Proceedings - International Conference on Data Engineering (publishedVersion)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- intersectionality, algorithmic fairness, differential privacy, protected characteristics, generalization guarantees, privacy guarantees, census data, learning algorithms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
