On the apparent conflict between individual and group fairness
Open accessThe DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Abstract
A distinction has been drawn in fair machine learning research between 'group' and 'individual' fairness measures. Many technical research papers assume that both are important, but conflicting, and propose ways to minimise the trade-offs between these measures. This paper argues that this apparent conflict is based on a misconception. It draws on discussions from within the fair machine learning research, and from political and legal philosophy, to argue that individual and group fairness are not fundamentally in conflict. First, it outlines accounts of egalitarian fairness which encompass plausible motivations for both group and individual fairness, thereby suggesting that there need be no conflict in principle. Second, it considers the concept of individual justice, from legal philosophy and jurisprudence, which seems similar but actually contradicts the notion of individual fairness as proposed in the fair machine learning literature. The conclusion is that the apparent conflict between individual and group fairness is more of an artefact of the blunt application of fairness measures, rather than a matter of conflicting principles. In practice, this conflict may be resolved by a nuanced consideration of the sources of 'unfairness' in a particular deployment context, and the carefully justified application of measures to mitigate it.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Essentially, all normative assumptions are therefore captured in the choice of similarity metric [7].”
From Algorithmic Unfairness through the Lens of EU Non-Discrimination Law: Or Why the Law is not a Decision Tree (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“In addition, in cases where substantive differences of opinion exist between expertssuch as in the discourse on computational definitions of fairness [6,8,37]-designers of audit standards must navigate these nuances while also balancing to achieve a set of criteria that provides compliance and is operationalizable at scale.”
From A Framework for Assurance Audits of Algorithmic Systems (2024) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mapping the Margins: Intersectionality, Identity Politics, and Violence against Women of Color
Crenshaw, K. (1991) · Stanford Law Review · doi:10.2307/1229039 · Cited by 32,792 · cited 1×
“They may miss unfairness against people as a result of their being at the intersection of multiple kinds of discrimination [15], or groups which are not (yet) defined in anti-discrimination law but may need protecting [49].”
BackgroundDwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · cited 10×
“…that follows is primarily theoretical, its implications are significant both for researchers and practitioners: • For Fair-ML researchers, the apparent conflict between individual and group measures continues to motivate new research papers many years after it was initially formulated (in [18]).”
BackgroundMethodInfluentialWhat Is the Point of Equality?
Anderson, E. S. (1999) · Ethics · doi:10.1086/233897 · Cited by 3,095 · cited 2×
“Egalitarianism is the notion that people need to be treated in some senses equally, or that certain things need to be distributed according to some egalitarian principle [2, 13, 20, 42, 45, 51].”
BackgroundOn the Currency of Egalitarian Justice
Cohen, G. A. (1989) · Ethics · doi:10.1086/293126 · Cited by 2,917 · cited 1×
“things need to be distributed according to some egalitarian principle [2, 13, 20, 42, 45, 51].”
BackgroundFair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 2×
“…recidivism risk scoring algorithm, it was illustrated that it is impossible to simultaneously achieve two group-fairness measures; namely, false positive equality (which reflects the assumption of structural bias) and equal calibration (which reflects the assumption that disparities are benign) [11].”
BackgroundEquality and equal opportunity for welfare
Arneson, R. J. (1989) · Philosophical Studies · doi:10.1007/bf00646210 · Cited by 1,970 · cited 1×
“A prominent approach, known as luck egalitarianism, holds that inequalities between people are only just provided they are not the result of brute luck [4, 29, 46].”
BackgroundFriedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 1×
“Research on bias, fairness and discrimination in socio-technical systems has a long history which significantly predates specific work on fairness in machine learning (see e.g. [23, 24]).”
BackgroundWhat is Equality? Part 1: Equality of Welfare
Dworkin, R. (2018) · The Notion of Equality · doi:10.4324/9781315199795-6 · Cited by 1,180 · cited 1×
“things need to be distributed according to some egalitarian principle [2, 13, 20, 42, 45, 51].”
Background(2013) · Cited by 1,126
Bovens, M., & Zouridis, S. (2002) · Public Administration Review · doi:10.1111/0033-3352.00168 · Cited by 964 · cited 1×
“king are being automated or augmented with algorithms, the need for occasional individualised consideration is one of the reasons for keeping a human-in-the-loop, or ‘screen-level bureaucrat’ on hand [1, 8, 12, 48]. By scrutinising additional information about the individual that the algorithm does not consider, and c…”
Background
Research works printed but not linked: 9
- 10.2139/ssrn.3452030 · doi:10.2139/ssrn.3452030
- Danielle Keats Citron. 2007. Technological due process. Wash. UL Rev. 85 (2007), 1249.
- Oscar Gandy. 1995. It's discrimination, stupid. Resisting the virtual life: The culture and politics of information (1995), 35--47.
- bell hooks. 2014. Ain't IA Woman: Black women and feminism. Routledge New York, NY.
- RenÄl'e Jorgensen Bolinger. 2019. What is the Right to Individualized Judgement? Working Paper (on file with author) (2019).
- Sarah Moss. 2018. Probabilistic knowledge. Oxford University Press.
- John Rawls. 2009. A theory of justice. Harvard university press.
- Amartya Sen. 1992. Inequality reexamined. Clarendon Press.
- Larry S Temkin. 1986. Inequality. Philosophy & Public Affairs (1986), 99--121.
Cited by
Why fairness cannot be automated: Bridging the gap between EU non-discrimination law and AI
Wachter, S., Mittelstadt, B., & Russell, C. (2021) · Computer Law & Security Review · doi:10.1016/j.clsr.2021.105567 · Cited by 336
A clarification of the nuances in the fairness metrics landscape
Castelnovo, A., Crupi, R., Greco, G., Regoli, D., Penco, I. G., & Cosentini, A. C. (2022) · Scientific Reports · doi:10.1038/s41598-022-07939-1 · Cited by 218
“…with gender, the issue weather it is fair to allow for a certain gender discrimination as long as justified by income can be seen as an instance of the conflict between individual or group notions, but it is rather a conflict about the underlying ethical and legal assumptions (Binns, 2020).”
BackgroundInfluentialFairness in recommender systems: research landscape and future directions
Deldjoo, Y., Jannach, D., Bellogin, A., Difonzo, A., & Zanzonelli, D. (2024) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-023-09364-z · Cited by 228
“As a result, group and individual fairness are frequently viewed as trade-offs, which is not always immediately evident (Binns 2020).”
Fairness in Information Access Systems
Ekstrand, M. D., Das, A., Burke, R., & Diaz, F. (2022) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000079 · Cited by 133
Algorithmic Impact Assessments and Accountability: The Co-construction of Impacts
Metcalf, J., Moss, E., Watkins, E. A., Singh, R., & Elish, M. C. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445935 · Cited by 195
“…methods have demonstrated the ability to describe the former (disparate classifica-tory probabilities) in any number of important ways [1, 12, 44, 110], computational methods are less well-suited to measure the ways algorithmic systems produce and distribute the risk that people and…”
An empirical characterization of fair machine learning for clinical risk prediction
Pfohl, S. R., Foryciarz, A., & Shah, N. H. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103621 · Cited by 164
“Age Group Race and Ethnicity Cohort construction Model training and evaluation Hospital Mortality Prolonged Length of Stay Attribute: Sex Female Male Attribute: Race and Ethnicity Asian Black Hispanic Other White Attribute: Age Group [18-30) [30-45) [45-55) [55-65) [65-75) [75-90) 30-Day Readmission Performance Metric…”
BackgroundFairness in Graph Mining: A Survey
Dong, Y., Ma, J., Wang, S., Chen, C., & Li, J. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3265598 · Cited by 227
AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications
Chen, P., Wu, L., & Wang, L. (2023) · Applied Sciences · doi:10.3390/app131810258 · Cited by 155
Assessing public value failure in government adoption of artificial intelligence
Schiff, D. S., Schiff, K. J., & Pierson, P. (2022) · Public Administration · doi:10.1111/padm.12742 · Cited by 155
Policy advice and best practices on bias and fairness in AI
Álvarez, J. M., Colmenarejo, A. B., Elobaid, A., Fabbrizzi, S., Fahimi, M., Ferrara, A., Ghodsi, S., Mougan, C., Papageorgiou, I. T., Lobo, P. R., et al. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09746-w · Cited by 161
Demographic bias in misdiagnosis by computational pathology models
Vaidya, A. J., Chen, R. J., Williamson, D. F. K., Song, A. H., Jaume, G., Yang, Y., Hartvigsen, T., Dyer, E. C., Lu, M. Y., Lipková, J., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02885-z · Cited by 179
Human Judgment in algorithmic loops: Individual justice and automated decision‐making
Binns, R. (2022) · Regulation & Governance · doi:10.1111/rego.12358 · Cited by 134
Self-citationThe Causal Fairness Field Guide: Perspectives From Social and Formal Sciences
Carey, A. N., & Wu, X. (2022) · AI and Ethics · doi:10.3389/fdata.2022.892837 · Cited by 135
A Critical Survey on Fairness Benefits of Explainable AI
Deck, L., Schoeffer, J., De‐Arteaga, M., & Kühl, N. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658990 · Cited by 144
“Further, employing XAI to improve measures of individual fairness should be thoroughly grounded on normative assumptions about the source of existing group disparities [31].”
BackgroundMethodOn the Applicability of Machine Learning Fairness Notions
Makhlouf, K., Zhioua, S., & Palamidessi, C. (2021) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3468507.3468511 · Cited by 123
“Compared to group fairness notions, individual fairness notions have the drawback that they can result in “unjust disparities in outcomes between groups” [12].”
BackgroundInfluentialFairness Perceptions of Artificial Intelligence: A Review and Path Forward
Narayanan, D., Nagpal, M., McGuire, J., Schweitzer, S., & De Cremer, D. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2023.2210890 · Cited by 80
“Moreover, as computer scientists have argued, training machine learning algorithms to produce fair outcomes for individuals can sometimes come into conflict with producing fair outcomes for groups (Binns, 2020; Fleisher, 2021).”
BackgroundIn-Processing Modeling Techniques for Machine Learning Fairness: A Survey
Wan, M., Zha, D., Liu, N., & Zou, N. (2023) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3551390 · Cited by 99
“A model cannot be fair under all fairness measurements, since those fairness definitions are mutually exclusive [12].”
BackgroundBuyl, M., Cociancig, C., Frattone, C., & Roekens, N. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533169 · Cited by 119
“The nature of AI applications and more precisely the fact that output is statistically probable yet inevitably uncertain given the inherent opacity of AI, makes it difficult for claimants to realize they are treated unequally, to gather the necessary information, and to seek redress [22].”
BackgroundMachine learning fairness notions: Bridging the gap with real-world applications
Makhlouf, K., Zhioua, S., & Palamidessi, C. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102642 · Cited by 111
“Compared to group fairness notions, individual fairness notions have the drawback that they can result in “unjust disparities in outcomes between groups” [12].”
BackgroundInfluentialLearning Fair Node Representations with Graph Counterfactual Fairness
Ma, J., Guo, R., Wan, M., Yang, L., Zhang, A., & Li, J. (2022) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3488560.3498391 · Cited by 138
“Fairness in machine learning has attracted significant attention recently [5, 26, 39].”
BackgroundWhat's Fair About Individual Fairness?
Fleisher, W. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462621 · Cited by 141
“Along with (Binns 2020), I think this idea of us-ing the methods together should be standard.”
BackgroundMethodInfluentialMeasuring Fairness in Ranked Results: An Analytical and Empirical Comparison
Raj, A., & Ekstrand, M. D. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3532018 · Cited by 106
A translational perspective towards clinical AI fairness
Liu, M., Ning, Y., Teixayavong, S., Mertens, M., Xu, J., Ting, D. S. W., Cheng, L. T.-E., Ong, J. C. L., Teo, Z. L., Tan, T. F., et al. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00918-4 · Cited by 82
“Due to the trade-offs between the metrics, it is mathematically impossible to optimize all metrics simultaneously, except in highly restrictive cases 26,33,35 .”
The Use of Responsible Artificial Intelligence Techniques in the Context of Loan Approval Processes
Purificato, E., Lorenzo, F., Fallucchi, F., & Luca, E. W. D. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2081284 · Cited by 89
“Furthermore, different people have different sensibilities about what is fair and what is not, and what is fair considering individuals or populations as a group (Binns, 2020).”
BackgroundA Framework for Assurance Audits of Algorithmic Systems
Lam, K., Lange, B., Blili-Hamelin, B., Davidovic, J., Brown, S., & Hasan, A. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658957 · Cited by 75
“In addition, in cases where substantive differences of opinion exist between experts— such as in the discourse on computational definitions of fairness [6, 8, 37]—designers of audit standards must navigate these nuances while also balancing to achieve a set of criteria that provides compliance and is…”
BackgroundLee, M. S. A., Floridi, L., & Singh, J. (2021) · AI and Ethics · doi:10.1007/s43681-021-00067-y · Cited by 71
“Another paper distinguishes between “benign” disparities and “structural bias” that should be corrected [6].”
InfluentialToward involving end-users in interactive human-in-the-loop AI fairness
Nakao, Y., Stumpf, S., Ahmed, S., Naseer, A., & Strappelli, L. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3514258 · Cited by 69
“Fairness is often associated with group fairness and individual fairness [9].”
Kasinidou, M., Kleanthous, S., Barlas, P., & Otterbacher, J. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445931 · Cited by 57
“Therefore, for developers to enable full judgement of an algorithmic process, it may not be enough to give information about the process in the abstract, but provide concrete details about the cases involved [4].”
BackgroundCan Explainable AI Explain Unfairness? A Framework for Evaluating Explainable AIPreprint
Alikhademi, K., Richardson, B., Drobina, E., & Gilbert, J. E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.07483 · Cited by 79
“Additionally, statistical fairness is limited by its need for clearly defined categories [9].”
Algorithmic fairness through group parities? The case of COMPAS-SAPMOC
Lagioia, F., Rovatti, R., & Sartor, G. (2022) · AI & Society · doi:10.1007/s00146-022-01441-y · Cited by 79
“It measures the proportion of erroneous predictions for each class relative to the total predictions (Binns 2020; Chouldechova 2017; Barocas and Selbst 2016).”
BackgroundFairness and Decision-making in Collaborative Shift Scheduling Systems
Uhde, A., Schlicker, N., Wallach, D. P., & Hassenzahl, M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376656 · Cited by 64
“This implies fundamental issues when treating it as a purely mathematical decision problem [6].”
BackgroundCommunity-in-the-loop: towards pluralistic value creation in AI, or—why AI needs business ethics
Häußermann, J. J., & Lütge, C. (2022) · AI and Ethics · doi:10.1007/s43681-021-00047-2 · Cited by 57
“In fact, even if an AI system is completely ethically designed on a technical level—if this is possible at all and whatever that may mean with regard to say fairness, privacy or safety in particular [13, 44, 68]—major ethical questions may arise.”
BackgroundImproving Recommendation Fairness via Data Augmentation
Chen, L., Wu, L., Zhang, K., Hong, R., Lian, D., Zhang, Z., Zhou, J., & Wang, M. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583341 · Cited by 89
“Individual fairness refers to producing similar predictions for similar individuals, and group fairness argues not discriminating a particular user group based on the sensitive attribute [5, 7, 52, 54].”
BackgroundAssessing trustworthy AI: Technical and legal perspectives of fairness in AI
Kattnig, M., Angerschmid, A., Reichel, T., & Kern, R. (2024) · Computer law & security review · doi:10.1016/j.clsr.2024.106053 · Cited by 77
Evaluating Robustness of Counterfactual Explanations
Artelt, A., Vaquet, V., Velioğlu, R., Hinder, F., Brinkrolf, J., Schilling, M., & Hammer, B. (2021) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci50451.2021.9660058 · Cited by 105
Normative Challenges of Risk Regulation of Artificial Intelligence
Orwat, C., Bareis, J., Folberth, A., Jahnel, J., & Wadephul, C. (2024) · NanoEthics · doi:10.1007/s11569-024-00454-9 · Cited by 73
“Widely discussed trade-offs include the fairness-accuracy trade-off [101, 102] and the decision between group fairness and individual fairness [103].”
On the Moral Justification of Statistical Parity
Hertweck, C., Heitz, C., & Loi, M. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445936 · Cited by 46
“However, it may be objected that similarities in the CS under the influence of injustice do not provide a suitable reference point to define what is a fair prediction or decision [6].”
BackgroundInfluentialInternational Scientific Report on the Safety of Advanced AI (Interim Report)Preprint
Bengio, Y., Mindermann, S., Privitera, D., Besiroglu, T., Bommasani, R., Casper, S., Choi, Y., Goldfarb, D. A., Heidari, H., Khalatbari, L., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.05282 · Cited by 79
Are “Intersectionally Fair” AI Algorithms Really Fair to Women of Color? A Philosophical Analysis
Kong, Y. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533114 · Cited by 58
“” It appeals to the moral intuition that there should be no discrimination against certain groups based on their race, gender, class, and other identity categories [5, 32, 36].”
BackgroundInherent Limitations of AI Fairness
Buyl, M., & Bie, T. D. (2024) · Communications of the ACM · doi:10.1145/3624700 · Cited by 73
“36 Group fairness expresses the principles of individual fairness 6 by looking at the sum of discrimination toward an entire group rather than individual contributions.”
BackgroundEnvisioning Equitable Speech Technologies for Black Older Adults
Brewer, R. N., Harrington, C., & Heldreth, C. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594005 · Cited by 42
“Current approaches can hyper-focus on quantitative methods that ignore structural inequity (e.g., group vs. individual fairness)(e.g., [7]).”
BackgroundBalayn, A., Yurrita, M., Yang, J., & Gadiraju, U. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604674 · Cited by 64
“Using ML for certain tasks might be questioned, for instance because it means making decisions for people by comparing them to others instead of following the principle of individual justice [9, 26], or because it reproduces historical, potentially harmful, data patterns [81].”
BackgroundNaderi, L., Karamidehkordi, E., Badsar, M., & Moghadas, M. (2024) · Journal of Hydrology Regional Studies · doi:10.1016/j.ejrh.2024.101878 · Cited by 65
Towards Environmentally Equitable AI via Geographical Load Balancing
Li, P., Yang, J., Wierman, A., & Ren, S. (2024) · ACM International Conference on Future Energy Systems (e-Energy) · doi:10.1145/3632775.3661938 · Cited by 60
Algorithmic individual fairness and healthcare: a scoping review
Anderson, J. W., & Visweswaran, S. (2024) · JAMIA Open · doi:10.1093/jamiaopen/ooae149 · Cited by 60
“45 Second, GF may disregard relevant features that are not protected attributes, whereas IF may lead to more accurate and fair outcomes by considering all relevant features.”
Fairness metrics and bias mitigation strategies for rating predictions
Ashokan, A., & Haas, C. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102646 · Cited by 47
What Is Fair? Exploring the Artists’ Perspective on the Fairness of Music Streaming Platforms
Ferraro, A., Serra, X., & Bauer, C. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85616-8_33 · Cited by 69
Barlas, P., Kyriakou, K., Guest, O., Kleanthous, S., & Otterbacher, J. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3432931 · Cited by 40
“In contrast, individual fairness concerns the notion of consistency, holding that similar individuals should receive similar treatment by the system [9].”
Making fair ML software using trustworthy explanation
Chakraborty, J., Peng, K., & Menzies, T. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3418932 · Cited by 53
“Now what should the employer do? Should the employer continue to interview the female candidates or adjust the model again to ensure that any ‘more qualified’ male get interview calls instead? So, there is conflict between Group Fairness & Individual Fairness[20].”
BackgroundFair Enough: Searching for Sufficient Measures of Fairness
Majumder, S., Chakraborty, J., Bai, G. R., Stolee, K. T., & Menzies, T. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3585006 · Cited by 54
“As evidence, we note that the majority fairness decision is always the same within the clusters for each dataset.”
BackgroundAntidiscrimination Insurance Pricing: Regulations, Fairness Criteria, and Models
Xin, X., & Huang, F. (2024) · North American Actuarial Journal · doi:10.1080/10920277.2023.2190528 · Cited by 43
“An inevitable conflict may exist between group fairness and individual fairness; see also Binns (2020).”
BackgroundGroup-level Fairness Maximization in Online Bipartite Matching
Ma, W., Xu, P., & Xu, Y. (2022) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/psce2714 · Cited by 56
What-is and How-to for Fairness in Machine Learning: A Survey, Reflection, and Perspective
Tang, Z., Zhang, J., & Zhang, K. (2023) · ACM Computing Surveys · doi:10.1145/3597199 · Cited by 61
“The connection between group-level and individual-level fairness notions beyond their seemingly apparent conflicts also draws attentions [21, 65, 169].”
BackgroundLogothetis, S. B., Green, D., Holland, M., & Moubayed, N. A. (2023) · Scientific Reports · doi:10.1038/s41598-023-40661-0 · Cited by 51
“We investigate two forms of undesirable bias: individual, representing how dissimilarly we treat individuals who deserve similar outcomes 68 , and group-based, measuring the inequality of predictions between demographic groups defined by protected characteristics 69 .”
The Impact of Artificial Intelligence on Digital Marketing StrategiesPreprint
Wilson, G., Johnson, O., & Brown, W. (2024) · Preprints.org · doi:10.20944/preprints202408.0276.v1 · Cited by 50
Using Artificial Intelligence in the workplace: What are the main ethical risks?
Pero, A. S. D., Wyckoff, P., & Vourc'h, A. (2022) · OECD social employment and migration working papers · doi:10.1787/840a2d9f-en · Cited by 62
Perception of Fairness in Algorithmic Decisions: Future Developers' Perspective
Kleanthous, S., Kasinidou, M., Barlas, P., & Otterbacher, J. (2021) · Patterns · doi:10.1016/j.patter.2021.100380 · Cited by 48
Ghasemaghaei, M., & Kordzadeh, N. (2024) · Information Systems Journal · doi:10.1111/isj.12572 · Cited by 46
“On the other hand, individual un-fairness entails inequitable treatment of decision subjects based on the characteristics of their broader group, rather than considering their relevant attributes, such as merits and quali-fications (Binns 2020).”
Requirements for Trustworthy Artificial Intelligence and its Application in Healthcare
Kim, M., Sohn, H., Choi, S. K., & Kim, S. (2023) · Healthcare Informatics Research · doi:10.4258/hir.2023.29.4.315 · Cited by 45
“As such, procedural and technical improvements are needed to quantify the reliability of an algorithm.”
BackgroundAvoiding Disparity Amplification under Different Worldviews
Yeom, S., & Tschantz, M. C. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445892 · Cited by 40
“Although individual fairness is sometimes considered to be in conflict with group-based notions of fairness, Binns [3] argues otherwise, instead pointing to the difference in worldviews as the truly important factor.”
BackgroundBeigang, F. (2022) · Minds and Machines · doi:10.1007/s11023-022-09615-9 · Cited by 43
“To generate the prediction, the system might, as mentioned above, take as input data information about the applicant’s repayment history, education, and employment (see Lee & Floridi 2020).”
MethodWeerts, H., Xenidis, R., Tarissan, F., Olsen, H. P., & Pechenizkiy, M. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594044 · Cited by 34
“Essentially, all normative assumptions are therefore captured in the choice of similarity metric [7].”
BackgroundAdomavičius, G., & Yang, M. (2019) · ACM Transactions on Management Information Systems · doi:10.1145/3519420 · Cited by 37
“same gender group) [183], and between group fairness and individual fairness [28].”
BackgroundBiasRV: uncovering biased sentiment predictions at runtime
Yang, Z., Asyrofi, M. H., & Lo, D. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3473117 · Cited by 37
“This evaluation metric is also similar to the group fairness concept proposed by previous researchers [3, 4], which is described as the goal that privileged and unprivileged groups are treated similarly.”
ResultBias on Demand: A Modelling Framework That Generates Synthetic Data With Bias
Baumann, J., Castelnovo, A., Crupi, R., Inverardi, N., & Regoli, D. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594058 · Cited by 43
“Several works emphasise how the moral appropriateness of certain notions of fairness heavily depends on the context, such as Baumann and Loi [9], who provide an ethical argument in favour of the sufficiency criterion in the context of personalised insurance premiums or Binns [11], who argues that in other applications…”
BackgroundMethodInfluentialMhasawade, V., & Chunara, R. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462587 · Cited by 43
“op concepts of fairness with respect to individual level sensitive attributes, while here we develop fairness accounting for multilevel sensitive attributes. Inspired from research in social sciences [6, 21, 49], the work here extends the idea to multi-level sensitive attributes. In our considered setting, a decision…”
BackgroundFairRankVis: A Visual Analytics Framework for Exploring Algorithmic Fairness in Graph Mining Models
Xie, T., Ma, Y., Kang, J. L., Tong, H., & Maciejewski, R. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3114850 · Cited by 35
“However, con-flicting concepts of group and individual fairness [3] can lead to cases where an algorithm that has been debiased at the group level now introduces bias at the individual level.”
BackgroundTrustworthy Artificial Intelligence and Process Mining: Challenges and Opportunities
Pery, A., Rafiei, M., Simon, M. A., & van der Aalst, W. M. P. (2022) · Lecture notes in business information processing · doi:10.1007/978-3-030-98581-3_29 · Cited by 31
Characterizing Intersectional Group Fairness with Worst-Case ComparisonsPreprint
Ghosh, A., Genuit, L., & Reagan, M. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.01673 · Cited by 44
“Binns (2019) introduces those two notions and discusses the motivations behind individual and group fairness.”
BackgroundSurvey on Fair Reinforcement Learning: Theory and PracticePreprint
Gajane, P., Saxena, A., Tavakol, M., Fletcher, G., & Pechenizkiy, M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.10032 · Cited by 34
What is fair? Proxy discrimination vs. demographic disparities in insurance pricing
Lindholm, M., Richman, R., Tsanakas, A., & Wüthrich, M. V. (2024) · Scandinavian Actuarial Journal · doi:10.1080/03461238.2024.2364741 · Cited by 37
“The potential conflict between those two classes of fairness criteria is discussed in Binns (2020) and Friedler et al. (2016), using, respectively, discursive and technical arguments but reaching consistent conclusions: that such conflicts demonstrate the need to clarify ideas about justice and the…”
Assessing Multilingual Fairness in Pre-trained Multimodal Representations
Wang, J., Liu, Y., & Wang, X. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.211 · Cited by 47
Smith, J. J., Beattie, L., & Cramer, H. S. M. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583204 · Cited by 34
“Generalized Cross Entropy [6] Group Utility System should distribute utility equally be-tween groups of consumers.”
BackgroundToward fair graph neural networks via real counterfactual samples
Wang, Z., Qiu, M., Chen, M., Salem, M. B., Yao, X., & Zhang, W. (2024) · Knowledge and Information Systems · doi:10.1007/s10115-024-02161-z · Cited by 46
De-centering the (Traditional) user: Multistakeholder evaluation of recommender systems
Burke, R., Adomavičius, G., Bogers, T., Noia, T. D., Kowald, D., Neidhardt, J., Özgöbek, Ö., Pera, M. S., Tintarev, N., & Ziegler, J. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103560 · Cited by 30
Counterfactually Fair Automatic Speech Recognition
Sar, L., Hasegawa‐Johnson, M., & Yoo, C. D. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3126949 · Cited by 46
“A full discussion of whether there is a trade-off between individual and group fairness is out of scope of this study and we refer to [41] for a relevant discussion.”
BackgroundEthical implications of fairness interventions: what might be hidden behind engineering choices?
Tubella, A. A., Barsotti, F., Koçer, R. G., & Mendez, J. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09636-z · Cited by 29
“Understanding the inherent trade-offs and implications behind a fairness definition is therefore crucial for organisations and practitioners to justify and trace their implementation choices (Binns, 2020).”
BackgroundA Hippocratic Oath for Mathematicians? Mapping the Landscape of Ethics in Mathematics
Müller, D., Chiodo, M., & Franklin, J. (2022) · Science and Engineering Ethics · doi:10.1007/s11948-022-00389-y · Cited by 30
“Recent research has shown that even considerations such as fairness and bias are often interpreted with different and competing notions and results: individual fairness does not necessarily imply or contradict group fairness, nor vice versa (Binns 2020).”
BackgroundMao, Y., Ye, Z., Yuan, X., & Zhong, S. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3416042 · Cited by 33
“Algorithmic bias has been extensively studied in the field of machine learning, where individual fairness and group fairness are two common fairness notions [56].”
BackgroundLimeOut: An Ensemble Approach to Improve Process Fairness
Bhargava, V., Couceiro, M., & Napoli, A. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-65965-3_32 · Cited by 36
Kumar, I. E., Hines, K. E., & Dickerson, J. P. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534154 · Cited by 24
“Individual fairness enjoys ideological alignment with what many would consider to be fair decision-making; Binns [12] has argued that it shares a motivation with Aristotle’s conception of justice as consistency.”
Design and Appropriation of Computer-supported Self-scheduling Practices in Healthcare Shift Work
Uhde, A., Laschke, M., & Hassenzahl, M. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449219 · Cited by 28
“Instead, particularized, case specific justice [6] may be important.”
BackgroundBerk, R. A., Kuchibhotla, A. K., & Tchetgen, E. T. (2023) · Sociological Methods & Research · doi:10.1177/00491241231155883 · Cited by 33
“Perhaps more fundamentally, it may be that class fairness and individual fairness are inherently incompatible (Binns 2020; Zemel et al. 2013).”
Bias in AI-Based Decision-Making
Kadiresan, A., Baweja, Y., & Ogbanufe, O. (2022) · Educational communications and technology: issues and innovations · doi:10.1007/978-3-030-84729-6_19 · Cited by 24
Causality-Aided Trade-Off Analysis for Machine Learning Fairness
Ji, Z., Ma, P., Wang, S., & Li, Y. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00105 · Cited by 29
“However, the limitation of AFI results in reduced effectiveness.”
BackgroundThe Data Protection Impact Assessment as a Tool to Enforce Non-Discriminatory AI
Ivanova, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-55196-4_1 · Cited by 23
The Ethics of Online Controlled Experiments (A/B Testing)
Polonioli, A., Ghioni, R., Greco, C., Juneja, P., Tagliabue, J., Watson, D., & Floridi, L. (2023) · Minds and Machines · doi:10.1007/s11023-023-09644-y · Cited by 30
“We agree with recent literature showing that these characterizations of fairness may be mutually compatible (Binns, 2020).”
Unified Group Fairness on Federated LearningPreprint
Zhang, F., Kuang, K., Liu, Y., Chen, L., Wu, C., Wu, F. Q., Lu, J., Shao, Y., & Xiao, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.04986 · Cited by 28
“Fairness in machine learning has attracted much attention, which can be divided into two branches: individual fairness and group fairness (Zemel et al. 2013; Awasthi et al. 2020; Binns 2020).”
Background“Democratizing AI” and the Concern of Algorithmic Injustice
Lin, T.-A. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00792-2 · Cited by 26
“For example, in considering suitable responses to algorithmic injustice, Zimmermann et al. (2020) indicated that “Broaching questions of algorithmic justice via the democratic process would give members of communities most impacted by algorithmic bias more direct democratic power over crucial decisions concerning weak…”
InfluentialGenetic adversarial training of decision trees
Ranzato, F., & Zanella, M. (2021) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3449639.3459286 · Cited by 28
“Hyperparameters for training random forests have been selected by a randomized grid search on the number of trees ranging in the interval [5, 100], maximum depth in [5, 100], and either Gini G or entropy H for split purity.”
BackgroundVerifying Fairness in Quantum Machine Learning
Guan, J., Fang, W. S., & Ying, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-13188-2_20 · Cited by 28
Faire: Repairing Fairness of Neural Networks via Neuron Condition Synthesis
Li, T., Xie, X., Wang, J., Guo, Q., Liu, A., Ma, L., & Liu, Y. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3617168 · Cited by 29
“However, the evaluations of group fairness and individual fairness are totally diferent [10].”
BackgroundSong, Z., Zhang, Y., & King, I. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615091 · Cited by 37
Mitigating Multisource Biases in Graph Neural Networks via Real Counterfactual Samples
Wang, Z., Narasimhan, G., Yao, X., & Zhang, W. (2023) · Proceedings · doi:10.1109/icdm58522.2023.00073 · Cited by 40
On prediction-modelers and decision-makers: why fairness requires more than a fair prediction model
Scantamburlo, T., Baumann, J., & Heitz, C. (2024) · AI & Society · doi:10.1007/s00146-024-01886-3 · Cited by 28
“This type of fairness intends to avoid systematic disadvantages of algorithmic decisions with respect to a sensitive attribute (such as gender, age, or race) (Binns 2020; Barocas et al. 2019).”
BeFair: Addressing Fairness in the Banking Sector
Castelnovo, A., Crupi, R., Gamba, G. D., Greco, G., Naseer, A., Regoli, D., & González, B. S. M. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9377894 · Cited by 29
“Nevertheless, it is still not clear whether these two notions are mutually compatible [5], [6].”
BackgroundAzman, Ö., & Tümkaya, S. (2025) · F1000Research · doi:10.12688/f1000research.160011.1 · Cited by 47
Nuka, T. F., & Ogunola, A. A. (2024) · International Journal of Science and Research Archive · doi:10.30574/ijsra.2024.13.2.2258 · Cited by 29
Artificial Intelligence in Project Management: Balancing Automation and Human Judgment
Al-Arafat, M., Kabir, M. E., Morshed, A., & Islam, M. M. (2025) · Innovatech Engineering Journal · doi:10.70937/faet.v1i02.47 · Cited by 27
Berk, R. A., Kuchibhotla, A. K., & Tchetgen, E. T. (2022) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-033021-120649 · Cited by 27
Metadata
- Authors
- Binns, R.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Binns, Reuben
- ORCID
- Binns, Reuben
- Venue
- Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2020-01-27
- Pages
- 514-524
- OpenAlex
- W2996424229
- Cited by
- 312
- References
- 46
- Citations by year
- 2026: 26 · 2025: 59 · 2024: 61 · 2023: 62 · 2022: 45 · 2021: 41 · 2020: 17
- FWCI
- 42.79
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- EPSRC (N02334X/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Political Science and International Relations › Political Philosophy and Ethics
- Social Sciences › Law › Discrimination and Equality Law
- Keywords
- individual fairness, group fairness, algorithmic fairness, individual justice, legal theory, fairness metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.