On the apparent conflict between individual and group fairness
Open accessThe DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Abstract
A distinction has been drawn in fair machine learning research between 'group' and 'individual' fairness measures. Many technical research papers assume that both are important, but conflicting, and propose ways to minimise the trade-offs between these measures. This paper argues that this apparent conflict is based on a misconception. It draws on discussions from within the fair machine learning research, and from political and legal philosophy, to argue that individual and group fairness are not fundamentally in conflict. First, it outlines accounts of egalitarian fairness which encompass plausible motivations for both group and individual fairness, thereby suggesting that there need be no conflict in principle. Second, it considers the concept of individual justice, from legal philosophy and jurisprudence, which seems similar but actually contradicts the notion of individual fairness as proposed in the fair machine learning literature. The conclusion is that the apparent conflict between individual and group fairness is more of an artefact of the blunt application of fairness measures, rather than a matter of conflicting principles. In practice, this conflict may be resolved by a nuanced consideration of the sources of 'unfairness' in a particular deployment context, and the carefully justified application of measures to mitigate it.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Essentially, all normative assumptions are therefore captured in the choice of similarity metric [7].”
From Algorithmic Unfairness through the Lens of EU Non-Discrimination Law: Or Why the Law is not a Decision Tree (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“In addition, in cases where substantive differences of opinion exist between expertssuch as in the discourse on computational definitions of fairness [6,8,37]-designers of audit standards must navigate these nuances while also balancing to achieve a set of criteria that provides compliance and is operationalizable at scale.”
From A Framework for Assurance Audits of Algorithmic Systems (2024) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mapping the Margins: Intersectionality, Identity Politics, and Violence against Women of Color
Crenshaw, K. (1991) · Stanford Law Review · doi:10.2307/1229039 · Cited by 32,792 · cited 1×
“They may miss unfairness against people as a result of their being at the intersection of multiple kinds of discrimination [15], or groups which are not (yet) defined in anti-discrimination law but may need protecting [49].”
BackgroundDwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · cited 10×
“…that follows is primarily theoretical, its implications are significant both for researchers and practitioners: • For Fair-ML researchers, the apparent conflict between individual and group measures continues to motivate new research papers many years after it was initially formulated (in [18]).”
BackgroundMethodInfluentialWhat Is the Point of Equality?
Anderson, E. S. (1999) · Ethics · doi:10.1086/233897 · Cited by 3,095 · cited 2×
“Egalitarianism is the notion that people need to be treated in some senses equally, or that certain things need to be distributed according to some egalitarian principle [2, 13, 20, 42, 45, 51].”
BackgroundOn the Currency of Egalitarian Justice
Cohen, G. A. (1989) · Ethics · doi:10.1086/293126 · Cited by 2,917 · cited 1×
“things need to be distributed according to some egalitarian principle [2, 13, 20, 42, 45, 51].”
BackgroundFair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 2×
“…recidivism risk scoring algorithm, it was illustrated that it is impossible to simultaneously achieve two group-fairness measures; namely, false positive equality (which reflects the assumption of structural bias) and equal calibration (which reflects the assumption that disparities are benign) [11].”
BackgroundEquality and equal opportunity for welfare
Arneson, R. J. (1989) · Philosophical Studies · doi:10.1007/bf00646210 · Cited by 1,970 · cited 1×
“A prominent approach, known as luck egalitarianism, holds that inequalities between people are only just provided they are not the result of brute luck [4, 29, 46].”
BackgroundFriedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 1×
“Research on bias, fairness and discrimination in socio-technical systems has a long history which significantly predates specific work on fairness in machine learning (see e.g. [23, 24]).”
BackgroundWhat is Equality? Part 1: Equality of Welfare
Dworkin, R. (2018) · The Notion of Equality · doi:10.4324/9781315199795-6 · Cited by 1,180 · cited 1×
“things need to be distributed according to some egalitarian principle [2, 13, 20, 42, 45, 51].”
Background(2013) · Cited by 1,126
Bovens, M., & Zouridis, S. (2002) · Public Administration Review · doi:10.1111/0033-3352.00168 · Cited by 964 · cited 1×
“king are being automated or augmented with algorithms, the need for occasional individualised consideration is one of the reasons for keeping a human-in-the-loop, or ‘screen-level bureaucrat’ on hand [1, 8, 12, 48]. By scrutinising additional information about the individual that the algorithm does not consider, and c…”
BackgroundInherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943 · cited 2×
“corresponds to the actual frequency of the event happening) [35].”
BackgroundInherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941 · cited 2×
“corresponds to the actual frequency of the event happening) [35].”
Background(2011) · Choice Reviews Online · doi:10.5860/choice.49-0546 · Cited by 738 · cited 1×
“lication of philosophical accounts of justice to Fair-ML might not always be possible; for instance, those cited above who reject liberal egalitarian tenets like the choice / circumstance distinction [14, 28, 53] might be sceptical about the possibility of ever finding features which are untainted by oppressive structu…”
BackgroundDiscrimination-aware data mining
Pedreshi, D., Ruggieri, S., & Turini, F. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401959 · Cited by 676 · cited 1×
“These early papers used fairness measures based on statistical parity between protected groups (e.g. gender, race) in each outcome class, and hence are classed as group fairness measures [10, 39].”
BackgroundFairness in Machine Learning: Lessons from Political Philosophy
Binns, R. (2017) · arXiv · doi:10.48550/arxiv.1712.03586 · Cited by 676 · cited 2×
“to be distributed according to some egalitarian principle [2, 13, 20, 42, 45, 51]. An overview of egalitarian theories of justiceandtheirrelationshiptoFair-MLisbeyond theremitofthis paper(forwhich,see[6]).Whileegalitarians advocateforequality, they usually do not mean that everyone should be allocated an equal share o…”
BackgroundSelf-citationWhere fairness fails: data, algorithms, and the limits of antidiscrimination discourse
Hoffmann, A. L. (2019) · Information, Communication & Society · doi:10.1080/1369118x.2019.1573912 · Cited by 535 · cited 2×
“[6, 26, 27] statutes, case law and institutional history, judges ought to come to the same conclusion.”
BackgroundVeale, M., Van Kleek, M., & Binns, R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174014 · Cited by 490 · cited 1×
“In contexts in which public administration and private decision-making are being automated or augmented with algorithms, the need for occasional individualised consideration is one of the reasons for keeping a human-in-the-loop, or ‘screen-level bureaucrat’ on hand [1, 8, 12, 48].”
BackgroundSelf-citationPreventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
Kearns, M., Neel, S., Roth, A., & Wu, Z. S. (2017) · arXiv · doi:10.48550/arxiv.1711.05144 · Cited by 469 · cited 2×
“black women), also known as fairness gerrymandering [34].”
BackgroundBuilding Classifiers with Independency Constraints
Calders, T., Kamiran, F., & Pechenizkiy, M. (2009) · 2009 IEEE International Conference on Data Mining Workshops · doi:10.1109/icdmw.2009.83 · Cited by 468 · cited 1×
“gender, race) in each outcome class, and hence are classed as group fairness measures [10, 39].”
Background50 Years of Test (Un)fairness: Lessons for Machine Learning
Hutchinson, B., & Mitchell, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287600 · Cited by 297 · cited 1×
“p fairness measure. 1Formalised definitions of fairness which foreshadow those in Fair-ML did appear in areas other than ML, such as testing in education and hiring. See Hutchinson & Mitchell 2019 [30]. On the Apparent Conflict Between Individual and Group Fairness FAT* ’20, January 27–30, 2020, Barcelona, Spain 2.…”
BackgroundInfluentialWesten, P. (1982) · Harvard Law Review · doi:10.2307/1340593 · Cited by 288 · cited 1×
“This has lead some to criticise Aristotle’s notion of equality as being ‘empty’ [44, 52].”
BackgroundOn the (im)possibility of fairness
Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2016) · arXiv · doi:10.48550/arxiv.1609.07236 · Cited by 247 · cited 3×
“The point above - that it is necessary to make contestable empirical and normative assumptions in order to pursue the aims of Fair-ML in a meaningful way - is closely related to arguments put forward in previous work by Friedler et al. [22].”
BackgroundResultInfluentialRoemer, J. E. (1985) · Economics and Philosophy · doi:10.1017/s0266267100002455 · Cited by 237 · cited 1×
“utes or circumstancespeoplecanrightlybeheldresponsiblefor.Theymay disagree, for instance, about whether the distribution of natural talents should be regarded as a matter of luck and therefore unjust [43], or instead whether people should be permitted to reap the rewards of their natural talents [42]. Some critics que…”
BackgroundFairness in Learning: Classic and Contextual Bandits
Joseph, M., Kearns, M., Morgenstern, J., & Roth, A. (2016) · arXiv · doi:10.48550/arxiv.1605.07139 · Cited by 199 · cited 3×
“To avoid these problems, various papers have proposed an individuallevel fairness measure [18, 32, 36].”
BackgroundStreet-Level Algorithms: A Theory at the Gaps Between Policy and Decisions
Alkhatib, A., & Bernstein, M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300760 · Cited by 188 · cited 1×
“In contexts in which public administration and private decision-making are being automated or augmented with algorithms, the need for occasional individualised consideration is one of the reasons for keeping a human-in-the-loop, or ‘screen-level bureaucrat’ on hand [1, 8, 12, 48].”
BackgroundBorn free and equal?: a philosophical inquiry into the nature of discrimination
(2014) · Choice Reviews Online · doi:10.5860/choice.51-5521 · Cited by 159
iFair: Learning Individually Fair Data Representations for Algorithmic Decision Making
Lahoti, P., Gummadi, K. P., & Weikum, G. (2019) · 2019 IEEE 35th International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2019.00121 · Cited by 157
Cooper, B. (2015) · The Oxford Handbook of Feminist Theory · doi:10.1093/oxfordhb/9780199328581.013.20 · Cited by 146
A Moral Framework for Understanding Fair ML through Economic Models of Equality of Opportunity
Heidari, H., Loi, M., Gummadi, K. P., & Krause, A. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287584 · Cited by 139 · cited 1×
“As proposed in recent work by Hei-dari et. al [26], features can be separated into those for which individuals are not responsible, and those for which they are.”
Method50 Years of Test (Un)fairness: Lessons for Machine Learning
Hutchinson, B., & Mitchell, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.10104 · Cited by 118 · cited 1×
“p fairness measure. 1Formalised definitions of fairness which foreshadow those in Fair-ML did appear in areas other than ML, such as testing in education and hiring. See Hutchinson & Mitchell 2019 [30]. On the Apparent Conflict Between Individual and Group Fairness FAT* ’20, January 27–30, 2020, Barcelona, Spain 2.…”
BackgroundInfluentialFailing to Treat Persons as Individuals
Beeghly, E. (2018) · Ergo an Open Access Journal of Philosophy · doi:10.3998/ergo.12405314.0005.026 · Cited by 71
When bad things happen to good people: Luck egalitarianism and costly rescues
Thaysen, J. D., & Albertsen, A. (2017) · Politics, Philosophy & Economics · doi:10.1177/1470594x16666017 · Cited by 68 · cited 1×
“to help others [2, 47].”
Eliciting and Enforcing Subjective Individual Fairness.Preprint
Jung, C., Kearns, M. J., Neel, S., Roth, A., Stapleton, L., & Wu, Z. S. (2019) · arXiv (Cornell University) · Cited by 59
Woozley, A. D. (1979) · The Philosophical Quarterly · doi:10.2307/2219180 · Cited by 46
The Promise and Peril of Predictive Analytics in Higher Education: A Landscape Analysis.
Ekowo, M., & Palmer, I. (2016) · Cited by 46
Calibration for the (Computationally-Identifiable) Masses
Hébert-Johnson, Ú., Kim, M. P., Reingold, O., & Rothblum, G. N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.08513 · Cited by 44 · cited 1×
“subgroup doesn’t scale well to the large number of possible combinations, and may lead to overfitting [25, 34].”
BackgroundA Brief Introduction of the Puzzle of Discretion
Pratt, A. C., & Sossin, L. M. (2009) · Canadian Journal of Law and Society / Revue Canadienne Droit et Société · doi:10.1017/s082932010001005x · Cited by 42 · cited 2×
“sures to evaluate the fairness of its lending algorithm?If it optsfor groupfairness, this couldinvolve, for example, re-weighting the historical data to edit feature values to increase group fairness [41]. Alternatively, the bank could opt for individualfairness. This wouldinvolve creating a task-relevant similarity m…”
BackgroundMethod(2013) · Cited by 42
Race Ipsa Loquitur: Of Reasonable Racists, Intelligent Bayesians, and Involuntary Negrophobes
Armour, J. D. (1994) · Stanford Law Review · doi:10.2307/1229093 · Cited by 37
Can Luck Egalitarianism Justify the Fact that Some are Worse Off than Others?
Huseby, R. (2015) · Journal of Applied Philosophy · doi:10.1111/japp.12122 · Cited by 34 · cited 1×
“A prominent approach, known as luck egalitarianism, holds that inequalities be-tween people are only just provided they are not the result of brute luck [4, 29, 46].”
BackgroundHuman Judgement in Algorithmic Loops; Individual Justice and Automated Decision-MakingPreprint
Binns, R. (2019) · SSRN Electronic Journal · doi:10.31228/osf.io/kz4s2 · Cited by 20 · cited 1×
“5For a sceptical appraisal of such claims, see [7].”
BackgroundSelf-citationEinzelfallgerechtigkeit versus Generalisierung
Britz, G. (2025) · Mohr Siebeck eBooks · doi:10.1628/978-3-16-166857-9 · Cited by 10
(2008) · Cited by 9
On Treating Unlike Cases Alike
Schauer, F. F. (2016) · Constitutional Commentary · doi:10.24926/26390742.2379 · Cited by 7 · cited 3×
“addition to saying that like FAT* ’20, January 27–30, 2020, Barcelona, Spain Reuben Binns cases should be treated alike, Aristotle described a different (and possibly contrary) position (as quoted in [44]): ‘There are some things about which it is not possible to pronounce rightly in general terms ... the raw material…”
BackgroundInfluentialA Unified Theory of Detention, with Application to Preventive Detention for Suspected Terrorists
Walen, A. D. (2011) · Digital Commons at University of Maryland Carey Law (University of Maryland Francis King Carey School of Law) · Cited by 4
A Right to Reasonable Inferences
Wachter, S., & Mittelstadt, B. (2019) · Columbia Business Law Review · doi:10.7916/cblr.v2019i2.3424 · cited 1×
“They may miss unfairness against people as a result of their being at the intersection of multiple kinds of discrimination [15], or groups which are not (yet) defined in anti-discrimination law but may need protecting [49].”
Background
Research works printed but not linked: 9
- 10.2139/ssrn.3452030 · doi:10.2139/ssrn.3452030
- Danielle Keats Citron. 2007. Technological due process. Wash. UL Rev. 85 (2007), 1249.
- Oscar Gandy. 1995. It's discrimination, stupid. Resisting the virtual life: The culture and politics of information (1995), 35--47.
- bell hooks. 2014. Ain't IA Woman: Black women and feminism. Routledge New York, NY.
- RenÄl'e Jorgensen Bolinger. 2019. What is the Right to Individualized Judgement? Working Paper (on file with author) (2019).
- Sarah Moss. 2018. Probabilistic knowledge. Oxford University Press.
- John Rawls. 2009. A theory of justice. Harvard university press.
- Amartya Sen. 1992. Inequality reexamined. Clarendon Press.
- Larry S Temkin. 1986. Inequality. Philosophy & Public Affairs (1986), 99--121.
Cited by
Why fairness cannot be automated: Bridging the gap between EU non-discrimination law and AI
Wachter, S., Mittelstadt, B., & Russell, C. (2021) · Computer Law & Security Review · doi:10.1016/j.clsr.2021.105567 · Cited by 336
A clarification of the nuances in the fairness metrics landscape
Castelnovo, A., Crupi, R., Greco, G., Regoli, D., Penco, I. G., & Cosentini, A. C. (2022) · Scientific Reports · doi:10.1038/s41598-022-07939-1 · Cited by 218
“…with gender, the issue weather it is fair to allow for a certain gender discrimination as long as justified by income can be seen as an instance of the conflict between individual or group notions, but it is rather a conflict about the underlying ethical and legal assumptions (Binns, 2020).”
BackgroundInfluentialFairness in recommender systems: research landscape and future directions
Deldjoo, Y., Jannach, D., Bellogin, A., Difonzo, A., & Zanzonelli, D. (2024) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-023-09364-z · Cited by 228
“As a result, group and individual fairness are frequently viewed as trade-offs, which is not always immediately evident (Binns 2020).”
Fairness in Information Access Systems
Ekstrand, M. D., Das, A., Burke, R., & Diaz, F. (2022) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000079 · Cited by 133
Algorithmic Impact Assessments and Accountability: The Co-construction of Impacts
Metcalf, J., Moss, E., Watkins, E. A., Singh, R., & Elish, M. C. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445935 · Cited by 195
“…methods have demonstrated the ability to describe the former (disparate classifica-tory probabilities) in any number of important ways [1, 12, 44, 110], computational methods are less well-suited to measure the ways algorithmic systems produce and distribute the risk that people and…”
Metadata
- Authors
- Binns, R.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Binns, Reuben
- ORCID
- Binns, Reuben
- Venue
- Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2020-01-27
- Pages
- 514-524
- OpenAlex
- W2996424229
- Cited by
- 312
- References
- 46
- Citations by year
- 2026: 26 · 2025: 59 · 2024: 61 · 2023: 62 · 2022: 45 · 2021: 41 · 2020: 17
- FWCI
- 42.79
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- EPSRC (N02334X/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Political Science and International Relations › Political Philosophy and Ethics
- Social Sciences › Law › Discrimination and Equality Law
- Keywords
- individual fairness, group fairness, algorithmic fairness, individual justice, legal theory, fairness metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.