Fairness Under Composition
Open accessAbstract
Algorithmic fairness, and in particular the fairness of scoring and classification algorithms, has become a topic of increasing social concern and has recently witnessed an explosion of research in theoretical computer science, machine learning, statistics, the social sciences, and law. Much of the literature considers the case of a single classifier (or scoring function) used once, in isolation. In this work, we initiate the study of the fairness properties of systems composed of algorithms that are fair in isolation; that is, we study fairness under composition. We identify pitfalls of naïve composition and give general constructions for fair composition, demonstrating both that classifiers that are fair in isolation do not necessarily compose into fair systems and also that seemingly unfair components may be carefully combined to construct fair systems. We focus primarily on the individual fairness setting proposed in [Dwork, Hardt, Pitassi, Reingold, Zemel, 2011], but also extend our results to a large class of group fairness definitions popular in the recent literature, exhibiting several cases in which group fairness definitions give misleading signals under composition.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Fairness is not necessarily composable: a collection of classifiers that are fair in isolation do not necessarily result in a fair outcome when used as part of a larger system.
“Furthermore, recent work has demonstrated that fairness is not necessarily composable, i.e., for several notions of fairness such as individual fairness [24], a collection of classifiers that are fair in isolation do not necessarily result in a fair outcome when they are used as part of a larger system [25].”
From Discrimination through Optimization (2019) · cites this paper as [25]
Skewed ad delivery can occur even if the ad platform refrains from refining the audience supplied by the advertisers according to the predicted relevance of the ad to individual users.
“skewed ad delivery can occur even if the ad platform refrains from refining the audience supplied by the advertisers according to the predicted relevance of the ad to individual users.”
From Discrimination through Optimization (2019) · cites this paper as [25]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
Cited by 91
Research works printed but not linked: 23
- [1][1] Amanda Bower, Sarah N. Kitchen, Laura Niss, Martin J. Strauss, Alexander Vargas, and Suresh Venkatasubramanian. 'Fair Pipelines'. In: CoRR abs/1707.00391 (2017). arXiv: 1707. 00391 . url : http://arxiv.org/abs/1707.00391 . - · arXiv
- [2][2] Alexandra Chouldechova. 'Fair prediction with disparate impact: A study of bias in recidivism prediction instruments'. In: arXiv preprint arXiv:1703.00056 (2017). - · arXiv
- [3][3] Amit Datta, Michael Carl Tschantz, and Anupam Datta. 'Automated experiments on ad privacy settings'. In: Proceedings on Privacy Enhancing Technologies 2015.1 (2015), pp. 92112. -
- [4][4] Cynthia Dwork. 'Differential privacy'. In: Encyclopedia of Cryptography and Security . Springer, 2011, pp. 338-340. -
- [5][5] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 'Fairness through awareness'. In: Proceedings of the 3rd innovations in theoretical computer science conference . ACM. 2012, pp. 214-226. -
- [6][6] Cynthia Dwork, Aaron Roth, et al. 'The algorithmic foundations of differential privacy'. In: Foundations and Trends R © in Theoretical Computer Science 9.3-4 (2014), pp. 211-407. -
- [7][7] Stephen Gillen, Christopher Jung, Michael Kearns, and Aaron Roth. 'Online Learning with an Unknown Fairness Metric'. In: arXiv preprint arXiv:1802.06936 (2018). - · arXiv
- [8][8] Moritz Hardt, Eric Price, Nati Srebro, et al. 'Equality of opportunity in supervised learning'. In: Advances in Neural Information Processing Systems . 2016, pp. 3315-3323. -
- [9][9] Ursula H´ ebert-Johnson, Michael P Kim, Omer Reingold, and Guy N Rothblum. 'Calibration for the (Computationally-Identifiable) Masses'. In: arXiv preprint arXiv:1711.08513 (2017). - · arXiv
- [10][10] Lily Hu and Yiling Chen. 'Fairness at Equilibrium in the Labor Market'. In: CoRR abs/1707.01590 (2017). arXiv: 1707.01590 . url : http://arxiv.org/abs/1707.01590 . - · arXiv
- [11][11] Faisal Kamiran and Toon Calders. 'Classifying without discriminating'. In: Computer, Control and Communication, 2009. IC4 2009. 2nd International Conference on . IEEE. 2009, pp. 1-6. -
- [12][12] Toshihiro Kamishima, Shotaro Akaho, and Jun Sakuma. 'Fairness-aware learning through regularization approach'. In: Data Mining Workshops (ICDMW), 2011 IEEE 11th International Conference on . IEEE. 2011, pp. 643-650. -
- [13][13] Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. 'Preventing fairness gerrymandering: Auditing and learning for subgroup fairness'. In: arXiv preprint arXiv:1711.05144 (2017). - · arXiv
- [14][14] Niki Kilbertus, Mateo Rojas-Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Sch¨ olkopf. 'Avoiding Discrimination through Causal Reasoning'. In: arXiv preprint arXiv:1706.02744 (2017). - · arXiv
- [15][15] Michael P Kim, Omer Reingold, and Guy N Rothblum. 'Fairness Through ComputationallyBounded Awareness'. In: arXiv preprint arXiv:1803.03239 (2018). - · arXiv
- [16][16] Jon M. Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 'Inherent Trade-Offs in the Fair Determination of Risk Scores'. In: CoRR abs/1609.05807 (2016). url : http://arxiv. org/abs/1609.05807 . - · arXiv
- [17][17] Peter Kuhn and Kailing Shen. 'Gender discrimination in job ads: Evidence from china'. In: The Quarterly Journal of Economics 128.1 (2012), pp. 287-336. -
- [18][18] Matt J Kusner, Joshua R Loftus, Chris Russell, and Ricardo Silva. 'Counterfactual Fairness'. In: arXiv preprint arXiv:1703.06856 (2017). - · arXiv
- [20][20] Lydia T Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. 'Delayed Impact of Fair Machine Learning'. In: arXiv preprint arXiv:1803.04383 (2018). - · arXiv
- [21][21] David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. 'Learning Adversarially Fair and Transferable Representations'. In: arXiv preprint arXiv:1802.06309 (2018). - · arXiv
- [22][22] Dino Pedreshi, Salvatore Ruggieri, and Franco Turini. 'Discrimination-aware data mining'. In: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining . ACM. 2008, pp. 560-568. -
- [23][23] Ya'acov Ritov, Yuekai Sun, and Ruofei Zhao. 'On conditional parity as a notion of nondiscrimination in machine learning'. In: arXiv preprint arXiv:1706.08519 (2017). - · arXiv
- [24][24] Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. 'Learning fair representations'. In: Proceedings of the 30th International Conference on Machine Learning (ICML-13) . 2013, pp. 325-333.
- Authors
- Dwork, C., & Ilvento, C.
- Affiliations
- Harvard University (US)
- ORCID
- Dwork, Cynthia
- Venue
- Schloss Dagstuhl – Leibniz-Zentrum für Informatik
- Publisher
- Schloss Dagstuhl – Leibniz-Zentrum für Informatik
- Published
- 2019
- Pages
- 20
- ISSN
- 2331-8422 (ISSN-L)
- OpenAlex
- W2962922829
- Cited by
- 91
- References
- 0
- Citations by year
- 2026: 2 · 2025: 7 · 2024: 4 · 2023: 9 · 2022: 10 · 2021: 22 · 2020: 17 · 2019: 18 · 2018: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Leibniz international proceedings in informatics (cc-by)
- Funders
- Alfred P. Sloan Foundation
- Microsoft Research
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Keywords
- algorithmic fairness, individual fairness, group fairness, scoring functions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
