A Reductions Approach to Fair Classification
Open accessThe DOI matches its DataCite record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
We present a systematic approach for achieving fairness in a binary classification setting. While we focus on two well-known quantitative definitions of fairness, our approach encompasses many other previously studied definitions as special cases. The key idea is to reduce fair classification to a sequence of cost-sensitive classification problems, whose solutions yield a randomized classifier with the lowest (empirical) error subject to the desired constraints. We introduce two reductions that work for any representation of the cost-sensitive classifier and compare favorably to prior baselines on a variety of data sets, while overcoming several of their disadvantages.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Rockafellar, R. T. (1970) · Princeton University Press eBooks · doi:10.1515/9781400873173 · Cited by 10,801
Scikit-learn: Machine Learning in Python
(2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078195 · Cited by 8,539
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · cited 1×
“In response to this scrutiny—and driven by ongoing debates and collaborations with lawyers, policy-makers, social scientists, and others (e.g., Barocas & Selbst, 2016)—machine learning researchers have begun to turn their attention to the topic of “fairness in machine learning,” and, in particular,…”
BackgroundEquality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 6×
“The second group of approaches eliminate the restriction to specific classifier families and treat the underlying classification method as a “black box,” while implementing a wrapper that either works by pre-processing the data or post-processing the classifier’s predictions (e.g., Kamiran & Calders, 2012; Feldman et…”
BackgroundMethodInfluentialFair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 2×
“…2017), overall accuracy equality (Berk et al., 2017), and treatment equality (Berk et al., 2017) can all be expressed as equation (1); in contrast, calibration (Kleinberg et al., 2017) and predictive parity (Chouldechova, 2017) cannot because to do so would require the event E j to depend on h .”
BackgroundRademacher and Gaussian Complexities: Risk Bounds and Structural Results
Bartlett, P. L., & Mendelson, S. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44581-1_15 · Cited by 1,949
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Data preprocessing techniques for classification without discrimination
Kamiran, F., & Calders, T. (2012) · Knowledge and Information Systems · doi:10.1007/s10115-011-0463-8 · Cited by 1,305
Online Learning and Online Convex Optimization
Shalev-Shwartz, S. (2012) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000018 · Cited by 1,246
Ledoux, M., & Talagrand, M. (1991) · doi:10.1007/978-3-642-20212-4 · Cited by 1,160
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Online Learning and Online Convex Optimization
Shalev-Shwartz, S. (2011) · now publishers, Inc. eBooks · doi:10.1561/9781601985477 · Cited by 1,095
Fairness in Criminal Justice Risk Assessments: The State of the Art
Berk, R., Heidari, H., Jabbari, S., Kearns, M., & Roth, A. (2021) · Sociological Methods & Research · doi:10.1177/0049124118782533 · Cited by 988
Optimized Pre-Processing for Discrimination Prevention
Calmon, F. P., Wei, D., Vinzamuri, B., Ramamurthy, K. N., & Varshney, K. R. (2017) · neural information processing systems · Cited by 978
Inherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943 · cited 6×
“For example, equality of opportunity (Hardt et al., 2016) (also known as balance for the positive class ; Kleinberg et al., 2017), balance for the negative class (Klein-berg et al., 2017), error-rate balance (Chouldechova, 2017), overall accuracy equality (Berk et al., 2017), and treatment equality…”
BackgroundMethodInfluentialInherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941
Exponentiated Gradient versus Gradient Descent for Linear Predictors
Kivinen, J., & Warmuth, M. K. (1997) · Information and Computation · doi:10.1006/inco.1996.2612 · Cited by 864
AdaCost: Misclassification Cost-Sensitive Boosting
Fan, W., Stolfo, S. J., Zhang, J., & Chan, P. K. (1999) · Cited by 612
Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
Kearns, M., Neel, S., Roth, A., & Wu, Z. S. (2017) · arXiv · doi:10.48550/arxiv.1711.05144 · Cited by 469 · cited 1×
“A preliminary version of this paper appeared at the FAT/ML workshop (Agarwal et al., 2017), and led to extensions with more general optimization objectives (Alabi et al., 2018) and combinatorial protected attributes (Kearns et al., 2018).”
BackgroundTheory of Classification: a Survey of Some Recent Advances
Boucheron, S., Bousquet, O., & Lugosi, G. (2005) · ESAIM: Probability and Statistics · doi:10.1051/ps:2005018 · Cited by 436
Game theory, on-line prediction and boosting
Freund, Y., & Schapire, R. E. (1996) · Proceedings of the ninth annual conference on Computational learning theory - COLT '96 · doi:10.1145/238061.238163 · Cited by 383
Fairness-aware Learning through Regularization Approach
Kamishima, T., Akaho, S., & Sakuma, J. (2011) · 2011 IEEE 11th International Conference on Data Mining Workshops · doi:10.1109/icdmw.2011.83 · Cited by 371
LSAC National Longitudinal Bar Passage Study. LSAC Research Report Series.
Wightman, L. F. (1998) · Cited by 355
Decoupled classifiers for fair and efficient machine learningPreprint
Dwork, C., Immorlica, N., Kalai, A. T., & Leiserson, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.06613 · Cited by 285
The cost of fairness in binary classification
Menon, A. K., & Williamson, R. C. (2018) · Cited by 202
Learning Non-Discriminatory Predictors
Woodworth, B., Gunasekar, S., Ohannessian, M. I., & Srebro, N. (2017) · arXiv · doi:10.48550/arxiv.1702.06081 · Cited by 182
Error limiting reductions between classification tasks
Beygelzimer, A., Dani, V., Hayes, T., Langford, J. C., & Zadrozny, B. (2005) · International Conference on Machine Learning · doi:10.1145/1102351.1102358 · Cited by 88
Self-citationSensitive Error Correcting Output Codes
Langford, J. C., & Beygelzimer, A. (2005) · Lecture notes in computer science · doi:10.1007/11503415_11 · Cited by 85
Self-citationEmpirical Risk Minimization under Fairness Constraints
Donini, M., Oneto, L., Ben-David, S., Shawe-Taylor, J. S., & Pontil, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.08626 · Cited by 82
Fairness in Criminal Justice Risk Assessments: The State of the ArtPreprint
Berk, R. A., Heidari, H., Jabbari, S., Kearns, M. J., & Roth, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.09207 · Cited by 34
Unleashing Linear Optimizers for Group-Fair Learning and OptimizationPreprint
Alabi, D., Immorlica, N., & Kalai, A. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.04503 · Cited by 29
Impartial Predictive Modeling and the Use of Proxy Variables
Johnson, K. D., Foster, D. P., & Stine, R. A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-96957-8_26 · Cited by 26
Impartial Predictive Modeling: Ensuring Group Fairness in Arbitrary ModelsPreprint
Johnson, K. D., Foster, D. P., & Stine, R. A. (2016) · arXiv (Cornell University) · Cited by 20
Impartial Predictive Modeling: Ensuring Fairness in Arbitrary Models
Johnson, K. D., Foster, D. P., & Stine, R. A. (2016) · ScholarlyCommons (University of Pennsylvania) · Cited by 20
Impartial Predictive Modeling and the Use of Proxy VariablesPreprint
Johnson, K. D., Foster, D. P., & Stine, R. A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.00528
(2008) · ScholarlyCommons (University of Pennsylvania)
Research works printed but not linked: 1
- Zafar, M. B., Valera, I., Rodriguez, M. G., and Gummadi, K. P. Fairness constraints: Mechanisms for fair classifica- tion. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 962-970, 2017.
Cited by
Explainable AI: a review of machine learning interpretability methods
Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2020) · Entropy · doi:10.3390/e23010018 · Cited by 2,922
“Agarwal et al. [95] introduced a systematic framework that incorporates many other previously outlined definitions of fairness, treating them as special cases.”
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631
Bias in data‐driven artificial intelligence systems—An introductory survey
Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M., Ruggieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., et al. (2020) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1356 · Cited by 1,043
Fairness in Criminal Justice Risk Assessments: The State of the Art
Berk, R., Heidari, H., Jabbari, S., Kearns, M., & Roth, A. (2021) · Sociological Methods & Research · doi:10.1177/0049124118782533 · Cited by 988
Metadata
- Authors
- Agarwal, A., Beygelzimer, A., Dudík, M., Langford, J., & Wallach, H.
- Affiliations
- Microsoft (United States) (US)
- Yahoo (United Kingdom) (GB)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- Pages
- 60-69
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1803.02453
- OpenAlex
- W2790744245
- Cited by
- 1,288
- References
- 37
- Citations by year
- 2026: 165 · 2025: 186 · 2024: 185 · 2023: 210 · 2022: 168 · 2021: 154 · 2020: 121 · 2019: 80 · 2018: 18
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Conference on Machine Learning
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- fair classification, algorithmic fairness, reduction, cost-sensitive classification, binary classification, fairness constraints
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.