The DOI matches its DataCite record; 4 of the 4 references it cites were found in a registry or an index.
Recovering from Biased Data: Can Fairness Constraints Improve Accuracy?
Open accessAbstract
Multiple fairness constraints have been proposed in the literature, motivated by a range of concerns about how demographic groups might be treated unfairly by machine learning classifiers. In this work we consider a different motivation; learning from biased training data. We posit several ways in which training data may be biased, including having a more noisy or negatively biased labeling process on members of a disadvantaged group, or a decreased prevalence of positive or negative examples from the disadvantaged group, or both. Given such biased training data, Empirical Risk Minimization (ERM) may produce a classifier that not only is biased but also has suboptimal accuracy on the true data distribution. We examine the ability of fairness-constrained ERM to correct this problem. In particular, we find that the Equal Opportunity fairness constraint (Hardt, Price, and Srebro 2016) combined with ERM will provably recover the Bayes Optimal Classifier under a range of bias models. We also consider other recovery methods including reweighting the training data, Equalized Odds, and Demographic Parity. These theoretical results provide additional motivation for considering fairness interventions even if an actor cares primarily about accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Fairness constraints are guaranteed to improve the properties of a predictor only under appropriate assumptions on the label noise.
“Similarly to noise in the sensitive attribute, Blum and Stangl [11], Jiang and Nachum [59] show that fairness constraints are guaranteed to improve the properties of a predictor only under appropriate assumptions on the label noise.”
From Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty (2021) · cites this paper as [11]
Section 3.3 (p. 11) says Equal Opportunity-constrained ERM recovers the Bayes-optimal classifier when Group A has sufficient mass and the signal is not too noisy. Section 3.1 (p. 9) also gives a recovery range bounded by the noise parameter η. The claim captures the dependence on noise assumptions, but overgeneralizes the guarantee to fairness constraints generally and omits the paper’s other conditions. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615
Inherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943
The Selective Labels Problem: Evaluating Algorithmic Predictions in the Presence of Unobservables
Lakkaraju, H., Kleinberg, J., Leskovec, J., Ludwig, J., & Mullainathan, S. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098066 · Cited by 138
Flagged first, then by citations
Research works printed but not linked: 15
- Dana Angluin and Philip Laird. Learning From Noisy Examples. Machine Learning , 2(4):343-370, Apr 1988. ISSN 1573-0565. doi: 10.1007/BF00116829. URL https://doi.org/10.1007/ BF00116829 . · DOI
- Marianne Bertrand and Sendhil Mullainathan. Are Emily and Greg More Employable than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination. American Economic Review , 94 (4):991-1013, 2004.
- Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Conference on Fairness, Accountability and Transparency , pages 77-91, 2018.
- Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 797-806. ACM, 2017.
- Maria De-Arteaga, Artur Dubrawski, and Alexandra Chouldechova. Learning under selective labels in the presence of expert consistency. arXiv preprint arXiv:1807.00905 , 2018. · arXiv
- Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard S. Zemel. Fairness Through Awareness. In Innovations in Theoretical Computer Science 2012, Cambridge, MA, USA, January 8-10, 2012 , pages 214-226, 2012. doi: 10.1145/2090236.2090255. URL https: //doi.org/10.1145/2090236.2090255 . · DOI · Paper page
- Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. On the (im)possibility of fairness. CoRR , abs/1609.07236, 2016. URL http://arxiv.org/abs/1609.07236 . · arXiv
- Moritz Hardt, Eric Price, and Nati Srebro. Equality of Opportunity in Supervised Learning. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29 , pages 3315-3323. Curran Associates, Inc., 2016. URL http://papers.nips.cc/paper/ 6374-equality-of-opportunity-in-supervised-learning.pdf .
- Heinrich Jiang and Ofir Nachum. Identifying and Correcting Label Bias in Machine Learning. CoRR , abs/1901.04966, 2019. URL http://arxiv.org/abs/1901.04966 . · arXiv
- Jon M. Kleinberg and Manish Raghavan. Selection Problems in the Presence of Implicit Bias. In 9th Innovations in Theoretical Computer Science Conference, ITCS 2018, January 11-14, 2018, Cambridge, MA, USA , pages 33:1-33:17, 2018. doi: 10.4230/LIPIcs.ITCS.2018.33. URL https://doi.org/10.4230/LIPIcs.ITCS.2018.33 . · DOI
- Kristian Lum and William Isaac. To predict and serve? Significance , 13(5):14-19, 2016.
- Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. On Fairness and Calibration. In Advances in Neural Information Processing Systems , pages 5680-5689, 2017.
- Rashida Richardson, Jason Schultz, and Kate Crawford. Dirty Data, Bad Predictions: How Civil Rights Violations Impact Police Data, Predictive Policing Systems, and Justice. New York University Law Review Online, Forthcoming , 2019.
- Camelia Simoiu, Sam Corbett-Davies, Sharad Goel, et al. The problem of infra-marginality in outcome tests for discrimination. The Annals of Applied Statistics , 11(3):1193-1216, 2017.
- Samuel Yeom and Michael Carl Tschantz. Discriminative but Not Discriminatory: A Comparison of Fairness Definitions under Different Worldviews. arXiv preprint arXiv:1808.08619 , 2018. · arXiv
Cited by
Sorted by citations
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Bhatt, U., Antorán, J., Zhang, Y., Liao, Q. V., Sattigeri, P., Fogliato, R., Melançon, G., Krishnan, R., Stanley, J., Tickoo, O., et al. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462571 · Cited by 300
Metadata
- Authors
- Blum, A., & Stangl, K.
- Affiliations
- Toyota Technological Institute at Chicago (US)
- ORCID
- Blum, Avrim
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1912.01094
- OpenAlex
- W2994007361
- Cited by
- 13
- References
- 4
- Citations by year
- 2024: 2 · 2023: 1 · 2022: 3 · 2021: 6 · 2020: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- fairness constraints, biased training data, empirical risk minimization, equal opportunities, equalized odds, demographic parity, Bayes optimal classifier, label noise, label bias, class imbalance, fairness-accuracy trade-off
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
