CoChat
Sign Up
PreprintCited by 152Ethics and Social Impacts of AIPrivacy-Preserving Technologies in DataImbalanced Data Classification Techniques

Identifying and Correcting Label Bias in Machine Learning

Open access
Jiang, Heinrich, Nachum, Ofir·2019·arXiv
DOI 10.48550/arxiv.1901.04966arXiv 1901.04966
Open this paper in CoChat — free accountOpen PDF

Abstract

Datasets often contain biases which unfairly disadvantage certain groups, and classifiers trained on such datasets can inherit these biases. In this paper, we provide a mathematical formulation of how this bias can arise. We do so by assuming the existence of underlying, unknown, and unbiased labels which are overwritten by an agent who intends to provide accurate labels but may have biases against certain groups. Despite the fact that we only observe the biased labels, we are able to show that the bias may nevertheless be corrected by re-weighting the data points without changing the labels. We show, with theoretical guarantees, that training on the re-weighted dataset corresponds to training on the unobserved but unbiased labels, thus leading to an unbiased machine learning classifier. Our procedure is fast and robust and can be used with virtually any learning algorithm. We evaluate on a number of standard machine learning fairness datasets and a variety of fairness notions, finding that our method outperforms standard approaches in achieving fair classification.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1901.04966

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Fairness constraints are guaranteed to improve the properties of a predictor only under appropriate assumptions on the label noise.

    “Similarly to noise in the sensitive attribute, Blum and Stangl [11], Jiang and Nachum [59] show that fairness constraints are guaranteed to improve the properties of a predictor only under appropriate assumptions on the label noise.”

    From Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty (2021) · cites this paper as [59]

    Section 2.1 (p. 2) says that recovering unbiased labels requires an assumption about their relationship to observed biased labels. The Introduction (p. 1) reports a theoretical guarantee for the paper’s re-weighted training method under its model. It does not establish the broader claim that fairness constraints improve a predictor only under appropriate assumptions about label noise. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 32

  1. Alekh Agarwal, Alina Beygelzimer, Miroslav Dud´ ık, John Langford, and Hanna Wallach. A reductions approach to fair classification. arXiv preprint arXiv:1803.02453 , 2018. · arXiv
  2. Sivaraman Balakrishnan, Srivatsan Narayanan, Alessandro Rinaldo, Aarti Singh, and Larry Wasserman. Cluster trees on manifolds. In Advances in Neural Information Processing Systems , pages 2679-2687, 2013.
  3. Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H Chi. Data decisions and theoretical implications when adversarially learning fair representations. arXiv preprint arXiv:1707.00075 , 2017. · arXiv
  4. Zdravko I Botev and Dirk P Kroese. The generalized cross entropy method, with applications to probability density estimation. Methodology and Computing in Applied Probability , 13(1):1-27, 2011.
  5. Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 797-806. ACM, 2017.
  6. Andrew Cotter, Maya Gupta, Heinrich Jiang, Nathan Srebro, Karthik Sridharan, Serena Wang, Blake Woodworth, and Seungil You. Training well-generalizing classifiers for fairness metrics and other data-dependent constraints. arXiv preprint arXiv:1807.00028 , 2018a. · arXiv
  7. Andrew Cotter, Heinrich Jiang, and Karthik Sridharan. Twoplayer games for efficient non-convex constrained optimization. arXiv preprint arXiv:1804.06500 , 2018b. · arXiv
  8. Andrew Cotter, Heinrich Jiang, Serena Wang, Taman Narayan, Maya Gupta, Seungil You, and Karthik Sridharan. Optimization with non-differentiable constraints with applications to fairness, recall, churn, and other goals. arXiv preprint arXiv:1809.04198 , 2018c. · arXiv
  9. Neil A Doherty, Anastasia V Kartasheva, and Richard D Phillips. Information effect of entry into credit ratings market: The case of insurers' ratings. Journal of Financial Economics , 106(2):308-330, 2012.
  10. Michele Donini, Luca Oneto, Shai Ben-David, John ShaweTaylor, and Massimiliano Pontil. Empirical risk minimization under fairness constraints. arXiv preprint arXiv:1802.08626 , 2018. · arXiv
  11. Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference , pages 214-226. ACM, 2012.
  12. Elad Eban, Mariano Schain, Alan Mackey, Ariel Gordon, Ryan Rifkin, and Gal Elidan. Scalable learning of nondecomposable objectives. In Artificial Intelligence and Statistics , pages 832-840, 2017.
  13. Harrison Edwards and Amos Storkey. Censoring representations with an adversary. arXiv preprint arXiv:1511.05897 , 2015. · arXiv
  14. Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. Certifying and removing disparate impact. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 259-268. ACM, 2015.
  15. Benjamin Fish, Jeremy Kun, and Ad´ am D Lelkes. Fair boosting: a case study. In Workshop on Fairness, Accountability, and Transparency in Machine Learning . Citeseer, 2015.
  16. Michael P Friedlander and Maya R Gupta. On minimizing distortion and relative entropy. IEEE Transactions on Information Theory , 52(1):238-245, 2006.
  17. Gabriel Goh, Andrew Cotter, Maya Gupta, and Michael P Friedlander. Satisfying real-world goals with dataset constraints. In Advances in Neural Information Processing Systems , pages 2415-2423, 2016.
  18. Dominique Guegan and Bertrand Hassani. Regulatory learning: how to supervise machine learning models? an application to credit scoring. The Journal of Finance and Data Science , 2018.
  19. Abel Ag Rb Guimaraes and Ghassem Tofighi. Detecting zones and threat on 3d body in security airports using deep learning machine. arXiv preprint arXiv:1802.00565 , 2018. · arXiv
  20. Moritz Hardt, Eric Price, Nati Srebro, et al. Equality of opportunity in supervised learning. In Advances in neural information processing systems , pages 3315-3323, 2016.
  21. Heinrich Jiang. Density level set estimation on manifolds with dbscan. In International Conference on Machine Learning , pages 1684-1693, 2017.
  22. Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807 , 2016. · arXiv
  23. Junpei Komiyama, Akiko Takeda, Junya Honda, and Hajime Shimao. Nonconvex optimization for regression with fairness constraints. In International Conference on Machine Learning , pages 2742-2751, 2018.
  24. Emmanouil Krasanakis, Eleftherios Spyromitros-Xioufis, Symeon Papadopoulos, and Yiannis Kompatsiaris. Adaptive sensitive reweighting to mitigate bias in fairnessaware classification. In Proceedings of the 2018 World Wide Web Conference on World Wide Web , pages 853862. International World Wide Web Conferences Steering Committee, 2018.
  25. Harikrishna Narasimhan. Learning with complex loss functions and constraints. In International Conference on Artificial Intelligence and Statistics , pages 1646-1654, 2018.
  26. Fabian Pedregosa, Ga¨ el Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. Journal of machine learning research , 12(Oct):2825-2830, 2011.
  27. Dino Pedreshi, Salvatore Ruggieri, and Franco Turini. Discrimination-aware data mining. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining , pages 560-568. ACM, 2008.
  28. Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. On fairness and calibration. In Advances in Neural Information Processing Systems , pages 5680-5689, 2017.
  29. Blake Woodworth, Suriya Gunasekar, Mesrob I Ohannessian, and Nathan Srebro. Learning non-discriminatory predictors. In Conference on Learning Theory , pages 1920-1953, 2017.
  30. Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. Fairness constraints: Mechanisms for fair classification. arXiv preprint arXiv:1507.05259 , 2015. · arXiv
  31. Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In International Conference on Machine Learning , pages 325-333, 2013.
  32. Indre ˇ Zliobaite, Faisal Kamiran, and Toon Calders. Handling conditional discrimination. In Data Mining (ICDM), 2011 IEEE 11th International Conference on , pages 9921001. IEEE, 2011.

Cited by

Sorted by citations

Authors
Jiang, H., & Nachum, O.
Affiliations
Google (United States) (US)
Venue
arXiv
Publisher
arXiv
Published
2019
Pages
702-712
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1901.04966
arXiv
1901.04966
OpenAlex
W2909334458
Cited by
152
References
8
Citations by year
2026: 1 · 2025: 10 · 2024: 28 · 2023: 24 · 2022: 21 · 2021: 46 · 2020: 14 · 2019: 8
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • International Conference on Artificial Intelligence and Statistics
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
Keywords
label bias, algorithmic fairness, fair classification, classifier bias, fairness guarantees

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review