CoChat
Sign Up
PreprintCited by 23Ethics and Social Impacts of AICriminal Justice and Corrections AnalysisArtificial Intelligence in Law

Fairness Evaluation in Presence of Biased Noisy Labels

Open access
Fogliato, Riccardo, G'Sell, Max, Chouldechova, Alexandra·2020·arXiv
DOI 10.48550/arxiv.2003.13808arXiv 2003.13808
Open this paper in CoChat — free accountOpen PDF

Abstract

Risk assessment tools are widely used around the country to inform decision making within the criminal justice system. Recently, considerable attention has been devoted to the question of whether such tools may suffer from racial bias. In this type of assessment, a fundamental issue is that the training and evaluation of the model is based on a variable (arrest) that may represent a noisy version of an unobserved outcome of more central interest (offense). We propose a sensitivity analysis framework for assessing how assumptions on the noise across groups affect the predictive bias properties of the risk assessment model as a predictor of reoffense. Our experimental results on two real world criminal justice data sets demonstrate how even small biases in the observed labels may call into question the conclusions of an analysis based on the noisy outcome.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2003.13808

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 40

  1. Jessa Bekker and Jesse Davis. Learning from positive and unlabeled data under the selected at random assumption. arXiv preprint arXiv:1808.08755 , 2018. · arXiv
  2. Jakramate Bootkrajang and Jeerayut Chaijaruwanich. Towards instance-dependent label noise-tolerant classification: a probabilistic approach. Pattern Analysis and Applications , pages 1-17, 2018.
  3. Stephen Boyd and Lieven Vandenberghe. Convex optimization . Cambridge university press, 2004.
  4. Timothy I Cannings, Yingying Fan, and Richard J Samworth. Classification with imperfect training labels. arXiv preprint arXiv:1805.11505 , 2018. · arXiv
  5. Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining , pages 785-794, 2016.
  6. Marc Claesen, Jesse Davis, Frank De Smet, and Bart De Moor. Assessing binary classifiers using only positive and unlabeled data. arXiv preprint arXiv:1504.06837 , 2015. · arXiv
  7. St´ ephan Cl´ emen¸ con, G´ abor Lugosi, Nicolas Vayatis, et al. Ranking and empirical minimization of ustatistics. The Annals of Statistics , 36(2):844-874, 2008.
  8. Sam Corbett-Davies and Sharad Goel. The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023 , 2018. · arXiv
  9. Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , KDD '17, pages 797-806, New York, NY, USA, 2017. ACM. ISBN 978-1-4503-4887-4. doi: 10.1145/ 3097983.3098095. URL http://doi.acm.org/10.1145/ 3097983.3098095 .
  10. Michele Donini, Luca Oneto, Shai Ben-David, John S Shawe-Taylor, and Massimiliano Pontil. Empirical risk minimization under fairness constraints. In Advances in Neural Information Processing Systems , pages 2791-2801, 2018.
  11. Marthinus C Du Plessis, Gang Niu, and Masashi Sugiyama. Analysis of learning from positive and unlabeled data. In Advances in neural information processing systems , pages 703-711, 2014.
  12. Charles Elkan and Keith Noto. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining , pages 213-220. ACM, 2008.
  13. Amanda B Gilman, Karl G Hill, BK Elizabeth Kim, Alyssa Nevell, J David Hawkins, and David P Farrington. Understanding the relationship between self-reported offending and official criminal charges across early adulthood. Criminal behaviour and mental health , 24(4):229-240, 2014.
  14. Y Yi Grace. Statistical Analysis with Measurement Error Or Misclassification . Springer, 2016.
  15. Moritz Hardt, Eric Price, Nati Srebro, et al. Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems , pages 3315-3323, 2016.
  16. Fengxiang He, Tongliang Liu, Geoffrey I Webb, and Dacheng Tao. Instance-dependent pu learning by bayesian optimal relabeling. arXiv preprint arXiv:1808.02180 , 2018. · arXiv
  17. James J Heckman. Sample selection bias as a specification error. Econometrica: Journal of the econometric society , pages 153-161, 1979.
  18. Michael J Hindelang, Travis Hirschi, and Joseph G Weis. Correlates of delinquency: The illusion of discrepancy between self-report and official measures. American sociological review , pages 995-1014, 1979.
  19. David Huizinga and Delbert S Elliott. Reassessing the reliability and validity of self-report delinquency measures. Journal of quantitative criminology , 2(4): 293-327, 1986.
  20. Kosuke Imai and Teppei Yamamoto. Causal inference with differential measurement error: Nonparametric identification and sensitivity analysis. American Journal of Political Science , 54(2):543-560, 2010.
  21. Shantanu Jain, Martha White, and Predrag Radivojac. Recovering true classifier performance in positive-unlabeled learning. In Thirty-First AAAI Conference on Artificial Intelligence , 2017.
  22. Nathan Kallus and Angela Zhou. Residual unfairness in fair machine learning from prejudiced data. arXiv preprint arXiv:1806.02887 , 2018. · arXiv
  23. Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. In Advances in Neural Information Processing Systems , pages 4066-4076, 2017.
  24. Roderick JA Little and Donald B Rubin. Statistical analysis with missing data , volume 793. Wiley, 2019.
  25. Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting. IEEE Transactions on pattern analysis and machine intelligence , 38(3):447-461, 2016.
  26. Joshua R Loftus, Chris Russell, Matt J Kusner, and Ricardo Silva. Causal reasoning for algorithmic fairness. arXiv preprint arXiv:1805.05859 , 2018. · arXiv
  27. Aditya Menon, Brendan Van Rooyen, Cheng Soon Ong, and Bob Williamson. Learning from corrupted binary labels via class-probability estimation. In International Conference on Machine Learning , pages 125-134, 2015.
  28. Aditya Krishna Menon, Brendan Van Rooyen, and Nagarajan Natarajan. Learning from binary labels with instance-dependent corruption. arXiv preprint arXiv:1605.00751 , 2016. · arXiv
  29. Razieh Nabi and Ilya Shpitser. Fair inference on outcomes. In Proceedings of the... AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence , volume 2018, page 1931. NIH Public Access, 2018.
  30. Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. Learning with noisy labels. In Advances in neural information processing systems , pages 1196-1204, 2013.
  31. Alex R Piquero. Understanding race/ethnicity differences in offending across the life course: Gaps and opportunities. Journal of developmental and lifecourse criminology , 1(1):21-32, 2015.
  32. Alex R Piquero and Robert W Brame. Assessing the race-crime and ethnicity-crime relationship in a sample of serious adolescent delinquents. Crime & Delinquency , 54(3):390-422, 2008.
  33. James M Robins, Andrea Rotnitzky, and Daniel O Scharfstein. Sensitivity analysis for selection bias and unmeasured confounding in missing data and causal inference models. In Statistical models in epidemiology, the environment, and clinical trials , pages 1-94. Springer, 2000.
  34. Paul R Rosenbaum. Sensitivity analysis in observational studies. Wiley StatsRef: Statistics Reference Online , 2014.
  35. Donald B Rubin. Inference and missing data. Biometrika , 63(3):581-592, 1976.
  36. Clayton Scott. A rate of convergence for mixture proportion estimation, with application to learning from noisy labels. In Artificial Intelligence and Statistics , pages 838-846, 2015.
  37. Clayton Scott. A generalized neyman-pearson criterion for optimal domain adaptation. arXiv preprint arXiv:1810.01545 , 2018. · arXiv
  38. Clayton Scott and Gilles Blanchard. Novelty detection: Unlabeled data definitely help. In Artificial Intelligence and Statistics , pages 464-471, 2009.
  39. Clayton Scott, Gilles Blanchard, and Gregory Handy. Classification with asymmetric label noise: Consistency and maximal denoising. In Conference On Learning Theory , pages 489-511, 2013.
  40. Jennifer L Skeem and Christopher T Lowenkamp. Risk, race, & recidivism: Predictive bias and disparate impact. Available at SSRN , 2015.

Cited by

Sorted by citations

Authors
Fogliato, R., G'Sell, M., & Chouldechova, A.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
Pages
2325-2336
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2003.13808
arXiv
2003.13808
OpenAlex
W3014986830
Cited by
23
References
5
Citations by year
2026: 1 · 2025: 1 · 2024: 6 · 2023: 2 · 2022: 2 · 2021: 10 · 2020: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • International Conference on Artificial Intelligence and Statistics
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Sociology and Political Science › Criminal Justice and Corrections Analysis
  • Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
Keywords
algorithmic fairness, risk assessment tools, criminal justice, racial bias, sensitivity analysis, arrest data, model evaluation, predictive bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review