An algorithm for removing sensitive information: Application to race-independent recidivism prediction
Open accessAbstract
Predictive modeling is increasingly being employed to assist human decision-makers. One purported advantage of replacing or augmenting human judgment with computer models in high stakes settings—such as sentencing, hiring, policing, college admissions, and parole decisions—is the perceived “neutrality” of computers. It is argued that because computer models do not hold personal prejudice, the predictions they produce will be equally free from prejudice. There is growing recognition that employing algorithms does not remove the potential for bias, and can even amplify it if the training data were generated by a process that is itself biased. In this paper, we provide a probabilistic notion of algorithmic bias. We propose a method to eliminate bias from predictive models by removing all information regarding protected variables from the data to which the models will ultimately be trained. Unlike previous work in this area, our procedure accommodates data on any measurement scale. Motivated by models currently in use in the criminal justice system that inform decisions on pre-trial release and parole, we apply our proposed method to a dataset on the criminal histories of individuals at the time of sentencing to produce “race-neutral” predictions of re-arrest. In the process, we demonstrate that a common approach to creating “race-neutral” models—omitting race as a covariate—still results in racially disparate predictions. We then demonstrate that the application of our proposed method to these data removes racial disparities from predictions with minimal impact on predictive accuracy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Applications in perturbation-based data repair have shown that accuracy is not significantly affected.
“Applications in perturbationbased data repair [98,125,136,152,154,172] have shown that accuracy is not signiicantly afected.”
From Fairness in machine learning: A survey (2024) · cites this paper as [154]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701 · see citing works
<b>mice</b>: Multivariate Imputation by Chained Equations in<i>R</i>
Buuren, S. V., & Groothuis-Oudshoorn, K. (2011) · Journal of Statistical Software · doi:10.18637/jss.v045.i03 · Cited by 13,870 · see citing works
Multiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792 · see citing works
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · see citing works
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · see citing works
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · see citing works
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · see citing works
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · see citing works
Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995 · see citing works
Inherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943 · see citing works
Inherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941 · see citing works
Consumer credit-risk models via machine-learning algorithms
Khandani, A. E., Kim, A. J., & Lo, A. W. (2010) · Journal of Banking & Finance · doi:10.1016/j.jbankfin.2010.06.001 · Cited by 864 · see citing works
Three naive Bayes approaches for discrimination-free classification.
Calders, T., & Verwer, S. (2010) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-010-0190-x · Cited by 764 · see citing works
Violent offenders: Appraising and managing risk, 2nd ed.
Quinsey, V., Harris, G., Rice, M., & Cormier, C. A. (2006) · American Psychological Association eBooks · doi:10.1037/11367-000 · Cited by 720 · see citing works
Zafar, M. B., Valera, I., Gomez Rodriguez, M., & Gummadi, K. P. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052660 · Cited by 632 · see citing works
The Phantom Menace: Omitted Variable Bias in Econometric Research
Clarke, K. A. (2005) · Conflict Management and Peace Science · doi:10.1080/07388940500339183 · Cited by 484 · cited 1× · see citing works
“In the case of regression models, this is known as omitted variable bias [Clarke, 2005].”
MethodClassifying without discriminating
Kamiran, F., & Calders, T. (2009) · 2009 2nd International Conference on Computer, Control and Communication · doi:10.1109/ic4.2009.4909197 · Cited by 409 · see citing works
Inherent Tradeoffs in the Fair Determination of Risk Scores
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603197 · Cited by 347 · see citing works
Evaluating the Predictive Validity of the Compas Risk and Needs Assessment System
Brennan, T., Dieterich, W. A., & Ehret, B. (2008) · Criminal Justice and Behavior · doi:10.1177/0093854808326545 · Cited by 341 · see citing works
A multidisciplinary survey on discrimination analysis
Romei, A., & Ruggieri, S. (2014) · The Knowledge Engineering Review · doi:10.1017/s0269888913000039 · Cited by 288 · see citing works
Auditing black-box models for indirect influence
Adler, P., Falk, C., Friedler, S. A., Nix, T., Rybeck, G., Scheidegger, C., Smith, B., & Venkatasubramanian, S. (2018) · Knowledge and Information Systems · doi:10.1007/s10115-017-1116-3 · Cited by 280 · see citing works
A Note on Asymptotic Joint Normality
Mallows, C. L. (1972) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177692631 · Cited by 258 · see citing works
Hoffman, M., Kahn, L. B., & Li, D. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx042 · Cited by 252 · see citing works
The Multiple Adaptations of Multiple Imputation
Reiter, J. P., & Raghunathan, T. E. (2007) · Journal of the American Statistical Association · doi:10.1198/016214507000000932 · Cited by 245 · see citing works
Flores, A. W., Bechtel, K., & Lowenkamp, C. T. (2016) · Federal probation · Cited by 244 · see citing works
Examining Racial Disparities in Drug Arrests
Mitchell, O., & Caudy, M. S. (2013) · Justice Quarterly · doi:10.1080/07418825.2012.761721 · Cited by 237 · see citing works
Releasing Multiply Imputed, Synthetic Public use Microdata: An Illustration and Empirical Study
Reiter, J. P. (2004) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2004.00343.x · Cited by 233 · see citing works
Interpretable classification models for recidivism prediction
Zeng, J., Ustun, B., & Rudin, C. (2017) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/rssa.12227 · Cited by 220 · see citing works
Law, Social Standing and Racial Disparities in Imprisonment
Bridges, G. S., & Crutchfield, R. D. (1988) · Social Forces · doi:10.1093/sf/66.3.699 · Cited by 184 · see citing works
Berk, R., Sherman, L., Barnes, G., Kurtz, E., & Ahlman, L. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00556.x · Cited by 178 · see citing works
Risk assessment and release decision-making: toward resolving the great debate.
Dvoskin, J. A., & Heilbrun, K. (2001) · PubMed · Cited by 113 · see citing works
Sugli estremi dei momenti delle funzioni di ripartizione doppia
Dall’Aglio, G. (1956) · French digital mathematics library (Numdam) · Cited by 84 · see citing works
Actuarial Models for Assessing Prison Violence Risk
Cunningham, M. D., & Sorensen, J. R. (2006) · Assessment · doi:10.1177/1073191106287791 · Cited by 76 · see citing works
Alpert, G. P., Smith, M. R., & Dunham, R. G. (2004) · Justice Research and Policy · doi:10.3818/jrp.6.1.2004.43 · Cited by 66 · see citing works
Hoffman, M., Kahn, L., & Li, D. (2015) · National Bureau of Economic Research · doi:10.3386/w21709 · Cited by 57 · see citing works
Glaser, J. (2014) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195370409.001.0001 · Cited by 57 · see citing works
Rudovsky, D. (2001) · Penn Carey Law Legal Scholarship Repository (University of Pennsylvania) · Cited by 33 · see citing works
Testing for Racial Discrimination in Police Searches of Motor VehiclesPreprint
Simoiu, C., Corbett‐Davies, S., & Goel, S. (2016) · arXiv (Cornell University) · Cited by 8 · see citing works
The Problem of Infra-Marginality in Outcome Tests for DiscriminationPreprint
Simoiu, C., Corbett‐Davies, S., & Goel, S. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2811449 · Cited by 6 · see citing works
Transportation Distance and the Central Limit TheoremPreprint
Ekisheva, S., & Houdré, C. (2006) · arXiv (Cornell University) · doi:10.48550/arxiv.math/0607089 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Fairness in Criminal Justice Risk Assessments: The State of the Art
Berk, R., Heidari, H., Jabbari, S., Kearns, M., & Roth, A. (2021) · Sociological Methods & Research · doi:10.1177/0049124118782533 · Cited by 988 · see citing works
Algorithmic Fairness: Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-042720-125902 · Cited by 687 · see citing works
Self-citationFairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646 · see citing works
Towards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548 · see citing works
The Frontiers of Fairness in Machine LearningPreprint
Chouldechova, A., & Roth, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.08810 · Cited by 372 · see citing works
A snapshot of the frontiers of fairness in machine learning
Chouldechova, A., & Roth, A. (2020) · Communications of the ACM · doi:10.1145/3376898 · Cited by 362 · see citing works
Oneto, L., & Chiappa, S. (2020) · Studies in computational intelligence · doi:10.1007/978-3-030-43883-8_7 · Cited by 315 · see citing works
A clarification of the nuances in the fairness metrics landscape
Castelnovo, A., Crupi, R., Greco, G., Regoli, D., Penco, I., & Cosentini, A. (2021) · Scientific Reports · doi:10.1038/s41598-022-07939-1 · Cited by 283 · see citing works
Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey
Hort, M., Chen, Z., Zhang, J. M., Harman, M., & Sarro, F. (2023) · ACM Journal on Responsible Computing · doi:10.1145/3631326 · Cited by 248 · see citing works
Lessons from archives: strategies for collecting sociocultural data in machine learning
Jo, E. S., & Gebru, T. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372829 · Cited by 240 · see citing works
Fairness in credit scoring: Assessment, implementation and profit implications
Kozodoi, N., Jacob, J., & Lessmann, S. (2022) · European Journal of Operational Research · doi:10.1016/j.ejor.2021.06.023 · Cited by 198 · see citing works
A sociotechnical view of algorithmic fairness
Dolata, M., Feuerriegel, S., & Schwabe, G. (2021) · Information Systems Journal · doi:10.1111/isj.12370 · Cited by 177 · see citing works
Predictive Policing and the Politics of Patterns
Kaufmann, M., Egbert, S., & Leese, M. (2018) · The British Journal of Criminology · doi:10.1093/bjc/azy060 · Cited by 170 · see citing works
Achieving Fairness through Adversarial Learning: an Application to Recidivism PredictionPreprint
Wadsworth, C., Vera, F., & Piech, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.00199 · Cited by 139 · see citing works
Does Object Recognition Work for Everyone?Preprint
DeVries, T., Misra, I., Wang, C., & van der Maaten, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02659 · Cited by 132 · see citing works
Fairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646 · see citing works
Does mitigating ML's impact disparity require treatment disparity?Preprint
Lipton, Z. C., Chouldechova, A., & McAuley, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.07076 · Cited by 125 · see citing works
Algorithmic Political Bias in Artificial Intelligence Systems
Peters, U. (2022) · Philosophy & Technology · doi:10.1007/s13347-022-00512-8 · Cited by 111 · see citing works
Steimers, A., & Schneider, M. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19063641 · Cited by 97 · see citing works
Obtaining Fairness using Optimal Transport TheoryPreprint
del Barrio, E., Gamboa, F., Gordaliza, P., & Loubes, J.-M. (2018) · International Conference on Machine Learning · doi:10.48550/arxiv.1806.03195 · Cited by 223 · see citing works
Fairness in Algorithmic Decision Making: An Excursion Through the Lens of Causality
Khademi, A., Lee, S., Foley, D., & Honavar, V. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313559 · Cited by 87 · see citing works
On the Applicability of Machine Learning Fairness Notions
Makhlouf, K., Zhioua, S., & Catuscia Palamidessi, A. (2021) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3468507.3468511 · Cited by 83 · see citing works
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. (2021) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3465416.3483305 · Cited by 82 · see citing works
Bursting the Filter Bubble: Fairness-Aware Network Link Prediction
Masrour, F., Wilson, T., Yan, H., Tan, P., & Esfahanian, A. H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5429 · Cited by 82 · see citing works
Inherent Tradeoffs in Learning Fair RepresentationsPreprint
Zhao, H., & Gordon, G. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.08386 · Cited by 82 · see citing works
Wasserstein Fair ClassificationPreprint
Jiang, R., Pacchiano, A., Stepleton, T., Jiang, H., & Chiappa, S. (2019) · Conference on Uncertainty in Artificial Intelligence · doi:10.48550/arxiv.1907.12059 · Cited by 225 · see citing works
Machine learning fairness notions: Bridging the gap with real-world applications
Makhlouf, K., Zhioua, S., & Catuscia Palamidessi, A. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102642 · Cited by 76 · see citing works
Risk Prediction of Theft Crimes in Urban Communities: An Integrated Model of LSTM and ST-GCN
Han, X., Hu, X., Wu, H., Shen, B., & Wu, J. (2020) · IEEE Access · doi:10.1109/access.2020.3041924 · Cited by 74 · see citing works
Fairness risk measuresPreprint
Williamson, R. C., & Menon, A. K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08665 · Cited by 64 · see citing works
Social impacts of algorithmic decision-making: A research agenda for the social sciences
Gerdon, F., Bach, R. L., Kern, C. A., & Kreuter, F. (2022) · Big Data & Society · doi:10.1177/20539517221089305 · Cited by 61 · see citing works
Algorithmic Risk Assessment in the Hands of Humans
Stevenson, M., & Doleac, J. L. (2019) · doi:10.2139/ssrn.3489440 · Cited by 60 · see citing works
Matching Code and Law: Achieving Algorithmic Fairness with Optimal Transport
Zehlike, M., Hacker, P., & Wiedemann, E. (2019) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-019-00658-8 · Cited by 58 · see citing works
Fair regression for health care spending
Zink, A., & Rose, S. (2019) · Biometrics · doi:10.1111/biom.13206 · Cited by 57 · see citing works
Mitigating Bias in Algorithmic Systems—A Fish-eye View
Orphanou, K., Otterbacher, J., Kleanthous, S., Batsuren, K., Giunchiglia, F., Bogina, V., Tal, A. S., Hartman, A., & Kuflik, T. (2022) · ACM Computing Surveys · doi:10.1145/3527152 · Cited by 54 · see citing works
Conditional Learning of Fair Representations
Zhao, H., Coston, A., Adel, T., & Gordon, G. J. (2019) · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) · doi:10.48550/arxiv.1910.07162 · Cited by 52 · see citing works
Gradient Reversal against Discrimination: A Fair Neural Network Learning Approach
Raff, E., & Sylvester, J. (2018) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00029 · Cited by 51 · see citing works
Accuracy and Fairness for Juvenile Justice Risk Assessments
Berk, R. (2019) · Journal of Empirical Legal Studies · doi:10.1111/jels.12206 · Cited by 50 · see citing works
Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learning
Wang, Y., Wang, X., Beutel, A., Prost, F., Chen, J., & Chi, E. H. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467326 · Cited by 50 · see citing works
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603207 · Cited by 49 · see citing works
Investigating Algorithmic Risk and Race
Hamilton, M. (2021) · UCLA Criminal Justice Law Review · doi:10.5070/cj85154807 · Cited by 48 · see citing works
A general approach to fairness with optimal transport
Silvia, C., Ray, J., Tom, S., Aldo, P., Heinrich, J., & John, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5771 · Cited by 43 · see citing works
Antidiscrimination Insurance Pricing: Regulations, Fairness Criteria, and Models
Xin, X., & Huang, F. (2024) · North American Actuarial Journal · doi:10.1080/10920277.2023.2190528 · Cited by 43 · see citing works
Repairing without Retraining: Avoiding Disparate Impact with Counterfactual DistributionsPreprint
Wang, H., Ustun, B., & Calmon, F. (2019) · International Conference on Machine Learning · doi:10.48550/arxiv.1901.10501 · Cited by 100 · see citing works
Fogliato, R., Xiang, A., Lipton, Z. C., Nagin, D. S., & Chouldechova, A. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462538 · Cited by 41 · see citing works
Costs and Benefits of Fair Representation Learning
McNamara, D., Ong, C. S., & Williamson, R. C. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3317964 · Cited by 38 · see citing works
Provably Fair RepresentationsPreprint
McNamara, D., Ong, C. S., & Williamson, R. C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.04394 · Cited by 38 · see citing works
Noise-tolerant fair classification
Lamy, A. L., Zhong, Z., Menon, A. K., & Verma, N. (2019) · arXiv · doi:10.48550/arxiv.1901.10837 · Cited by 36 · see citing works
A Geometric Solution to Fair Representations
He, Y., Burghardt, K., & Lerman, K. (2020) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3375627.3375864 · Cited by 33 · see citing works
Lum, K., Zhang, Y., & Bower, A. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533105 · Cited by 31 · see citing works
Self-citationPersonalization and targeting: how to experiment, learn & optimize
Lemmens, A., Roos, J. M. T., Gabel, S., Ascarza, E., Bruno, H. A., Gordon, B. R., Israeli, A., Feit, E. M., Mela, C. F., & Netzer, O. (2025) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2025.07.004 · Cited by 30 · see citing works
Lu, M., Zhao, Q., Zhang, J., Pohl, K. M., Fei-Fei, L., Niebles, J. C., & Adeli, E. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01077 · Cited by 30 · see citing works
Review of Mathematical frameworks for Fairness in Machine LearningPreprint
del Barrio, E., Gordaliza, P., & Loubes, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.13755 · Cited by 30 · see citing works
Predictive Algorithms in Justice Systems and the Limits of Tech-Reformism
Ugwudike, P. (2022) · International Journal for Crime, Justice and Social Democracy · doi:10.5204/ijcjsd.2189 · Cited by 29 · see citing works
The fairness‐accuracy Pareto front
Wei, S. H., & Niethammer, M. (2021) · Statistical Analysis and Data Mining The ASA Data Science Journal · doi:10.1002/sam.11560 · Cited by 29 · see citing works
Ireland, L. (2019) · Journal of Crime and Justice · doi:10.1080/0735648x.2019.1655781 · Cited by 27 · see citing works
del Barrio, E., Gordaliza, P., & Loubes, J. (2019) · Information and Inference A Journal of the IMA · doi:10.1093/imaiai/iaz016 · Cited by 27 · see citing works
Achieving Fairness via Post-Processing in Web-Scale Recommender Systems✱
Nandy, P., DiCiccio, C. J., Venugopalan, D., Logan, H., Basu, K., & Karoui, N. E. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533136 · Cited by 26 · see citing works
A Proposal for Identifying and Managing Bias in Artificial Intelligence
Schwartz, R., Down, L., Jonas, A., & Tabassi, E. (2021) · doi:10.6028/nist.sp.1270-draft · Cited by 26 · see citing works
Deep fair models for complex data: Graphs labeling and explainable face recognition
Franco, D., Navarin, N., Donini, M., Anguita, D., & Oneto, L. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.05.109 · Cited by 25 · see citing works
Fairness On The Ground: Applying Algorithmic Fairness Approaches to Production SystemsPreprint
Bakalar, C., Barreto, R. D. C., Bergman, S., Bogen, M., Chern, B., Corbett‐Davies, S., Hall, M. A., Kloumann, I. M., Lam, M. S., Candela, J. Q., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.06172 · Cited by 25 · see citing works
Integrating Technical and Legal Concepts of Privacy
Sokolovska, A., & Kocarev, L. (2018) · IEEE Access · doi:10.1109/access.2018.2836184 · Cited by 24 · see citing works
Algorithmic Risk Assessment in the Hands of Humans
Stevenson, M. T., & Doleac, J. L. (2024) · American Economic Journal Economic Policy · doi:10.1257/pol.20220620 · Cited by 24 · see citing works
Algorithmic bias: Social science research integration through the 3-D Dependable AI Framework
Ukanwa, K. (2024) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2024.101836 · Cited by 24 · see citing works
Johnson, G. M. (2023) · Journal of Moral Philosophy · doi:10.1163/17455243-20234372 · Cited by 24 · see citing works
Fairness Evaluation in Presence of Biased Noisy Labels
Fogliato, R., G'Sell, M., & Chouldechova, A. (2020) · arXiv · doi:10.48550/arxiv.2003.13808 · Cited by 23 · see citing works
Enhancing and Accelerating Social Science Via Automation: Challenges and Opportunities
Yarkoni, T., Eckles, D., Heathers, J. A. J., Levenstein, M. C., Smaldino, P. E., & Lane, J. I. (2021) · Harvard Data Science Review · doi:10.1162/99608f92.df2262f5 · Cited by 21 · see citing works
BEAT Unintended Bias in Personalized Policies
Ascarza, E., & Israeli, A. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2115293119 · Cited by 21 · see citing works
Principal Fairness for Human and Algorithmic Decision-Making
Imai, K., & Jiang, Z. (2023) · Statistical Science · doi:10.1214/22-sts872 · Cited by 20 · see citing works
Pachamanova, D. A., Tilson, V., & Dwyer‐Matzky, K. (2021) · INFORMS Transactions on Education · doi:10.1287/ited.2021.0251ca · Cited by 20 · see citing works
Predicting Race And Ethnicity To Ensure Equitable Algorithms For Health Care Decision Making
Cabreros, I., Agniel, D. M., Martino, S. C., Damberg, C. L., & Elliott, M. N. (2022) · Health Affairs · doi:10.1377/hlthaff.2022.00095 · Cited by 19 · see citing works
Montasari, R. (2023) · Advances in information security · doi:10.1007/978-3-031-21920-7_5 · Cited by 19 · see citing works
Removing biased data to improve fairness and accuracyPreprint
Verma, S., Ernst, M. D., & Just, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.03054 · Cited by 19 · see citing works
All of the Fairness for Edge Prediction with Optimal TransportPreprint
Laclau, C., Redko, I., Choudhary, M., & Largeron, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.16326 · Cited by 19 · see citing works
Tackling Algorithmic Bias in Neural-Network Classifiers using Wasserstein-2 Regularization
Risser, L., González-Sanz, A., Vincenot, Q., & Loubes, J. (2022) · Journal of Mathematical Imaging and Vision · doi:10.1007/s10851-022-01090-2 · Cited by 18 · see citing works
Removing the Influence of Group Variables in High-Dimensional Predictive Modelling
Aliverti, E., Lum, K., Johndrow, J. E., & Dunson, D. B. (2021) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/rssa.12613 · Cited by 18 · see citing works
Self-citationLearning Fair Representations for Kernel ModelsPreprint
Tan, Z., Yeom, S., Fredrikson, M., & Talwalkar, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.11813 · Cited by 18 · see citing works
Liu, J., & Li, D. M. (2024) · Asian Journal of Criminology · doi:10.1007/s11417-024-09429-x · Cited by 16 · see citing works
The Role of Accuracy in Algorithmic Process Fairness Across Multiple Domains
Albach, M., & Wright, J. R. (2021) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3465456.3467620 · Cited by 15 · see citing works
Improving Reliability Estimation for Individual Numeric Predictions: A Machine Learning Approach
Adomavičius, G., & Wang, Y. (2021) · INFORMS journal on computing · doi:10.1287/ijoc.2020.1019 · Cited by 14 · see citing works
Berk, R. A., Kuchibhotla, A. K., & Tchetgen, E. T. (2023) · Sociological Methods & Research · doi:10.1177/00491241231155883 · Cited by 14 · see citing works
Application of fairness to healthcare, organizational justice, and finance: A survey
Birzhandi, P., & Cho, Y.-S. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.119465 · Cited by 13 · see citing works
Parole Rules in the United States: Conditions of Parole in Historical Perspective, 1956–2020
Wiggins, B., Rhine, E. E., Crye, B., Tu, R., & Mitchell, K. L. (2021) · Criminal Justice Review · doi:10.1177/07340168211020811 · Cited by 10 · see citing works
Fair-by-design explainable models for prediction of recidivism
Soares, E., & Angelov, P. (2019) · arXiv · doi:10.48550/arxiv.1910.02043 · Cited by 9 · see citing works
Fairness Criteria through the Lens of Directed Acyclic Graphs
Baer, B. R., Gilbert, D. E., & Wells, M. T. (2020) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780190067397.013.31 · Cited by 9 · see citing works
What About Applied Fairness?Preprint
Sylvester, J., & Raff, E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05250 · Cited by 8 · see citing works
Formalising Anti-Discrimination Law in Automated Decision Systems
Sargeant, H., & Magnusson, M. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732015 · Cited by 7 · see citing works
Cooperative Multi-Agent Fairness and Equivariant Policies
Grupen, N. A., Selman, B., & Lee, D. D. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i9.21166 · Cited by 7 · see citing works
Using Bias Parity Score to Find Feature-Rich Models with Least Relative Bias
Jain, B., Huber, M., Elmasri, R. A., & Fegaras, L. (2020) · Technologies · doi:10.3390/technologies8040068 · Cited by 7 · see citing works
Closer than they Appear: A Bayesian Perspective on Individual-Level Heterogeneity in Risk Assessment
Lum, K., Dunson, D. B., & Johndrow, J. E. (2022) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/rssa.12792 · Cited by 7 · see citing works
Self-citationImproving Fairness in Criminal Justice Algorithmic Risk Assessments Using Conformal Prediction SetsPreprint
Berk, R. A., & Kuchibhotla, A. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.11664 · Cited by 7 · see citing works
Mitigating Bias in Algorithmic Systems - A Fish-Eye View
Orphanou, K., Otterbacher, J., Kleanthous, S., Batsuren, K., Giunchiglia, F., Bogina, V., Tal, A. S., Hartman, A., & Kuflik, T. (2021) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.6240582 · Cited by 6 · see citing works
The impact of overbooking on a pre-trial risk assessment tool
Lum, K., Boudin, C., & Price, M. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372846 · Cited by 6 · see citing works
Self-citationEfficiency and fairness in recurring data-driven risk assessments of violent recidivism
Karimi-Haghighi, M., & Castillo, C. (2021) · ACM Symposium on Applied Computing · doi:10.1145/3412841.3441975 · Cited by 6 · see citing works
Adversarial Privacy Preservation under Attribute Inference Attack
Zhao, H., Chi, J., Tian, Y., & Gordon, G. J. (2019) · Cited by 6 · see citing works
Fairness with Continuous Optimal TransportPreprint
Chiappa, S., & Pacchiano, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.02084 · Cited by 6 · see citing works
Gao, Y., Hao, J., & Zhou, B. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103858 · Cited by 5 · see citing works
Algorithms, policing, and race
Ugwudike, P. (2023) · Routledge eBooks · doi:10.4324/9781003176619-16 · Cited by 5 · see citing works
Jain, B., Huber, M., Elmasri, R. A., & Fegaras, L. (2020) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/3389189.3397990 · Cited by 5 · see citing works
Referee-Meta-Learning for Fast Adaptation of Locational Fairness
Chen, W., Xie, Y., Jia, X., He, E., Bao, H., An, B., & Zhou, X. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30197 · Cited by 5 · see citing works
Learning fair models and representations
Oneto, L. (2020) · Intelligenza Artificiale · doi:10.3233/ia-190034 · Cited by 4 · see citing works
- Authors
- Johndrow, J. E., & Lum, K.
- Affiliations
- Stanford University (US)
- Venue
- The Annals of Applied Statistics
- Publisher
- Institute of Mathematical Statistics
- Published
- 2019-03-01
- Volume
- 13
- Issue
- 1
- Pages
- 189-220
- ISSN
- 1932-6157 (ISSN-L)
- 1941-7330
- OpenAlex
- W2600853029
- Cited by
- 141
- References
- 40
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 12 · 2023: 14 · 2022: 16 · 2021: 32 · 2020: 28 · 2019: 19 · 2018: 8 · 2017: 1
- FWCI
- 41.52
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- The Annals of Applied Statistics (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Project Euclid (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Criminal Justice and Corrections Analysis
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Keywords
- algorithmic bias, sensitive information removal, recidivism prediction, protected characteristics, parole decisions, sentencing, racial disparities, criminal justice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
