Layers of Bias: A Unified Approach for Understanding Problems With Risk Assessment
The DOI matches its Crossref record; 57 of the 62 references it cites were found in a registry or an index.
Abstract
Scholars in several fields, including quantitative methodologists, legal scholars, and theoretically oriented criminologists, have launched robust debates about the fairness of quantitative risk assessment. As the Supreme Court considers addressing constitutional questions on the issue, we propose a framework for understanding the relationships among these debates: layers of bias. In the top layer, we identify challenges to fairness within the risk-assessment models themselves. We explain types of statistical fairness and the tradeoffs between them. The second layer covers biases embedded in data. Using data from a racially biased criminal justice system can lead to unmeasurable biases in both risk scores and outcome measures. The final layer engages conceptual problems with risk models: Is it fair to make criminal justice decisions about individuals based on groups? We show that each layer depends on the layers below it: Without assurances about the foundational layers, the fairness of the top layers is irrelevant.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“While both [501, 470] showed that the momentum variants of local SGD can converge to stationary points of non-convex objective functions at the same rate as synchronous mini-batch SGD, it”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [470]
In the citing paper
“This framework generalizes the server momentum framework proposed by Hsu et al. [237], Wang et al. [470], which allows momentum without increasing communication costs.”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [470]
In the citing paper
“Similar dynamics have been found in credit rating agencies [20,40], supply chain audits [69], and accounting audits [58].”
From Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance (2022) · cites this paper as [20]
In the citing paper
“Criminal justice researchers have also established that racial bias permeates arrest records, as Black Americans and other minority groups are disproportionately stopped, searched, and arrested by police for the same behaviors exhibited by White Americans [71][72][73].”
From Efficiency vs. Equity: A Structured Interdisciplinary Review of AI in Criminal Justice Risk Assessments (2026) · cites this paper as [71]
In the citing paper
“Over the past few decades, there has been a push for sentencing guidelines and other tools that reduce judicial discretion when making decisions about pre-trial detention and custodial sentences [71].”
From Efficiency vs. Equity: A Structured Interdisciplinary Review of AI in Criminal Justice Risk Assessments (2026) · cites this paper as [71]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Bertrand, M., & Mullainathan, S. (2004) · American Economic Review · doi:10.1257/0002828042002561 · Cited by 4,504
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159
The Politics of Blame Avoidance
Weaver, R. K. (1986) · Journal of Public Policy · doi:10.1017/s0143814x00004219 · Cited by 2,145
Punishment and inequality in America
(2007) · Choice Reviews Online · doi:10.5860/choice.44-4178 · Cited by 2,064
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Clinical versus mechanical prediction: a meta-analysis
Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000) · Psychological Assessment · doi:10.1037/1040-3590.12.1.19 · Cited by 1,640
STEFFENSMEIER, D., ULMER, J., & KRAMER, J. (1998) · Criminology · doi:10.1111/j.1745-9125.1998.tb01265.x · Cited by 1,592
The Recent Past and Near Future of Risk and/or Need Assessment
Andrews, D. A., Bonta, J., & Wormith, J. S. (2006) · Crime & Delinquency · doi:10.1177/0011128705281756 · Cited by 1,507
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Research works printed but not linked: 28
- Austin A., 2017, Federal Probation, 81
- Ayoub L. H., 2015, Coming home to Harlem: A randomized controlled trial of the Harlem Parole Reentry Court
- Barocas S., 2016, California Law Review, 104, 721
- Blackmon D. A., 2008, Slavery by another name: The re-enslavement of Black people in America from the Civil War to World War II
- Bonta J., 2007, Rehabilitation, 6, 1
- Chouldechova A., G’Sell M. (2017). Fairer and more accurate, but for whom? arXiv, 1707.00046. Retrieved from http://arxiv.org/abs/1707.00046 · arXiv
- Cooprider K., 2009, Federal Probation, 73, 12
- Dieterich W., 2016, Northpointe, Inc
- Flores A. W., 2016, Federal Probation, 80, 38
- Gouldin L., 2016, Brigham Young University Law Review, 2016, 837
- Kang J., 2011, UCLA Law Review, 59, 1126
- Kleinberg J., 2016, arXiv, 1609, 05807
- Kroll J. A., 2016, University of Pennsylvania Law Review, 165, 633
- Laura and John Arnold Foundation, Pretrial justice
- Levinson J. D., 2007, Duke Law Journal, 57, 345
- Levinson J. D., 2010, Ohio State Journal of Criminal Law, 8, 187
- Lowenkamp C. T., 2008, Federal Probation, 72, 2
- Lowenkamp C. T., 2009, Federal Probation, 73, 33
- Mayson S. G., 2017, The Yale Law Journal, 127, 490
- Mayson S. G. (2018). Bias in, bias out (University of Georgia School of Law Legal Studies Research Paper No. 2018–35). The Yale Law Journal. Advance online publication. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3257004
- Oleson J. C., 2011, SMU Law Review, 64, 1329
- Richardson L. S., 2012, The Yale Law Journal, 122, 2626
- Starr S. B., 2014, Stanford Law Review, 66, 803
- Summers C., 2010, Pretrial risk assessment
- VanNostrand M., 2009, Federal Probation, 73, 5
- Westervelt E., 2017, NPR: All Things Considered
- Wexler R., 2017, Stanford Law Review, 70, 1343
- Zliobaite I. (2017). Fairness-aware machine learning: A perspective. arXiv: 1708.00754. Retrieved from http://arxiv.org/abs/1708.00754 · arXiv
Cited by
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
Algorithmic Fairness: Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-042720-125902 · Cited by 687
Self-citationJacobs, A. Z., & Wallach, H. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445901 · Cited by 362
“[19] speak to “layers of bias” in risk assessment models: “we show that each layer depends on the layers below it: Without assurances about the foundational layers, the fairness of the top layers is irrelevant.”
BackgroundRoles for computing in social change
Abebe, R., Barocas, S., Kleinberg, J., Levy, K., Raghavan, M., & Robinson, D. G. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372871 · Cited by 219
“Some advocates and scholars have argued that any feasible formalization of the problem, given the available data, will have drawbacks so serious as to draw into question the use of actuarial methods at all [39, 65, 67, 71].”
BackgroundUnderstanding artificial intelligence ethics and safety
Leslie, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05684 · Cited by 134
Metadata
- Authors
- Eckhouse, L., Lum, K., Conti-Cook, C., & Ciccolini, J.
- Affiliations
- University of Denver (US)
- Human Rights Data Analysis Group
- The Legal Aid Society (New York)
- Corresponding author
- Eckhouse, Laurel
- Venue
- Criminal Justice and Behavior
- Publisher
- SAGE Publications
- Published
- 2018-11-23
- Volume
- 46
- Issue
- 2
- Pages
- 185-209
- ISSN
- 0093-8548 (ISSN-L)
- 1552-3594
- OpenAlex
- W2901856814
- Cited by
- 156
- References
- 62
- Citations by year
- 2026: 16 · 2025: 16 · 2024: 18 · 2023: 19 · 2022: 26 · 2021: 29 · 2020: 20 · 2019: 12
- FWCI
- 30.30
- Type
- journal article
- Language
- en
- Update notices
- correction (2018-12-14)
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Criminal Justice and Behavior (publishedVersion)
- Funders
- Human Rights Data Analysis Group
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Criminal Justice and Corrections Analysis
- Social Sciences › Law › Artificial Intelligence in Law
- Keywords
- risk assessment, algorithmic fairness, data bias, racial bias in criminal justice, predictive policing, Supreme Court, constitutional question
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Layers of Bias: A Unified Approach for Understanding Problems With Risk Assessment. https://app.cochat.ai/papers/10.1177/0093854818811379
- The paper
- Eckhouse, L., Lum, K., Conti-Cook, C., & Ciccolini, J. (2019). Layers of Bias: A Unified Approach for Understanding Problems With Risk Assessment. Criminal Justice and Behavior. https://doi.org/10.1177/0093854818811379
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.