CoChat

The false promise of risk assessments: epistemic reform and the limits of fairness

Open access
Green, Ben · 2020 · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

The DOI matches its Crossref record; 118 of the 120 references it cites were found in a registry or an index; 2 of the 97 sources its claims cite were checked for support.

Abstract

Risk assessments have proliferated in the United States criminal justice system. The theory of change motivating their adoption involves two key assumptions: first, that risk assessments will reduce human biases by making objective decisions, and second, that risk assessments will promote criminal justice reform. In this paper I interrogate both of these assumptions, concluding that risk assessments are an ill-advised tool for challenging the centrality and legitimacy of incarceration within the criminal justice system. First, risk assessments fail to provide objectivity, as their use creates numerous sites of discretion. Second, risk assessments provide no guarantee of reducing incarceration; instead, they risk legitimizing the criminal justice system's structural racism. I then consider, via an "epistemic reform," the path forward for criminal justice reform. I reinterpret recent results regarding the "impossibility of fairness" as not simply a tension between mathematical metrics but as evidence of a deeper tension between notions of equality. This expanded frame challenges the formalist, colorblind proceduralism at the heart of the criminal justice system and suggests a more structural approach to reform. Together, this analysis highlights how algorithmic fairness narrows the scope of judgments about justice and how "fair" algorithms can reinforce discrimination.

Overview

Integrity signals

Registry

Crossref · 10.1145/3351095.3372869

Cites

118 of 118 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

2 Valid · 0 Not valid · 42 Not checked · 104 Own results and interpretations

85 more claims

Create a free account to see all 148 claims and to verify the 95 references still open.

See all claims — free account
  • It may state two separate claims as one. 42
  • Probably not a substantial claim. 27

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Research works printed but not linked: 16

  1. Paul Butler. 2017. Chokehold: Policing Black Men. The New Press.
  2. Julie Ciccolini and Cynthia Conti-Cook. 2018. Rationing Justice: Risk As- sessment Instruments in the American Criminal Justice System. EuropeNow (2018). https://www.europenowjournal.org/2018/11/07/rationing-justice-risk- assessment-instruments-in-the-american-criminal-justice-system/
  3. The Leadership Conference Education Fund. 2018. The Use of Pretrial "Risk Assessment" Instruments: A Shared Statement of Civil Rights Concerns. (2018). https://leadershipconferenceedfund.org/pretrial-risk-assessment/
  4. FWD.us. 2018. Every Second: The Impact of the Incarceration Crisis on Amer- ica's Families. (2018). https://everysecond.fwd.us/downloads/EverySecond. FWD.us.pdf
  5. Ben Green. 2018. Data Science as Political Action: Grounding Data Science in a Politics of Justice. arXiv preprint arXiv:1811.03435 (2018). · arXiv
  6. Ben Green and Lily Hu. 2018. The Myth in the Methodology: Towards a Re- contextualization of Fairness in Machine Learning. In Machine Learning: The Debates workshop at the 35th International Conference on Machine Learning.
  7. Kamala Harris and Rand Paul. 2017. Pretrial Integrity and Safety Act of 2017. 115th Congress (2017).
  8. Sheila Jasanoff. 2004. The idiom of co-production. In States of Knowledge: The Co-Production of Science and the Social Order, Sheila Jasanoff (Ed.). Routledge, Chapter 1, 1-12.
  9. Sheila Jasanoff. 2004. Ordering knowledge, ordering society. In States of Knowl- edge: The Co-Production of Science and the Social Order, Sheila Jasanoff (Ed.). Routledge, Chapter 2, 13-45.
  10. Alec Karakatsanis. 2015. Policing, Mass Imprisonment, and the Failure of American Lawyers. Harvard Law Review Forum 128 (2015), 253.
  11. Alec Karakatsanis. 2019. The Punishment Bureaucracy: How to Think About "Criminal Justice Reform". The Yale Law Journal Forum 128 (2019), 848-935.
  12. Sandra G. Mayson. 2018. Dangerous Defendants. Yale Law Journal 127, 3 (2018), 490-568.
  13. Allegra M. McLeod. 2013. Confronting Criminal Law's Violence: The Possibili- ties of Unfinished Alternatives. Unbound: Harvard Journal of the Legal Left 8 (2013), 109-132.
  14. Martha Minow. 1991. Making All the Difference: Inclusion, Exclusion, and Ameri- can Law. Cornell University Press.
  15. Harvard Law Review. 2019. Introduction. Harvard Law Review 132, 6 (2019), 1568-1574.
  16. Crystal S. Yang. 2017. Toward an Optimal Bail System. New York University Law Review 92, 5 (2017), 1399-1493.

Cited by

See all 128

Metadata

Authors
Green, B.
Affiliations
Harvard University (US)
Corresponding author
Green, Ben
Venue
Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
Publisher
ACM
Published
2020-01-27
Pages
594-606
OpenAlex
W3004155201
Cited by
80
References
120
Citations by year
2026: 5 · 2025: 9 · 2024: 18 · 2023: 18 · 2022: 14 · 2021: 12 · 2020: 4
FWCI
12.45
Type
conference paper
Language
en
Open access
Locations
ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
Funders
National Science Foundation (DGE1745303)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Sociology and Political Science › Criminal Justice and Corrections Analysis
  • Social Sciences › Law › Criminal Justice and Penology
Keywords
risk assessment, criminal justice reform, algorithmic fairness, algorithmic discrimination, structural racism, incarceration, equality, algorithmic decision-making

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.