CoChat

Transparency, Governance and Regulation of Algorithmic Tools Deployed in the Criminal Justice System: a UK Case Study

Open access
Zilka, Miri, Sargeant, Holli, Weller, Adrian · 2022 · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society

The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.

Abstract

We present a survey of tools used in the criminal justice system in the UK in three categories: data infrastructure, data analysis, and risk prediction. Many tools are currently in deployment, offering potential benefits, including improved efficiency and consistency. However, there are also important concerns. Transparent information about these tools, their purpose, how they are used, and by whom is difficult to obtain. Even when information is available, it is often insufficient to enable a satisfactory evaluation. More work is needed to establish governance mechanisms to ensure that tools are deployed in a transparent, safe and ethical way. We call for more engagement with stakeholders and greater documentation of the intended goal of a tool, how it will achieve this goal compared to other options, and how it will be monitored in deployment. We highlight additional points to consider when evaluating the trustworthiness of deployed tools and make concrete proposals for policy.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3514094.3534200

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Stages in the criminal justice pipeline where predictive tools are used. In the top layer, tools are used to help decide where to look for crime. In all other layers, tools are used to help decide how to process an individual.
Figure 1 Stages in the criminal justice pipeline where predictive tools are used. In the top layer, tools are used to help decide where to look for crime. In all other layers, tools are used to help decide how to process an individual.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 20

  1. Julia Angwin , Jeff Larson , Surya Mattu , and Lauren Kirchner . 2016. Machine bias . In Ethics of Data and Analytics . Auerbach Publications , 254--264. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. In Ethics of Data and Analytics. Auerbach Publications, 254--264.
  2. Tao Cheng . 2021. Crime, Policing and Citizenship (CPC) - Space-Time Interactions of Dynamic Networks . https://gtr.ukri.org/projects?ref=EP%2FJ004197%2F1 Accessed : 2021 . Tao Cheng. 2021. Crime, Policing and Citizenship (CPC) - Space-Time Interactions of Dynamic Networks. https://gtr.ukri.org/projects?ref=EP%2FJ004197%2F1 Accessed: 2021.
  3. Alexandra Chouldechova . 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5, 2 ( 2017 ), 153--163. Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5, 2 (2017), 153--163.
  4. Diane Coyle and Adrian Weller . 2020. "Explaining" machine learning reveals policy challenges. Science 368, 6498 ( 2020 ), 1433--1434. Diane Coyle and Adrian Weller. 2020. "Explaining" machine learning reveals policy challenges. Science 368, 6498 (2020), 1433--1434.
  5. Danielle Ensign , Sorelle A Friedler , Scott Neville , Carlos Scheidegger , and Suresh Venkatasubramanian . 2018 . Runaway feedback loops in predictive policing . In Conference on Fairness, Accountability and Transparency. PMLR, 160--171 . Danielle Ensign, Sorelle A Friedler, Scott Neville, Carlos Scheidegger, and Suresh Venkatasubramanian. 2018. Runaway feedback loops in predictive policing. In Conference on Fairness, Accountability and Transparency. PMLR, 160--171.
  6. Seena Fazel , Matthias Burghart , Thomas Fanshawe , Sharon Danielle Gil , John Monahan, and Rongqin Yu. 2021 . The predictive performance of criminal risk assessment tools used at sentencing. Seena Fazel, Matthias Burghart, Thomas Fanshawe, Sharon Danielle Gil, John Monahan, and Rongqin Yu. 2021. The predictive performance of criminal risk assessment tools used at sentencing.
  7. Seena Fazel and Achim Wolf . 2018. Selecting a risk assessment tool to use in practice: a 10-point guide. Evidence-based mental health 21, 2 ( 2018 ), 41--43. Seena Fazel and Achim Wolf. 2018. Selecting a risk assessment tool to use in practice: a 10-point guide. Evidence-based mental health 21, 2 (2018), 41--43.
  8. Centre for Data Ethics and Innovation. 2021. Review into bias in algorithmic decision-making. Centre for Data Ethics and Innovation. 2021. Review into bias in algorithmic decision-making.
  9. Fieke Jansen . 2018. Data Driven Policing in the Context of Europe. Data Justice Lab ( 2018 ). Fieke Jansen. 2018. Data Driven Policing in the Context of Europe. Data Justice Lab (2018).
  10. Deirdre K Mulligan , Daniel Kluttz , and Nitin Kohli . 2019. Shaping our tools: Contestability as a means to promote responsible algorithmic decision making in the professions. Available at SSRN 3311894 ( 2019 ). Deirdre K Mulligan, Daniel Kluttz, and Nitin Kohli. 2019. Shaping our tools: Contestability as a means to promote responsible algorithmic decision making in the professions. Available at SSRN 3311894 (2019).
  11. The National Police Chiefs Council (NPCC) , Association of Police, and Crime Commissioners . 2020 . National Policing Digital Strategy 2020--2030. https://www.apccs.police.uk/media/4886/national-policing-digital-strategy-2020--2030.pdf The National Police Chiefs Council (NPCC), Association of Police, and Crime Commissioners. 2020. National Policing Digital Strategy 2020--2030. https://www.apccs.police.uk/media/4886/national-policing-digital-strategy-2020--2030.pdf
  12. ACRO Criminal Records Office . [n. d.]. Police National Computer Services. https://www.acro.police.uk/PNC-services Accessed : 2022 . ACRO Criminal Records Office. [n. d.]. Police National Computer Services. https://www.acro.police.uk/PNC-services Accessed: 2022.
  13. Frank Pasquale . 2019. The Black Box Society . Harvard University Press , Cambridge, MA . Frank Pasquale. 2019. The Black Box Society. Harvard University Press, Cambridge, MA.
  14. The Metropolitan Police . [n. d.]. Gangs violence matrix. https://www.met.police.uk/police-forces/metropolitan-police/areas/about-us/about-the-met/gangs-violence-matrix/ Accessed : 2021 . The Metropolitan Police. [n. d.]. Gangs violence matrix. https://www.met.police.uk/police-forces/metropolitan-police/areas/about-us/about-the-met/gangs-violence-matrix/ Accessed: 2021.
  15. West Yorkshire Police . [n. d.]. National Data Analytics Solution (NDAS) Privacy Notice. https://www.westyorkshire.police.uk/advice/modern-slavery/national-data-analytics-solution-ndas-privacy-notice Accessed : 2022 . West Yorkshire Police. [n. d.]. National Data Analytics Solution (NDAS) Privacy Notice. https://www.westyorkshire.police.uk/advice/modern-slavery/national-data-analytics-solution-ndas-privacy-notice Accessed: 2022.
  16. Predpol. [n. d.]. Kent Police Use PredPol To Prevent Violent Crime. https://www.predpol.com/kent-police-use-predpol-to-prevent-violent-crime/ Accessed : 2022 . Predpol. [n. d.]. Kent Police Use PredPol To Prevent Violent Crime. https://www.predpol.com/kent-police-use-predpol-to-prevent-violent-crime/ Accessed: 2022.
  17. Edward Santow . 2021 . Intelligence: Coding calues into the future . In The Public Square Project, Peter Lewis and Jordan Guiao (Eds.). Melbourne University Press , Melbourne . Edward Santow. 2021. Intelligence: Coding calues into the future. In The Public Square Project, Peter Lewis and Jordan Guiao (Eds.). Melbourne University Press, Melbourne.
  18. Andrew Selbst . 2018 . Disparate Impact in Big Data Policing . Georgia Law Review 52 , 1 (2018), 3373 . Andrew Selbst. 2018. Disparate Impact in Big Data Policing. Georgia Law Review 52, 1 (2018), 3373.
  19. Daniel Sprick . 2019. Predictive Policing in China: An Authoritarian Dream of Public Security. Naveiñ Reet: Nordic Journal of Law and Social Research (NNJLSR) No 9 ( 2019 ). Daniel Sprick. 2019. Predictive Policing in China: An Authoritarian Dream of Public Security. Naveiñ Reet: Nordic Journal of Law and Social Research (NNJLSR) No 9 (2019).
  20. Niche Technology . [n. d.]. Niche RMS Customer Profiles. https://nicherms.com/region/uk/ Accessed : 2021 . Niche Technology. [n. d.]. Niche RMS Customer Profiles. https://nicherms.com/region/uk/ Accessed: 2021.

Cited by

See all 42

Metadata

Authors
Zilka, M., Sargeant, H., & Weller, A.
Affiliations
University of Cambridge (GB)
Venue
Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society
Publisher
ACM
Published
2022-07-26
Pages
880-889
ISSN
3065-8365 (ISSN-L)
OpenAlex
W4281799859
Cited by
28
References
15
Citations by year
2026: 3 · 2025: 11 · 2024: 8 · 2023: 6
FWCI
6.21
Type
conference paper
Language
en
Open access
Locations
  • Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • Apollo (University of Cambridge) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
  • Apollo (cc-by)
Funders
  • General Sir John Monash Foundation
  • EPSRC (EP/V025279/1)
  • Leverhulme Trust (ECF-2021-429)
  • The Alan Turing Institute
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Law › Artificial Intelligence in Law
  • Social Sciences › Law › Digitalization, Law, and Regulation
Keywords
algorithmic tools, criminal justice system, algorithmic transparency, governance, regulation, risk prediction, trustworthiness, stakeholder engagement, algorithmic accountability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.