Out With the Old and in With the New? An Empirical Comparison of Supervised Learning Algorithms to Predict Recidivism
The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Recent research has produced mixed results as to whether newer machine learning algorithms outperform older, more traditional methods such as logistic regression in predicting recidivism. In this study, we compared the performance of 12 supervised learning algorithms to predict recidivism among offenders released from Minnesota prisons. Using multiple predictive validity metrics, we assessed the performance of these algorithms across varying sample sizes, recidivism base rates, and number of predictors in the data set. The newer machine learning algorithms generally yielded better predictive validity results. LogitBoost had the best overall performance, followed by Random forests, MultiBoosting, bagged trees, and logistic model trees. Still, the gap between the best and worst algorithms was relatively modest, and none of the methods performed the best in each of the 10 scenarios we examined. The results suggest that multiple methods, including machine learning algorithms, should be considered in the development of recidivism risk assessment instruments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
An empirical comparison of supervised learning algorithms
Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143865 · Cited by 2,780
Internal validation of predictive models
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J. M., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, J. D. F. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00341-9 · Cited by 2,746
Steyerberg, E. W., Bleeker, S. E., Moll, H. A., Grobbee, D. E., & Moons, K. G. M. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00047-7 · Cited by 611
Berk, R., Sherman, L., Barnes, G., Kurtz, E., & Ahlman, L. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00556.x · Cited by 178 · cited 4×
“In contrast, the tools designed by Berk et al. (2009) and Barnes and Hyatt (2012) are fully automated and specific to one jurisdiction (Philadelphia).”
BackgroundMethodInfluential
Research works printed but not linked: 38
- 10.1177/0011128705281756 · DOI · Paper page
- Barnes G. C., 2012, Classifying adult probationers by forecasting future offending
- Bennett P. N., 2000, Assessing the calibration of Naïve Bayes’ posterior estimates
- 10.1111/1745-9133.12047 · DOI · Paper page
- 10.1007/s10940-013-9195-0 · DOI
- Bonta J., 2007, Risk-needs-responsivity model for offender assessment and rehabilitation
- 10.1007/BF00058655 · DOI · Paper page
- Breiman L., 1984, Classification and regression trees
- Breitenbach M., 2009, Rare association rule mining and knowledge discovery: Technologies for infrequent and critical event detection, 231
- 10.1177/0093854808326545 · DOI · Paper page
- Brennan T., 2000, Evaluation of reliability and validity of COMPAS Scales: National aggregate sample
- 10.1111/1745-9133.12055 · DOI
- Burgess E. W., 1928, The workings of the indeterminate sentence law and the parole system in Illinois, 221
- 10.1111/1745-9133.12059 · DOI
- Cullen F. T., 2000, Policies, processes, and decisions of the criminal justice system, 3, 109
- 10.1177/0093854812453911 · DOI
- 10.1177/0887403413478821 · DOI · Paper page
- 10.1016/j.jcrimjus.2013.06.009 · DOI
- 10.1177/1079063211429470 · DOI
- Freund Y., 1996, Proceedings of the 13th International Conference on Machine Learning, 148
- 10.1214/aos/1016218223 · DOI · Paper page
- 10.1007/s11292-014-9221-8 · DOI
- 10.1007/s10994-009-5119-5 · DOI · Paper page
- 10.7208/chicago/9780226315997.001.0001 · DOI · Paper page
- 10.1007/978-0-387-84858-7 · DOI · Paper page
- Hess J., 2013, Risk assessment accuracy in corrections population management: Testing the promise of tree based ensemble predictions
- 10.5121/ijist.2012.2305 · DOI
- 10.1007/s10994-005-0466-3 · DOI · Paper page
- 10.1007/s10940-011-9137-7 · DOI
- Lowenkamp C. T., 2009, Federal Probation, 73, 33
- 10.1111/1745-9133.12057 · DOI · Paper page
- Ridgeway G., 2013, National Institute of Justice Journal, 34
- 10.1007/BF02354472 · DOI
- 10.1111/j.1467-985X.2012.01056.x · DOI · Paper page
- Turner S., 2009, Development of the California Static Risk Assessment Instrument (CSRA)
- 10.1007/978-1-4757-2440-0 · DOI · Paper page
- 10.1023/A:1007659514849 · DOI · Paper page
- 10.1162/neco.1996.8.7.1341 · DOI · Paper page
Cited by
Scheuerman, M. K., Paul, J. M., & Brubaker, J. R. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359246 · Cited by 252
Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits
Bandy, J. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449148 · Cited by 193
Bickman, L. (2020) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-020-01065-8 · Cited by 184
“Duwe and Kim (2017) compared 12 statistical methods including ML approaches on their accuracy in predicting recidivism among 22,772 offenders.”
Forks over knives: Predictive inconsistency in criminal justice algorithmic risk assessment tools
Greene, T., Shmueli, G., Fell, J., Lin, C.-F., & Liu, H.-W. (2022) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/rssa.12966 · Cited by 15
The great methods bake-off: Comparing performance of machine learning algorithms
Kigerl, A., Hamilton, Z., Kowalski, M., & Mei, X. (2022) · Journal of Criminal Justice · doi:10.1016/j.jcrimjus.2022.101946 · Cited by 8
Metadata
- Authors
- Duwe, G., & Kim, K.
- Venue
- Criminal Justice Policy Review
- Publisher
- SAGE Publications
- Published
- 2015-09-15
- Volume
- 28
- Issue
- 6
- Pages
- 570-600
- ISSN
- 0887-4034 (ISSN-L)
- 1552-3586
- OpenAlex
- W2337553400
- Cited by
- 59
- References
- 5
- FWCI
- 6.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.