Forecasting Murder Within a Population of Probationers and Parolees: A High Stakes Application of Statistical Learning

“Machine-leaning techniques such as random forest have already been documented to improve the prediction of future charges of homicide or attempted homicide using traditionally available demographic and social variables 104 .”

“Maryland; Philadelphia, Pennsylvania; and Washington, DC, to predict those individuals on probation or parole most likely to murder or commit other crimes (Berk, Sherman, Barnes, Kurtz, & Ahlman, 2009).”

“Examples here include models predicting the risk of persons under community supervision reoffending (Berk 2008, 2011; Berk and Bleich 2013; Berk et al. 2009; Wright et al. 1984) and models assessing the risk that a gang affiliate will be involved in violence as a function of their social relationships (Papachristos 20…”

“With over 50,000 active cases at any given time, a 30-year data set showing all clients who are later charged with murder or attempted murder yields an ample haystack in which to find the power few “needles.” Using a nonlinear data mining approach to exploratory analysis, Berk et al. (2007) have developed a relatively…”

“…jurisprudential concerns about “actuarial methods” dating from at least the time when sentencing guidelines first became popular (Feeley and Simon, 1994; Messinger and Berk, 1987), and more recent discussions about the role of race have introduced an important overlay (Berk, 2009; Harcourt, 2007).”

“There has been some controversy as to whether sophisticated machine learning methods (such as random forests, Breiman 2001b, Berk et al. 2009, Ritter 2013) are necessary to produce accurate predictive models of recidivism, or if traditional approaches such as logistic regression would suffice (see,…”

“…of individuals’ future behavior are used to inform judges regarding pre-trial release and bail setting, sentencing, and parole [Phillips et al., 2016, Dieterich et al., 2016, Brennan et al., 2009, Cunningham and Sorensen, 2006, Dvoskin and Heilbrun, 2001, Quinsey et al., 1998, Berk et al., 2009].”

“…levels (Berk et al. 2006), homicides committed by probationers or parolees to identify individuals who pose a very serious threat to public safety (Berk et al. 2009), arrests for three different categories of crime committed while on probation or parole to rationalize the intensity of supervision…”

“The goal of crime data mining is to understand patterns in criminal behavior in order to predict crime, anticipate criminal activity and prevent it (e.g., see [1]).”

“No prediction studies appear to have been done based on the prior performance histories of police officers using the methods pioneered by Berk et al. (2009).”

“[114] in describing the precursors of homicide by accounting for the relative costs of forecasting errors.”

“While prior research has explored applications of machine learning to domains where human judgments are part of the data labeling process (crime [7, 21, 43], medical diagnosis [6, 22]), the selective labels problem is o‰en overlooked, with evaluation of the models typically carried out by computing traditional metrics…”

“, 2002), criminology (Berk et al., 2009), and increasingly in the social sciences (Stachl et al.”

“Conclusions The implementation of quantitative and qualitative methods to predict criminal recidivism could be a useful tool in the field of criminal justice [3–5] However, although research in this area is steadily increasing [7], its use in judicial practice is still limited [8] due to the controversial views [9–12].”

“…or to test the use of ICT-like techniques on different populations as well as evaluating their performance by comparing them to other actuarial strategies (Silver et al., 2000; Stalans et al., 2004) or establishing prediction error rates using Out Of the Bag (OOB) observations (Berk et al., 2009).”

“In contrast, the tools designed by Berk et al. (2009) and Barnes and Hyatt (2012) are fully automated and specific to one jurisdiction (Philadelphia).”

“Although development of this method would require further research in order to identify additional variables and sources of data for inclusion in the predictionmodel tomake predictions, statistically validated forecasts have consistently been shown to be more accurate or at least as good as clinical predictions such a…”

“Berk et al. (2009) provide a detailed description of the basic statistical techniques used in this revised model to forecast risk.”

“A promising advance in targeting methods for prevention of high-harm domestic abuse is the development of “big data” forecasting using “random forests” algorithms of the kind that predict murder and other serious crime (Berk, Sherman, Barnes, Kurtz, and Ahlman, 2009) .”

“Moreover, attempts have been made in recent years to develop tools for predicting murder among offenders on probation and parole (see Berk et al. 2008).”

“Examples of categorical outcomes include whether or not an individual on probation or parole is arrested for a homicide [18], whether there is a repeat incident of domestic violence in a household [19], and different rule infractions for which a prison inmate may have been reported [20].”

“In many instances, relative costs can be elicited from stakeholders, at least to the satisfaction of those stakeholders (Berk et al. 2009; Berk 2009).”

“Machine learning is also finding its way into the criminology journals (Berk et al. 2009).”

“The forecast can be based, as in our earlier work (Berk et al., 2009), on criminal history, age, local crime rate, and other objective, nonprotected characteristics (thus excluding gender, race, religion, and national origin).”

“The random forest approach to forecasting (Barnes 2019; Barnes and Hyatt 2012; Berk et al. 2009) computes likelihood through a series of random decision trees, and provides far greater accuracy than OR calculations, regardless of the sample size.”

“2009 ), which yielded predictions of their future offending over the subsequent two years.”

“This approach to actuarial forecasting was strictly limited to predicting the future dangerousness of the offenders (Barnes & Hyatt, 2012; Berk, 2008; Berk et al., 2009), and their criminogenic needs were neither assessed, nor considered.”

“Machine learning techniques have enabled the search for instruments that can detect more complex patterns than traditional regression-based instruments (Berk and Bleich 2013; Berk et al. 2009).”

“While others concluded that the prediction accuracy of machine learning models could outperform more traditional methods (Berk & Bleich, 2013; Goss & Vozikis, 2002; Razi & Athappilly, 2005), the predictive accuracy of inductive models varies across this body of literature; machine learning models show promise in incre…”

“Nonetheless, it is important to note that RFs do possess some very attractive qualities, including overfit reduction (Breiman 2001b), ability to identify interactions, and tuning for relative costs of false positives/false negatives (Berk et al. 2009).”

“Studies of parole or probation outcomes have typically been concerned with any failures, from technical violations to the the most serious crimes (Berk, 2008b; Berk et al., 2009).”

“Although statistical methods exist for minimizing this capitalization on chance (eg, Berk, Sherman, Barnes, Kurtz, & Ahlman, 2010 ), these approaches require far greater sample sizes than are typically available in clinical research.”

“47 50 Model Formulation Our data consist of entities (crimes) V , each of which has a vector of features.”

“More recently, Berk et al. (2009) developed a risk assessment tool for the Philadelphia Adult Probation and Parole Department (APPD) to sort offenders into three categories: the highest-risk offenders are considered likely to commit a subsequent violent offense; medium-risk offenders are equally likely to commit a new…”

“In social science applications of tree-based algorithms, [16] and [17] demonstrates that random forests can help generate accurate forecasts of violent crime rates in the United States.”

“Model 2 is only plausible if the relationship between age and rearrest is linear, which is unlikely given past research on the age–crime curve (Berk et al. 2009; Gendreau, Little, and Goggin 1996).”

“However, there are quite a few risk-assessment instruments that are deployed with an underlying script from a regression model (see Northpointe, 2015) or a machine learning algorithm (see Berk et al., 2008).”

“…in a criminal justice/criminology in a number of areas such as forecasting victimization, recidivism, informing arraignment decisions, and determining the costs of intervention programs (e.g., Barnes & Hyatt, 2012; Berk et al., 2009; Berk et al., 2016; Manning et al., 2018; Neuilly et al., 2011).”

“It has been observed that robust tree-based classification algorithms can effectively predict such behaviors [9,39,40], which can more accurately divide individuals into non-risk and risk groups, especially when the risk assessment tool for problem behaviors is predetermined.”

“8 Table 1 raises several other important issues, but they are beyond the scope of this review (see Berk et al., forthcoming ).”

“The use of an inductive random-forest approach based on a training data set of 30,000 cases by Berk et al. (2009) attracted considerable attention, and while the benefits of an actuarial approach were promoted in preference to the existing clinical model, the challenges were certainly not glossed…”

“…have been applied to contexts ranging from credit card fraud detection (Aleskerov, Freisleben, & Rao, 1997; Patidar & Sharma, 2011) to disease diagnosis (Karabatak & Ince, 2009; Klöppel et al., 2012) to criminal justice settings such as predicting homicide risk among parolees (Berk et al., 2009).”

“A downside in using a non-parametric model include the interpretation of partial dependencies which cannot be interpreted in any causative or absolute way, but only as descriptive (Berk et al., 2009).”

“…domestic homicides), and the greater probability of certain people committing particular offences, perhaps more sociological methods should be adopted (e.g. Berk et al., 2009), and the standpoint, for a criminal investigation, should be extended to include an explanation as well as description.”

“…randomized controlled trial, the Philadelphia Adult Probation and Parole Department evaluated the effectiveness of a risk assessment tool developed by Berk et al. (2009), and found the tool reduced the burden on parolees without significantly increasing rates of reoffense (Ahlman and Kurtz, 2009).”

“It is becoming popular to identify and predict the criminal activities which can help to avoid the crimes[8] .”

“…ff erent databases, model designs, and study objectives, the performance of the forecasting techniques is superior to many of the technologies reported in the literature, e.g. Berk et al. (2006, 2009, 2016), Berk (2019), Neuilly et al. (2011), Kim and Duwe (2016), Zeng et al. (2017), among others.”

“At present, social scientists have made little use of these methods, and mainly with prediction objectives (see for instance Berk et al., 2006; Berk et al., 2009; Compton, 2019; McKay, 2019; Rigobon et al., 2019), although more substantive uses exist (for example Berk et al., 2005).”