Assessing the feasibility and performance of risk assessment instruments for early intervention and prevention services in Juvenile Justice

The argument that the implementation of AI/ML will lead to a substantial improvement in accuracy within the juvenile justice system does not appear to be consistently supported by the currently available evidence.

“Consequently, the argument that the implementation of AI/ML will lead to a substantial improvement in accuracy within the juvenile justice system does not appear to be consistently supported by the currently available evidence (Kigerl et al., 2022;Sheppard et al., 2024).”

Virtually identical performance was documented between complex techniques and traditional approaches such as logistic regression, especially when the sample size is large, or the evaluated scenario reduces predictive variability.

“Virtually identical performance was documented between complex techniques and traditional approaches such as logistic regression, especially when the sample size is large, or the evaluated scenario reduces predictive variability (Kigerl et al., 2022;Sheppard et al., 2024).”

The variability ranged from no predictive differences (0.74 vs. 0.74 in Sheppard et al., 2024) to moderate improvements (0.75 vs. 0.67 in Mori et al., 2024; 0.73 vs. 0.64 in Miron et al., 2021; and 0.69 vs. 0.64 in Ting et al., 2018).

“This variability ranged from no predictive differences (0.74 vs. 0.74 in Sheppard et al., 2024) to moderate improvements (0.75 vs. 0.67 in Mori et al., 2024;0.73 vs. 0.64 in Miron et al., 2021;and 0.69 vs. 0.64 in Ting et al., 2018).”

Sheppard et al. (2024) found similar performance across techniques.

“Similar performa nce across technique s.”

The transition from traditional statistical models to complex AI/ML architectures yields only marginal improvements in studies with large samples and preventative approaches.

“The results of studies with large samples (Kigerl et al., 2022) and preventative approaches (Sheppard et al., 2024) demonstrate that the transition from traditional statistical models to complex AI/ML architectures yields only marginal improvements.”