The great methods bake-off: Comparing performance of machine learning algorithms

Kigerl et al. (2022) found that sample size has a greater influence than algorithmic complexity.

“Sample size has a greater influence than algorithm ic complexit y.”

Kigerl et al. (2022) found that differences between model families are marginal (<0.01).

“The difference s between model families are marginal (<0.01).”

The argument that the implementation of AI/ML will lead to a substantial improvement in accuracy within the juvenile justice system does not appear to be consistently supported by the currently available evidence.

“Consequently, the argument that the implementation of AI/ML will lead to a substantial improvement in accuracy within the juvenile justice system does not appear to be consistently supported by the currently available evidence (Kigerl et al., 2022;Sheppard et al., 2024).”

The transition from traditional statistical models to complex AI/ML architectures yields only marginal improvements in studies with large samples and preventative approaches.

“The results of studies with large samples (Kigerl et al., 2022) and preventative approaches (Sheppard et al., 2024) demonstrate that the transition from traditional statistical models to complex AI/ML architectures yields only marginal improvements.”

Virtually identical performance was documented between complex techniques and traditional approaches such as logistic regression, especially when the sample size is large, or the evaluated scenario reduces predictive variability.

“Virtually identical performance was documented between complex techniques and traditional approaches such as logistic regression, especially when the sample size is large, or the evaluated scenario reduces predictive variability (Kigerl et al., 2022;Sheppard et al., 2024).”