Fragile Algorithms and Fallible Decision-Makers: Lessons from the Justice System

Most sources recognize that the broad promises of AI-enabled risk assessment have not been fully realized yet.

“Although most sources recognize that the broad promises of AI-enabled risk assessment have not been fully realized yet [6]”

The specific biases identified by the authors of the sources are race, gender, age, and economic status.

“The specific biases identified by the authors of our sources are race [14,22,33,46], gender [6,14,47], age [48,49], and economic status [17,[50][51][52].”

Most ARAs have been built from relatively simple models, with logistic regressions being the most commonly referenced in the literature.

“The burden of training models on datasets of limited size [6] and on using models that are relatively interpretable [17] mean that most ARAs have been built from relatively simple models, with logistic regressions being the most commonly referenced in the literature.”

Currently, many ARAs are derived from simple ML techniques.

“While there is a perception that deep learning will be used as a risk assessment tool in courtrooms in the future [22], currently, many ARAs are derived from simple ML techniques [6].”

AI-derived tools might be better predictors of recidivism than judges and can be less biased than human actors.

“While AI-derived tools might be better predictors of recidivism than judges [6] and can be less biased than human actors [6], these benefits are meaningless if the recommendations of AI systems are ignored.”