Machine Advice with a Warning about Machine Limitations: Experimentally Testing the Solution Mandated by the Wisconsin Supreme Court
Open accessAbstract
The Wisconsin Supreme Court allows machine advice in the courtroom only if accompanied by a series of warnings. We test 878 US lay participants with jury experience on fifty past cases where we know ground truth. The warnings affect their estimates of the likelihood of recidivism and their confidence, but not their decision whether to grant bail. Participants do not get better at identifying defendants who recidivated during the next two years. Results are essentially the same if participants are warned in easily accessible language, and if they are additionally informed about the low accuracy of machine predictions. The decision to grant bail is also unaffected by the warnings mandated by the Supreme Court if participants do not first decide without knowing the machine prediction. Oversampling cases where defendants committed violent crime does not change results either, whether coupled with machine predictions for general or for violent crime. Giving participants feedback and incentivizing them for finding ground truth has a small, weakly significant effect. The effect becomes significant at conventional levels when additionally using strong graphical warnings. Then participants are less likely to follow the advice. But the effect is counterproductive: they follow the advice less if it actually is closer to ground truth.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Comparing results across models is not straightforward because there is a wide variety of methods used, ranging from the type of artificial intelligence chosen to how exactly recidivism is defined or at what stage of the trial the prediction is made.
“However, recent literature warns that comparing results across models is not straightforward.This is because there is a wide variety of methods used, ranging from the type of artificial intelligence chosen to how exactly recidivism is defined or at what stage of the trial the prediction is made (Eaglin, 2016;Engel & Grgić-Hlača, 2021;Greene et al., 2022;Kigerl et al., 2022;Travaini et al., 2022).”
From Scoping review on the integration of artificial intelligence and machine learning with structured instruments to predict juvenile recidivism (2026) · cites this paper as Engel & Grgić-Hlača, 2021
The Introduction distinguishes general from violent recidivism, and the additional details (p. 333) distinguish predictions used in sentencing from those used in bail decisions. These passages support variation in the outcome predicted and the decision stage, but the paper does not establish the claimed difficulty of comparing results across AI models or attribute it to a variety of AI methods. (LLM, conf=0.83)
Experimental — accuracy not yet measuredGeneric policies on human oversight do not guarantee effective control unless they define when and how one may override the system, what justifications are required, and what audits are applied; otherwise, oversight may become a mere formality.
“Furthermore, generic policies on human oversight do not guarantee effective control unless they define when and how one may override the system, what justifications are required, and what audits are applied; otherwise, oversight may become a mere formality (Green, 2022;Engel & Grgić-Hlača, 2021).”
From Scoping review on the integration of artificial intelligence and machine learning with structured instruments to predict juvenile recidivism (2026) · cites this paper as Engel & Grgić-Hlača, 2021
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Prolific.ac—A subject pool for online experiments
Palan, S., & Schitter, C. (2018) · Journal of Behavioral and Experimental Finance · doi:10.1016/j.jbef.2017.12.004 · Cited by 3,450 · see citing works
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · see citing works
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · see citing works
The accuracy, fairness, and limits of predicting recidivism
Dressel, J., & Farid, H. (2018) · Science Advances · doi:10.1126/sciadv.aao5580 · Cited by 1,071 · see citing works
Text and Graphic Warnings on Cigarette Packages
Hammond, D., Fong, G. T., Borland, R., Cummings, K. M., McNeill, A., & Driezen, P. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2006.11.011 · Cited by 434 · see citing works
Assessing Risk Assessment in Action
Stevenson, M. T. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3016088 · Cited by 195 · see citing works
An impact assessment of machine learning risk forecasts on parole board decisions and recidivism
Berk, R. (2017) · Journal of Experimental Criminology · doi:10.1007/s11292-017-9286-2 · Cited by 169 · see citing works
Flagged first, then by citations
Research works printed but not linked: 10
- ---2019. Machine Learning Risk Assessments in Criminal Justice Settings. New York: Springer.
- Doshi-Velez, Finale & Been Kim. 2017. Towards a Rigorous Science of Interpretable Machine Learning. arXiv preprint arXiv:1702.08608. · arXiv
- Dwork, Cynthia, Moritz Hardt, Toniann Pitassi, Omer Reingold & Richard Zemel. 2012. Fairness Through Awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, 214-226.
- Farabee, David. 2005. Rethinking Rehabilitation: Why Can't We Reform Our Criminals? Washington, DC: AEI Press.
- N. C. J. Law Technol. 75. Grgi c-Hla ca, Nina, Christoph Engel & Krishna P. Gummadi. 2019. Human Decision Making with Machine Assistance: An Experiment on Bailing and Jailing. In Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 1-25.
- -Hla ca, Nina, Muhammad Bilal Zafar, Krishna P. Gummadi & Adrian Weller. 2018. Beyond Distributive Fairness in Algorithmic Decision Making: Feature Selection for Procedurally Fair Learning. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, 51-60.
- Groves, Robert M., Floyd J. Fowler Jr, Mick P. Couper, James M. Lepkowski, Eleanor Singer & Roger Tourangeau. 2011. Survey Methodology, vol. 561. Hoboken, NJ: John Wiley & Sons.
- Raghu, Maithra, Katy Blumer, Greg Corrado, Jon Kleinberg, Ziad Obermeyer & Sendhil Mullainathan. 2019. The Algorithmic Automation Problem: Prediction, Triage, and Human Effort. arXiv preprint arXiv:1903.12220. · arXiv
- Tan, Sarah, Julius Adebayo, Kori Inkpen & Ece. Kamar 2018. Investigating Humanþ Machine Complementarity for Recidivism Predictions. arXiv preprint arXiv:1808.09123. · arXiv
- Wilder, Bryan, Eric Horvitz & Ece. Kamar 2020. Learning to Complement Humans. arXiv preprint arXiv:2005.00582. · arXiv
Cited by
Sorted by citations
Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
Dahl, M., Magesh, V., Suzgun, M., & Ho, D. E. (2024) · Journal of Legal Analysis · doi:10.1093/jla/laae003 · Cited by 288 · see citing works
Argudo Palacios, A., & Mechraoui, A. (2026) · Cuadernos de RES PUBLICA en derecho y criminología · doi:10.46661/respublica.13244
“Furthermore, generic policies on human oversight do not guarantee effective control unless they define when and how one may override the system, what justifications are required, and what audits are applied; otherwise, oversight may become a mere formality (Green, 2022; Engel & Grgić-Hlača, 2021).”
- Authors
- Engel, C., & Grgić-Hlača, N.
- Affiliations
- Max Planck Institute for Behavioral Economics (DE)
- Max Planck Institute for Software Systems (DE)
- Corresponding author
- Engel, Christoph
- Venue
- Journal of Legal Analysis
- Publisher
- Oxford University Press (OUP)
- Published
- 2021-01-01
- Volume
- 13
- Issue
- 1
- Pages
- 284-340
- ISSN
- 1946-5319 (ISSN-L)
- 2161-7201
- OpenAlex
- W3194879917
- Cited by
- 16
- References
- 7
- Citations by year
- 2026: 3 · 2025: 1 · 2024: 7 · 2023: 1 · 2022: 4
- FWCI
- 9.97
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- The Journal of Legal Analysis (publishedVersion, cc-by-nc)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion, other-oa)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion, other-oa)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
- Social Sciences › Law › Jury Decision Making Processes
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- algorithmic decision-making, Wisconsin Supreme Court, bail decisions, recidivism prediction, algorithmic transparency, violent crime, human-machine decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
