CoChat
Sign Up
Cited by 116Ethics and Social Impacts of AIDecision-Making and Behavioral EconomicsArtificial Intelligence in Law

The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index; none of the 23 sources its claims cite is checked for support yet.

Human Decision Making with Machine Assistance: An Experiment on Bailing and Jailing

Open access
Grgić-Hlača, Nina, Engel, Christoph, Gummadi, Krishna P.·2019·Proceedings of the ACM on Human-Computer Interaction
DOI 10.1145/3359280
Open this paper in CoChat — free accountOpen PDF

Abstract

Much of political debate focuses on the concern that machines might take over. Yet in many domains it is much more plausible that the ultimate choice and responsibility remain with a human decision-maker, but that she is provided with machine advice. A quintessential illustration is the decision of a judge to bail or jail a defendant. In multiple jurisdictions in the US, judges have access to a machine prediction about a defendant's recidivism risk. In our study, we explore how receiving machine advice influences people's bail decisions. We run a vignette experiment with laypersons whom we test on a subsample of cases from the database of this prediction tool. In study 1, we ask them to predict whether defendants will recidivate before tried, and manipulate whether they have access to machine advice. We find that receiving machine advice has a small effect, which is biased in the direction of predicting no recidivism. In the field, human decision makers sometimes have a chance, after the fact, to learn whether the machine has given good advice. In study 2, after each trial we inform participants of ground truth. This does not make it more likely that they follow the advice, despite the fact that the machine is (on average) slightly more accurate than real judges. This also holds if initially the advice is mostly correct, or if it initially is mostly to predict (no) recidivism. Real judges know that their decisions affect defendants' lives. They may also be concerned about reelection or promotion. Hence a lot is at stake. In study 3 we emulate high stakes by giving participants a financial incentive. An incentive to find the ground truth, or to avoid false positive or false negatives, does not make participants more sensitive to machine advice. But an incentive to follow the advice is effective.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

In three vignette experiments using COMPAS/ProPublica bail cases, machine advice had only a small effect on lay participants’ recidivism predictions, and the effect was biased toward predicting no recidivism. Feedback about ground truth did not increase reliance on advice, while only incentives tied to following advice increased sensitivity to it.

Key findings

  • Study 1: participants changed their prediction after receiving machine advice in 390 of 5,150 cases; of those changes, 340 followed the advice and 50 opposed it. The estimated population change rate was 7.6% (p < .001).
  • Advice was asymmetric: the probability of shifting toward “no recidivism” when advised to do so was 8.45%, versus 5.7% for shifting toward “recidivism” when advised to do so. Participants were less accurate without advice (60.2%) than with advice (61.6%), and both were below the decision aid’s 68% accuracy.
  • Study 2: giving feedback about ground truth had no discernible effect on predictions or propensity to change after advice; varying whether advice was initially mostly correct/incorrect or mostly bail/jail advice also produced no significant effects.
  • Study 3: incentives to match ground truth or avoid false positives/false negatives did not increase reliance on advice. Incentives to align with machine advice did increase changing decisions after advice, but stronger incentives were not more effective than weak ones (Wald test p = .495); strong alignment incentives raised accuracy from 61.6% to 64.5% and lowered false positive rate from .348 to .296.

Methods

  • Three online vignette experiments on Prolific with U.S. participants who self-reported jury service. Study 1 had 103 participants who passed attention checks; Studies 2 and 3 each had 100, with 20 participants per treatment.
  • Stimuli were 50 randomly selected cases from the ProPublica COMPAS dataset of 7,214 defendants in Broward County, Florida, 2013–2014. The authors trained a logistic regression classifier to predict two-year recidivism; its accuracy was 68%, close to COMPAS.
  • Study 1 used a within-subjects design comparing pre-advice and post-advice predictions. Study 2 added between-subjects feedback and case-order treatments. Study 3 added between-subjects financial incentive treatments. Analyses used linear probability models with participant random effects.

Limitations

  • The paper states the main limitation is the subject pool: for ethical and control reasons, the authors could not study real judges deciding real cases, and testing real judges on hypothetical cases would be confounded by varied experience. They used laypersons eligible for jury duty, most of whom had served.
  • The paper states that feedback and incentives only emulate courtroom conditions: real judges do not receive feedback for every case, never immediately after deciding, and normally do not experience immediate financial consequences. Thus the manipulations estimate upper bounds.
  • Evident from the text: samples were more Caucasian, more educated, and more liberal than the U.S. population; only 74% (Study 1), 79% (Study 2), and 72% (Study 3) reported actual jury service despite being recruited from a jury-service pool.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3359280

Cites

72 of 72 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 of 23 cited sources checked · 0 supported · 0 flagged

Show all 46 claims

  • It may state two separate claims as one. 21
  • Probably not a substantial claim. 26
  • The quote only partly states this claim. 1
42 of 42

  • The paper relies on this without support.

    There is no literature that systematically investigates the effect of machine advice on the accuracy of decisions made by humans.

    “To the best of our knowledge, there is no literature that systematically investigates the effect of machine advice on the accuracy of decisions made by humans.”

  • The paper relies on this without support.

    Real judges do not receive feedback for each and every case, and they never receive feedback immediately after making a decision.

    “Real judges do not receive feedback for each and every case, and they never receive feedback immediately after making a decision.”

  • The paper relies on this without support.

    Real judges normally do not experience immediate financial consequences of their decision to bail or jail a defendant.

    “Furthermore, real judges normally do not experience immediate financial consequences of their decision to bail or jail a defendant.”

  • The paper relies on this without support.

    COMPAS produces a risk score on a scale from 0 to 10.

    “COMPAS produces a”

  • The paper relies on this without support.

    The COMPAS tool is not hypothetical, but actually used for consequential human decisions.

    “The tool is not hypothetical, but actually used for consequential human decisions.”

  • The paper relies on this without support.

    The COMPAS tool is managed by a commercial provider that has every interest in improving accuracy.

    “The tool is managed by a commercial provider that has every interest in improving accuracy.”

  • The paper relies on this without support.

    The law wants the bail decision to be based on the risk of recidivism, and the machine advice exclusively covers this risk.

    “However, the law wants this decision to be based on the risk of recidivism, and the machine advice exclusively covers this risk.”

  • The paper relies on this without support.

    Laypersons' decisions on the vignettes are a valid proxy for judges' decisions in the judicial domain.

    “In Study 1, we test laypersons on these vignettes, both without and with the benefit of advice.”
    “Using this approach, we are the first to provide causal evidence about the effect of machine advice on the accuracy of human decision making, in the judicial domain.”

  • The paper relies on this without support.

    The ground truth labels in the COMPAS dataset are accurate.

    “For all cases we know ground truth.”

  • The paper relies on this without support.

    The decision aid used in the study has an average accuracy of 68%.

    “our decision aid is only mildly reliable, with average accuracy of 68%, close to the accuracy of the COMPAS tool [4,24].”

  • The paper relies on this without support.

    Multiple US States give judges access to COMPAS predictions of recidivism risk for bail decisions.

    “This is not a contrived example; multiple US States give judges access to precisely this prediction, based on the commercial product COMPAS, sold by Northpointe.”

  • The authors state this as an assumption.

    The paper is the first to provide causal evidence about the effect of machine advice on the accuracy of human decision making in the judicial domain.

    “Using this approach, we are the first to provide causal evidence about the effect of machine advice on the accuracy of human decision making, in the judicial domain.”

  • The paper relies on this without support.

    The financial incentives used in Study 3 emulate higher stakes in the field.

    “A financial incentive (meant to emulate higher stakes) only matters if it is tied to following the advice, not if it is tied to the quality of the advice, or to its direction.”

  • The paper relies on this without support.

    How participants say they would decide, were they given authority, reflects how they would actually decide.

    “we could only ask how they would decide, were they to be given authority.”

  • The paper relies on this without support.

    Laypersons randomly drawn from the general population are an adequate substitute for real judges in studying decision making with machine advice.

    “This is why we turn to laypersons who are randomly drawn from the general population.”
15 of 15

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Net agreement gain is described as resolved disagreements

    Minor

    The 290 figure is the net gain in agreement: 340 participants switched to the advice, while 50 switched away from it. Thus 340 of the original disagreements were resolved, not 290.

  • The machine predicts recidivism in 16 cases, not 18

    Minor

    Table 3 records 1,339 + 309 = 1,648 pre-advice responses for cases where the machine predicts recidivism. With 103 participants responding to each case, that represents 16 cases, contradicting the stated 18.

2 of 2

Citations

Cites

Flagged first, then by citations

Show all 72 references

Research works printed but not linked: 5

  1. Sam Corbett-Davies and Sharad Goel . 2018. The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning. arXiv preprint arXiv:1808.00023 ( 2018 ). Sam Corbett-Davies and Sharad Goel. 2018. The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning. arXiv preprint arXiv:1808.00023 (2018). · arXiv
  2. DRI. 2012 . DRI 2012 National Poll on the Civil Justice System . tempurlhttp://www.dri.org/advocacy/center-for-law-and-public-policy/poll/2012-poll/showURL DRI. 2012. DRI 2012 National Poll on the Civil Justice System . tempurlhttp://www.dri.org/advocacy/center-for-law-and-public-policy/poll/2012-poll/showURL
  3. Brandon L Garrett and John Monahan . 2019. Judging Risk . California Law Review ( 2019 ). Brandon L Garrett and John Monahan. 2019. Judging Risk. California Law Review (2019).
  4. Gabriele Paolacci , Jesse Chandler , and Panagiotis G Ipeirotis . 2010. Running Experiments on Amazon Mechanical Turk . Judgment and Decision Making ( 2010 ). Gabriele Paolacci, Jesse Chandler, and Panagiotis G Ipeirotis. 2010. Running Experiments on Amazon Mechanical Turk . Judgment and Decision Making (2010).
  5. Pew Research Center. 2016. 2016 Party Identification Detailed Tables . tempurlhttp://www.people-press.org/2016/09/13/2016-party-identification-detailed-tables/showURL Pew Research Center. 2016. 2016 Party Identification Detailed Tables . tempurlhttp://www.people-press.org/2016/09/13/2016-party-identification-detailed-tables/showURL

Cited by

Sorted by citations

Items 101–116 of 116

Related

See all

Metadata

Authors
Grgić-Hlača, N., Engel, C., & Gummadi, K. P.
Affiliations
  • Max Planck Institute for Software Systems (DE)
  • Max Planck Institute for Behavioral Economics (DE)
ORCID
Venue
Proceedings of the ACM on Human-Computer Interaction
Publisher
Association for Computing Machinery (ACM)
Published
2019-11-07
Volume
3
Issue
CSCW
Pages
1-25
ISSN
2573-0142 (ISSN-L)
DOI
10.1145/3359280
OpenAlex
W2974075815
Cited by
113
References
72
Citations by year
2026: 14 · 2025: 12 · 2024: 20 · 2023: 16 · 2022: 25 · 2021: 18 · 2020: 7 · 2019: 1
FWCI
23.16
Type
journal article
Language
en
Open access
Licence
Locations
  • Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by)
  • SSRN Electronic Journal (publishedVersion)
  • EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
  • MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
  • MPG.PuRe (Max Planck Society) (submittedVersion)
Funders
European Research Council (789373)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
  • Social Sciences › Law › Artificial Intelligence in Law
Keywords
human decision-making, bail decisions, recidivism prediction, algorithmic decision support, judicial decision-making, financial incentives, trust in algorithms, human-machine interaction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review