Limits of Interventions in AI-Assisted Decision Making
Open accessAbstract
As AI decision-support systems are increasingly deployed in high-stakes domains, understanding how to prevent automation bias is critical. This study examines whether three common interventions, including accountability requirements, consequence framing, and sequential disclosure of AI recommendations, reduce over-reliance on AI in socially consequential decisions. We conducted a controlled experiment using simulated case studies of AI-assisted child maltreatment risk assessment. We found that accountability requirements increased deliberation time but did not reduce over-reliance on AI or improve alignment with expert judgment. Sequential disclosure revealed that participants frequently abandoned an expert-aligned initial assessment in favor of the AI recommendation, producing no meaningful improvement in appropriate reliance. Consequence framing, by contrast, reduced over-reliance without increasing under-reliance and produced the highest rate of decisions consistent with expert judgment. Across conditions, greater deliberation and more elaborate moral reasoning were associated with increased AI reliance and lower alignment with expert judgment. Furthermore, results from the sequential disclosure condition suggest that AI recommendations can override the cognitive anchoring effect of a participant’s own prior judgment. These findings highlight the limits of deliberation-based interventions and point to consequence framing as a more promising approach for calibrating human reliance on AI in high-stakes settings.
AI snapshot
Written from the paper’s full text.
TL;DR
In a controlled child-welfare analogue experiment, accountability and sequential disclosure failed to reduce AI over-reliance, while consequence framing improved expert-aligned decisions without increasing deliberation. The paper concludes that observable deliberation is not a reliable safeguard and that decision framing may better calibrate reliance.
Key findings
- Accountability increased deliberation time to about 39 seconds per decision versus 29 seconds in No AI and baseline, but did not reduce AI following: 69% [56,80] in accountability versus 54% [43,66] in No AI and 74% [63,83] in baseline.
- Consequence framing produced the highest expert-majority alignment at 67.1%, compared with 61.7% baseline, 59.8% accountability, and 61.0% No AI, without a credible increase in AI following relative to No AI (61% [48,74]; b = 0.04 [-0.54, 0.64]).
- In sequential disclosure, among participants who initially disagreed with AI (n = 164 disagreement instances), switching was 42.9% when AI was not expert-aligned versus 28.7% when it was; on unanimous expert cases, switching toward non-expert-aligned AI credibly exceeded switching toward expert-aligned AI (contrast = 0.31 [0.03, 0.61]). Final decisions after revision were 58.8% [53.6, 64.0] expert-aligned, below initial unaided decisions at 61.0% [55.9, 66.1].
- Greater deliberation and moral reasoning were associated with more AI reliance and lower expert alignment: within accountability, longer decision times predicted higher AI following (b = 0.63 [0.12, 1.16]); moral framing negatively predicted appropriate reliance (b = -0.62 [-1.26, -0.02]); case specificity negatively predicted expert alignment (b = -1.07 [-2.11, -0.07]); 98.1% of justifications did not mention AI.
Methods
- Between-subjects experiment with U.S. Prolific participants: 280 recruited, final analytic sample 224 (57 baseline, 57 consequence, 52 accountability, 58 No AI), contributing 1,344 case-level observations. Participants reviewed six child welfare vignettes randomly assigned from 16 fictional cases.
- AI recommendations came from a logistic regression classifier trained on NCANDS data with 78% held-out accuracy, displayed as HIGH RISK or LOW RISK; participants were told it was about 80% accurate. Expert majority judgment from three child welfare experts served as benchmark (Fleiss’ κ = 0.59; unanimous on 13 of 16 vignettes).
- Conditions: baseline, consequence framing, accountability with written justification, and sequential disclosure/No AI (initial decision before AI, with revision only if initial decision differed). Bayesian multilevel models used brms with participant- and case-specific random intercepts, 4 chains of 4,000 iterations; justification texts were coded via Claude API and validated against a human-coded subset of 30.
Limitations
- Stated: participants were lay adults recruited online, not child welfare professionals; the CPS-retained descriptive check was similar but the professional-exposure subgroup was small and cannot support inferential claims. The expert benchmark had moderate agreement and represents majority judgment, not ground truth. The No AI reference came from sequential-disclosure participants’ initial decisions rather than a fully independent naive group. Justification coding used AI-generated labels validated on a human-coded subset and identifies correlations rather than causes. The explainable, unbranded logistic-regression classifier may elicit different reliance from LLM-based, branded, or less interpretable systems.
- Evident from the text: the design cannot distinguish among anchoring, perceived authority, attentional reframing, affective responses, or doubts about initial review as mechanisms. The task is a controlled analogue with fictional vignettes, so it does not show how practicing caseworkers would behave.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
9 of 43 cited sources checked · 7 supported · 0 flagged · accuracy not yet measured
- Interpretationcitation
If institutional safeguards designed to promote careful reasoning instead formalize deference to algorithmic outputs, the consequence is a transfer of effective decision authority from accountable professionals to opaque statistical models.
“If institutional safeguards designed to promote careful reasoning instead formalize deference to algorithmic outputs, the consequence is not merely suboptimal prediction; it is a transfer of effective decision authority from accountable professionals to opaque statistical models, precisely the outcome that human oversight is meant to prevent (Green 2022).”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Green 2022 source
- Premisecitation
Professionals bear full professional and legal accountability for outcomes the algorithm influenced but does not share responsibility for.
“Empirical evidence from real clinical settings shows that when AI outputs conflict with professional judgment but provide no underlying reasoning, professionals often cannot evaluate which is correct (Lebovitz, Lifshitz-Assaf, and Levina 2022), yet they bear full professional and legal accountability for outcomes the algorithm influenced but does not share responsibility for (Busuioc 2021).”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Busuioc 2021 source
In “Consequences for Algorithmic Decision-Making” (page unavailable), Busuioc says public-sector decision-makers remain responsible for outcomes when they rely on algorithmic outputs. The passage does not establish that professionals generally bear full professional and legal accountability, or that no other party shares responsibility. (LLM, conf=0.86)
Experimental — accuracy not yet measured
- Premisecitation
Even when AI operates at the level of recommendation rather than direct action, the mere presence of an algorithmic suggestion can systematically shift human judgment toward the algorithm's conclusion.
“Even when AI operates at the level of recommendation rather than direct action, the mere presence of an algorithmic suggestion can systematically shift human judgment toward the algorithm's conclusion, a phenomenon known as automation bias (Goddard, Roudsari, and Wyatt 2012;Alon-Barkat and Busuioc 2023).”
Quote in the paperSubstantial claim2 of 2 sources checked- cites Goddard, Roudsari, and Wyatt 2012 source
- cites Alon-Barkat and Busuioc 2023 source
The Introduction (p. 1) describes algorithms as recommendations that support, rather than replace, human decisions, and the Automation Bias section (p. 3) describes undue deference to such advice. But the paper’s own findings (Automation Bias, p. 12; Selective Adherence, p. 13) do not show that the mere presence of an algorithmic suggestion generally shifts judgment toward it: they find no overall automation bias, and adherence is selective. (LLM, conf=0.88)
Experimental — accuracy not yet measured
- Premisecitation
When AI outputs conflict with professional judgment but provide no underlying reasoning, professionals often cannot evaluate which is correct.
“Empirical evidence from real clinical settings shows that when AI outputs conflict with professional judgment but provide no underlying reasoning, professionals often cannot evaluate which is correct (Lebovitz, Lifshitz-Assaf, and Levina 2022), yet they bear full professional and legal accountability for outcomes the algorithm influenced but does not share responsibility for (Busuioc 2021).”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Lebovitz, Lifshitz-Assaf, and Levina 2022 source
- Premisecitation
The institutional mechanisms most commonly proposed to counteract automation bias may be less effective than assumed.
“Critically, the institutional mechanisms most commonly proposed to counteract this shift may be less effective than assumed (Green 2022;Agudo et al. 2024;Coskun and Leidner 2024;Stevenson and Doleac 2024).”
Quote in the paperSubstantial claim2 of 3 sources checked
- Premisecitation
Bučinca, Malaya, and Gajos (2021) demonstrated that cognitive forcing functions reduce over-reliance on AI relative to standard simultaneous disclosure.
“Prior work provides some support for disclosure-based interventions : Bučinca, Malaya, and Gajos (2021) demonstrated that cognitive forcing functions reduce over-reliance on AI relative to standard simultaneous disclosure.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Bučinca, Malaya, and Gajos (2021) source
- Premisecitation
In AI-assisted decision-making, the algorithmic recommendation functions as an anchor.
“In AIassisted decision-making, the algorithmic recommendation functions as such an anchor (Rastogi et al. 2022).”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Rastogi et al. 2022 source
- Findingown data
Consequence framing produced the highest rate of decisions consistent with expert judgment.
“Consequence framing, by contrast, reduced overreliance without increasing under-reliance and produced the highest rate of decisions consistent with expert judgment.”
Quote in the paperSubstantial claim - Findingown data
Sequential disclosure produced no meaningful improvement in appropriate reliance.
“Sequential disclosure revealed that participants frequently abandoned an expertaligned initial assessment in favor of the AI recommendation, producing no meaningful improvement in appropriate reliance.”
Quote in the paperSubstantial claim - Findingown data
Participants in the sequential disclosure condition sometimes revised toward AI even when it diverged from expert-majority judgment.
“Participants in the sequential disclosure condition sometimes revised toward AI even when it diverged from expert-majority judgment, and more elaborate moral reasoning was associated with lower expert alignment.”
Quote in the paperSubstantial claim - Findingown data
Consequence framing reduced over-reliance.
“Consequence framing, by contrast, reduced overreliance without increasing under-reliance and produced the highest rate of decisions consistent with expert judgment.”
Quote in the paperSubstantial claim - Findingown data
Across conditions, more elaborate moral reasoning was associated with increased AI reliance.
“Across conditions, greater deliberation and more elaborate moral reasoning were associated with increased AI reliance and lower alignment with expert judgment.”
Quote in the paperSubstantial claim - Findingown data
Across conditions, greater deliberation was associated with increased AI reliance.
“Across conditions, greater deliberation and more elaborate moral reasoning were associated with increased AI reliance and lower alignment with expert judgment.”
Quote in the paperSubstantial claim - Findingown data
Across conditions, more elaborate moral reasoning was associated with lower alignment with expert judgment.
“Across conditions, greater deliberation and more elaborate moral reasoning were associated with increased AI reliance and lower alignment with expert judgment.”
- Also stated in: § Introduction · plain
“Participants in the sequential disclosure condition sometimes revised toward AI even when it diverged from expert-majority judgment, and more elaborate moral reasoning was associated with lower expert alignment.”
Quote in the paperSubstantial claim - Also stated in: § Introduction · plain
- Findingown data
Consequence framing did not increase under-reliance.
“Consequence framing, by contrast, reduced overreliance without increasing under-reliance and produced the highest rate of decisions consistent with expert judgment.”
Quote in the paperSubstantial claim - Findingown data
Accountability requirements did not improve alignment with expert judgment.
“We found that accountability requirements increased deliberation time but did not reduce over-reliance on AI or improve alignment with expert judgment.”
Quote in the paperSubstantial claim - Findingown data
Across conditions, greater deliberation was associated with lower alignment with expert judgment.
“Across conditions, greater deliberation and more elaborate moral reasoning were associated with increased AI reliance and lower alignment with expert judgment.”
Quote in the paperSubstantial claim - Findingown data
In the sequential disclosure condition, AI recommendations can lead participants to revise a prior independent judgment.
“Furthermore, results from the sequential disclosure condition show that AI recommendations can lead participants to revise a prior independent judgment.”
Quote in the paperSubstantial claim - Findingown data
More elaborate moral reasoning was associated with lower expert alignment.
“Participants in the sequential disclosure condition sometimes revised toward AI even when it diverged from expert-majority judgment, and more elaborate moral reasoning was associated with lower expert alignment.”
Quote in the paperSubstantial claim - Findingown data
In the sequential disclosure condition, participants frequently abandoned an expert-aligned initial assessment in favor of the AI recommendation.
“Sequential disclosure revealed that participants frequently abandoned an expertaligned initial assessment in favor of the AI recommendation, producing no meaningful improvement in appropriate reliance.”
Quote in the paperSubstantial claim - Findingown data
Accountability requirements did not reduce over-reliance on AI.
“We found that accountability requirements increased deliberation time but did not reduce over-reliance on AI or improve alignment with expert judgment.”
- Also stated in: § Introduction · plain
“Requiring written justification increased deliberation time but did not reduce AI deference.”
Quote in the paperSubstantial claim - Also stated in: § Introduction · plain
- Findingown data
Accountability requirements increased deliberation time.
“We found that accountability requirements increased deliberation time but did not reduce over-reliance on AI or improve alignment with expert judgment.”
- Also stated in: § Introduction · plain
“Requiring written justification increased deliberation time but did not reduce AI deference.”
Quote in the paperSubstantial claim - Also stated in: § Introduction · plain
- Findingown data
None of the interventions credibly moderated the association between AI-expert alignment and participants' tendency to follow AI.
“Across all conditions, none of the interventions credibly moderated the association between AI-expert alignment and participants' tendency to follow AI.”
- Also stated in: § What They Decided: AI Reliance Across Conditions · plain
“No condition made participants meaningfully more likely to follow AI when it was expert-aligned and to deviate when it was not.”
Quote in the paperSubstantial claim - Also stated in: § What They Decided: AI Reliance Across Conditions · plain
- Findingown data
Content coding of the 312 written justifications revealed that 98.1% were framed in purely case-based terms, with no reference to the AI recommendation.
“Content coding of the 312 written justifications revealed that 98.1% were framed in purely case-based terms, with no reference to the AI recommendation.”
Quote in the paperSubstantial claim - Findingown data
The consequence condition produced the highest rate of expert-aligned decisions (67.1%), compared with 61.7% in baseline, 59.8% in accountability, and 61.0% in No AI.
“The consequence condition produced the highest rate of expert-aligned decisions (67.1%), compared with 61.7% in baseline, 59.8% in accountability, and 61.0% in No AI (Figure 2).”
Quote in the paperSubstantial claim - Findingown data
The net result was a slight deterioration in aggregate decision quality: final decisions after the revision process (58.8% [53.6, 64.0]) fell below initial unaided rates (61.0% [55.9, 66.1]).
“The net result was a slight deterioration in aggregate decision quality: final decisions after the revision process (58.8% [53.6, 64.0]) fell below initial unaided rates (61.0%[55.9, 66.1]).”
Quote in the paperSubstantial claim - Findingown data
On cases where the AI recommendation did not match expert majority, consequence participants did not show a credible decrease in expert-aligned choices relative to No AI (b = -0.04 [-0.61, 0.51]).
“consequence participants did not show this pattern (b = -0.04[-0.61, 0.51]), performing at a level indistinguishable from the No AI reference on these cases.”
Quote in the paperSubstantial claim - Findingown data
In the No AI condition, participants spent noticeably longer on cases where AI and expert assessments diverged.
“In the No AI condition, participants spent noticeably longer on cases where AI and expert assessments diverged.”
Quote in the paperNot a substantial claim - Findingown data
The consequence condition showed a modest, noncredible increase in decision latency (31 s [26,37]) that overlapped substantially with baseline.
“The consequence condition showed a modest, noncredible increase (31 s [26,37]) that overlapped substantially with baseline (Figure 3).”
Quote in the paperNot a substantial claim - Findingown data
On cases where the AI recommendation did not match expert majority, accountability participants were credibly less likely to make an expert-aligned choice than those deciding without AI (b = -0.55 [-1.13, -0.02]).
“condition participants (b = -0.55[-1.13, -0.02])”
Quote in the paperSubstantial claim - Findingown data
Participants who saw the AI recommendation followed it more often than those who decided without it.
“Participants who saw the AI recommendation followed it more often than those who decided without it.”
Quote in the paperSubstantial claim - Findingown data
Decision latency varied by condition.
“Decision latency varied by condition.”
Quote in the paperSubstantial claim - Findingown data
Revision decisions were fast, with a mean revision page time of approximately 12 seconds across all disagreement instances.
“Revision decisions were fast, with a mean revision page time of approximately 12 seconds across all disagreement instances.”
Quote in the paperNot a substantial claim - Findingown data
The consequence condition showed no credible increase in AI following relative to No AI (61% [48,74]; b = 0.04 [-0.54, 0.64]).
“The consequence condition showed no credible increase in AI following relative to No AI (61% [48,74]; b = 0.04 [-0.54, 0.64]).”
Quote in the paperSubstantial claim - Findingown data
Within the accountability condition, longer decision times predicted higher AI following (b = 0.63 [0.12, 1.16]).
“Moreover, within the accountability condition, longer decision times predicted higher AI following (b = 0.63 [0.12, 1.16]; see note in Table 2).”
Quote in the paperNot a substantial claim - Findingown data
Within the accountability condition, longer decision times were associated with higher AI reliance.
“Within the accountability condition, longer decision times were associated with higher AI reliance.”
Quote in the paperSubstantial claim - Findingown data
In the No AI condition, participants' decisions matched the AI recommendation 54% [43,66] of the time.
“In the No AI condition, participants' decisions matched the AI recommendation 54% [43,66] of the time, reflecting the baseline rate of agreement between independent judgment and the algorithm on the same cases.”
Quote in the paperSubstantial claim - Findingown data
Agreement on the binary expert ratings was moderate (Fleiss' κ = 0.59).
“Agreement on the binary ratings was moderate (Fleiss' κ = 0.59).”
Quote in the paperNot a substantial claim - Findingown data
On cases where the AI recommendation did not match expert majority, baseline participants were credibly less likely to make an expert-aligned choice than those deciding without AI (b = -0.99 [-1.55, -0.44]).
“On cases where the AI recommendation did not match expert majority, baseline participants were credibly less likely to make an expert-aligned choice than those deciding without AI (b = -0.99[-1.55, -0.44]), as were accountability”
Quote in the paperSubstantial claim - Findingown data
On unanimous (3-0) cases, the probability of switching toward an AI recommendation that was not expert-aligned credibly exceeded the probability of switching when the AI recommendation was expert-aligned (contrast = 0.31 [0.03, 0.61]; P(not aligned > aligned) = 0.985).
“On unanimous (3-0) cases, where rater consensus was highest, the probability of switching toward an AI recommendation that was not expert-aligned credibly exceeded the probability of switching when the AI recommendation was expert-aligned (contrast = 0.31 [0.03, 0.61]; P (not aligned > aligned) = 0.985).”
Quote in the paperSubstantial claim - Interpretationreasoning
The human and the algorithm are frequently not reasoning from the same evidence, even when they receive the same case file.
“Consequently, the human and the algorithm are frequently not reasoning from the same evidence, even when they receive the same case file.”
Quote in the paperSubstantial claim - Study design and conductreasoning
The design does not establish any one mechanism for the results.
“These results are compatible with multiple explanations, including anchoring, perceived authority, and attentional reframing; the design does not establish any one mechanism.”
Quote in the paperSubstantial claim - Premisecitation
When decision-makers know or can infer the evaluator's preferred conclusion, accountability can produce the opposite effect, motivating justification of the expected position.
“When decision-makers know or can infer the evaluator's preferred conclusion, accountability can produce the opposite effect, motivating justification of the expected position (Tetlock 1983;Lerner and Tetlock 1999).”
Quote in the paperSubstantial claim0 of 2 sources checked - Interpretationown data
Consequence framing produced the highest expert-majority alignment without longer deliberation.
“Consequence framing produced the highest expert-majority alignment without longer deliberation.”
Quote in the paperSubstantial claim - Premisecitation
Accountability improves judgment primarily when the decision-maker has not yet committed to a position.
“Tetlock's model specifies that accountability improves judgment primarily when the decision-maker has not yet committed to a position.”
Quote in the paperSubstantial claim0 of 2 sources checked - Study design and conductnone given
The No AI reference comprises initial decisions from sequential disclosure participants rather than a fully independent group.
“The No AI reference comprises initial decisions from sequentialdisclosure participants rather than a fully independent group.”
Quote in the paperNot a substantial claim - Interpretationcitation
The association between longer decision times and higher AI reliance in the accountability condition is inconsistent with a simple claim that additional deliberation necessarily reduces heuristic reliance.
“This association is inconsistent with a simple claim that additional deliberation necessarily reduces heuristic reliance (Evans and Stanovich 2013;Kahneman 2011), but it does not show that deliberation caused deference.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Evans and Stanovich 2013 source
- cites Kahneman 2011
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
Tetlock's social contingency model holds that when decision-makers anticipate evaluation by an audience whose views are unknown, they engage in more effortful, integrative, and self-critical reasoning.
“Tetlock's social contingency model (Tetlock 1983(Tetlock , 1992) ) provides the theoretical foundation: when decision-makers anticipate evaluation by an audience whose views are unknown, they engage in more effortful, integrative, and self-critical reasoning.”
Quote in the paperSubstantial claim0 of 2 sources checked - Interpretationown data
More elaborate and emotionally engaged reasoning was most prevalent in decisions that aligned with AI but diverged from expert majority.
“indicating that more elaborate and emotionally engaged reasoning was most prevalent in decisions that aligned with AI but diverged from expert majority.”
Quote in the paperSubstantial claim - Interpretationown data
Participants were more likely to invoke moral reasoning when their decisions diverged from what AI recommended in a harmful direction.
“suggesting that participants were more likely to invoke moral reasoning when their decisions diverged from what AI recommended in a harmful direction.”
Quote in the paperSubstantial claim - Interpretationreasoning
The consequence condition's reduced AI following was concentrated on cases where deviation from AI was warranted.
“This pattern held despite the consequence condition's lower overall AI-following rate, suggesting that reduced AI following in this condition was concentrated on cases where deviation from AI was warranted.”
Quote in the paperSubstantial claim - Interpretationown data
Visible markers of careful reasoning can coexist with AI deference.
“These associations show that visible markers of careful reasoning can coexist with AI deference; they do not establish that moral reasoning or case-specific reasoning caused the decisions.”
Quote in the paperSubstantial claim - Interpretationown data
The present study establishes only that the consequence condition's effect did not depend on longer deliberation or more elaborate written moral reasoning.
“but the present study establishes only that the condition's effect did not depend on longer deliberation or more elaborate written moral reasoning.”
Quote in the paperSubstantial claim - Interpretationown data
Prior commitment did not reliably protect an initial judgment from later AI exposure.
“Thus, prior commitment did not reliably protect an initial judgment from later AI exposure.”
Quote in the paperSubstantial claim - Interpretationown data
Participants were more likely to revise away from expert-aligned initial judgments than toward them.
“That is, participants were more likely to revise away from expert-aligned initial judgments than toward them.”
Quote in the paperSubstantial claim - Study design and conductreasoning
Expert majority judgment serves as a stable reference point against which to assess whether participants adjusted their reliance based on the quality of the AI recommendation or followed it regardless.
“Rather, it serves as a stable reference point against which we can assess whether participants adjusted their reliance based on the quality of the AI recommendation or followed it regardless.”
Quote in the paperSubstantial claim - Interpretationreasoning
The locus of effective intervention is attentional frame, not deliberative depth.
“The locus of effective intervention is therefore attentional frame, not deliberative depth.”
Quote in the paperSubstantial claim - Interpretationown data
Uncalibrated switching was concentrated on higher-consensus cases.
“Uncalibrated switching was therefore concentrated on higher-consensus cases.”
Quote in the paperSubstantial claim - Premisecitation
The selective accessibility model proposes that the anchor primes semantically consistent knowledge, such that additional deliberation deepens rather than corrects the anchor's influence.
“The selective accessibility model (Mussweiler and Strack 1999) proposes that the anchor primes semantically consistent knowledge, such that additional deliberation deepens rather than corrects the anchor's influence.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Mussweiler and Strack 1999 source
- Study design and conductreasoning
No AI decisions derive from control condition participants prior to AI exposure rather than from a fully independent, naive No AI group.
“This reference should be interpreted with one important caveat: No AI decisions derive from control condition participants prior to AI exposure rather than from a fully independent, naive No AI group.”
Quote in the paperSubstantial claim - Premisecitation
Once an anchor is set, additional deliberation tends to build a case for the anchor's conclusion rather than question it.
“Anchoring is particularly important because it explains why thinking harder does not fix the problem: once an anchor is set, additional deliberation tends to build a case for the anchor's conclusion rather than question it (Tversky and Kahneman 1974;Mussweiler and Strack 1999).”
Quote in the paperSubstantial claim0 of 2 sources checked - Premisecitation
Under an attentional reframing account, consequence framing shifts the decision-maker's frame from responding to a recommendation to predicting an outcome, preserving independent judgment without requiring additional cognitive effort.
“Under an attentional reframing account, consequence framing shifts the decision-maker's frame from responding to a recommendation to predicting an outcome, preserving independent judgment without requiring additional cognitive effort (Loewenstein et al. 2001;Lerner and Keltner 2001).”
Quote in the paperNot a substantial claim0 of 2 sources checked
- May combine two claims. 47
- Classified as probably not a substantial claim. 25
May combine two claims.
Findingown dataConsequence framing showed the most consistent protective pattern without requiring more deliberation.
“Consequence framing showed the most consistent protective pattern without requiring more deliberation.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataOverreliance justifications had the highest rates of both moral framing (63%) and case specificity (42.6%) among all four reliance types.
“Overreliance justifications had the highest rates of both moral framing (63%) and case specificity (42.6%) among all four reliance types”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). May combine two claims.
Findingown dataParticipants in the No AI condition frequently revised their initial independent decisions to align with the AI recommendation after exposure, and did so at comparable rates whether the AI was expert-aligned or not.
“Participants in the No AI condition frequently revised their initial independent decisions to align with the AI recommendation after exposure, and did so at comparable rates whether the AI was expert-aligned or not.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataReasoning quality did not credibly predict appropriate reliance (b = 0.20 [-0.32, 0.74]) or expert-aligned decisions (b = 0.20 [-0.31, 0.70]).
“Reasoning quality did not credibly predict appropriate reliance (b = 0.20 [-0.32, 0.74]) or expert-aligned decisions (b = 0.20 [-0.31, 0.70]).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe accountability condition produced the only credible interaction between condition and AI-expert alignment on decision latency (b = 0.16 [0.01, 0.32]), with participants taking disproportionately longer on AI-expert-aligned cases.
“The accountability condition was the sole exception, producing the only credible interaction with AI-expert alignment (b = 0.16 [0.01, 0.32]; Table 2): participants took disproportionately longer on AI-”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataIn the baseline condition, the rate of following the AI recommendation was 74% [63,83], credibly higher than the No AI reference (b = 1.04 [0.47, 1.64]).
“When participants saw the AI recommendation alongside the case, this rate rose to 74% [63,83] in the baseline condition and 69% [56,80] in the accountability condition; both increases over the No AI reference were credible (baseline: b = 1.04 [0.47, 1.64]; account-”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataExperts were unanimous on 13 of 16 vignettes and split 2-1 on the remaining 3, with 87.5% pairwise agreement.
“Experts were unanimous on 13 of 16 vignettes and split 2-1 on the remaining 3, with 87.5% pairwise agreement.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataMoral framing and case specificity were negatively associated with expert alignment, and over-reliance justifications had among the highest reasoning-quality scores.
“Moral framing and case specificity were negatively associated with expert alignment, and over-reliance justifications had among the highest reasoning-quality scores.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataIn the baseline and consequence conditions, deliberation time was roughly equivalent across cases regardless of AI-expert alignment, so case-related variation in deliberation was attenuated when AI was visible.
“In the baseline and consequence conditions, deliberation time was roughly equivalent across cases regardless of AI-expert alignment, so case-related variation in deliberation was attenuated when AI was visible.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataCase specificity showed a negative association with both appropriate reliance (b = -1.07 [-2.15, -0.06]) and expert alignment (b = -1.07 [-2.11, -0.07]), with both associations credible.
“Case specificity showed a negative association with both appropriate reliance (b = -1.07[-2.15, -0.06]) and expert alignment (b = -1.07[-2.11, -0.07]), with both associations credible.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataReliance was primarily case-driven (SD case = 1.06 vs. SD pid = 0.51), while latency was primarily person-driven (SD pid = 0.53 vs. SD case = 0.21).
“The variance structure of the models further reveals a dissociation: reliance was primarily case-driven (SD case = 1.06 vs. SD pid = 0.51), while latency was primarily person-driven (SD pid = 0.53 vs. SD case = 0.21), indicating that what people decide depends on which case they face, while how long they take reflects stable individual differences.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataOver-reliance cases showed the highest rate of moral framing (approximately 63%), followed by Appropriate Deviation (approximately 41%) and Under-reliance (approximately 39%), while Appropriate Reliance cases had the lowest rate (approximately 31%).
“Over-reliance cases showed the highest rate of moral framing (approximately 63%), followed by Appropriate Deviation (approximately 41%) and Under-reliance (approximately 39%).Appropriate Reliance cases had the lowest rate (approximately 31%)”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataOn split (2-1) cases, the direction was similar but the interval spanned zero.
“On split (2-1) cases the direction was similar but the interval spanned zero.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataMoral framing was negatively associated with appropriate reliance (b = -0.62 [-1.26, -0.02]) and with expert alignment (b = -0.62 [-1.26, 0.03]); the association with appropriate reliance was credible but the association with expert alignment was not.
“Moral framing was negatively associated with both appropriate reliance (b = -0.62 [-1.26, -0.02]) and expert alignment (b = -0.62 [-1.26, 0.03]); the association with appropriate reliance was credible but the association with expert alignment was not.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Findingown dataThe final analytic sample comprised 224 participants (57 baseline, 57 consequence, 52 accountability, 58 No AI) contributing 1,344 case-level observations.
“The final analytic sample comprised 224 participants (57 baseline, 57 consequence, 52 accountability, 58 No AI) contributing 1,344 case-level observations.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe model coefficient for AI-expert alignment on switching was in the expected direction (b = -0.77 [-1.91, 0.30]), but the posterior interval spanned zero.
“The model coefficient for AI-expert alignment on switching was in the expected direction (b = -0.77[-1.91, 0.30]), but the posterior interval spanned zero.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataWithout AI exposure, participants spent longer on more ambiguous cases; with AI visible, this case-related variation was attenuated in the baseline and consequence conditions.
“Without AI exposure, participants spent longer on more ambiguous cases; with AI visible, this case-related variation was attenuated in the baseline and consequence conditions.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAmong participants whose initial decisions differed from the AI recommendation (n = 164 disagreement instances from 58 participants), the switching rate was 42.9% when the AI recommendation was not expert-aligned and 28.7% when it was expert-aligned.
“Among participants whose initial decisions differed from the AI recommendation (n = 164 disagreement instances from 58 participants), the switching rate was 42.9% when the AI recommendation was not expert-aligned and 28.7% when it was expert-aligned (Figure 5).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataIn the accountability condition, the rate of following the AI recommendation was 69% [56,80], credibly higher than the No AI reference (b = 0.61 [0.01, 1.21]).
“69% [56,80] in the accountability condition”
- Also stated in: § What They Decided: AI Reliance Across Conditions · plain
“ability: b = 0.61 [0.01, 1.21]; Table 1).”
Quote in the paperSubstantial claim- Also stated in: § What They Decided: AI Reliance Across Conditions · plain
May combine two claims.
Findingown dataParticipants in the accountability condition spent an estimated 39 seconds [32,47] on each decision page, approximately 34% longer than the No AI (29 s [24,35]) and baseline (29 s [24, 35]) conditions.
“Participants in the accountability condition spent an estimated 39 seconds [32,47] on each decision page-approximately 34% longer than the No AI (29 s [24,35]) and baseline (29 s [24, 35]) conditions.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationAlgorithms serve as a mental shortcut that reduces cognitive effort, so people default to agreement unless they are motivated to think carefully.
“First, algorithms serve as a mental shortcut that reduces cognitive effort, so people default to agreement unless they are motivated to think carefully (Evans 2008;Kahneman 2011).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Evans 2008 source
- cites Kahneman 2011
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationWhen humans receive an algorithmic recommendation, they defer to it at rates exceeding what would be warranted by the algorithm's demonstrated accuracy, even when available evidence contradicts the recommendation.
“The core finding is broadly consistent: when humans receive an algorithmic recommendation, they defer to it at rates exceeding what would be warranted by the algorithm's demonstrated accuracy, even when available evidence contradicts the recommendation (Skitka, Mosier, and Burdick 1999;Goddard, Roudsari, and Wyatt 2012).”
Quote in the paperNot a substantial claim0 of 2 sources checkedClassified as probably not a substantial claim.
PremisecitationIn child welfare, algorithmic risk assessment tools are being adopted to assist caseworkers in screening decisions that determine whether a family is investigated for alleged maltreatment.
“In child welfare, for example, algorithmic risk assessment tools are being adopted to assist caseworkers in screening decisions that determine whether a family is investigated for alleged maltreatment (Chouldechova et al. 2018;Vaithianathan et al. 2013).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Chouldechova et al. 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Vaithianathan et al. 2013 source
Classified as probably not a substantial claim.
PremisecitationThe algorithm's recommendation acts as an anchor, an initial reference point that shapes all subsequent reasoning.
“Third, the algorithm's recommendation acts as an anchor, meaning an initial reference point that shapes all subsequent reasoning.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationPeople treat algorithms as authoritative experts and defer to them rather than weighing the evidence themselves.
“Second, people treat algorithms as authoritative experts and defer to them rather than weighing the evidence themselves (Sundar 2008).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Sundar 2008
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
Study design and conductown dataParticipants were randomly assigned via Qualtrics to one of four between-subjects conditions.
“Participants were recruited through Prolific (U.S. residents, ≥95% approval rate, $11/hr) and randomly assigned via Qualtrics to one of four between-subjects conditions.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationThe anchoring-and-adjustment heuristic specifies that when a salient anchor is provided, subsequent reasoning is selectively biased toward anchor-consistent information, and adjustments are systematically insufficient.
“The anchoring-and-adjustment heuristic specifies that when a salient anchor is provided, subsequent reasoning is selectively biased toward anchor-consistent information, and adjustments are systematically insufficient (Tversky and Kahneman 1974;Mussweiler and Strack 1999).”
Quote in the paperSubstantial claim0 of 2 sources checkedClassified as probably not a substantial claim.
Study design and conductreasoningBecause initial decisions in the sequential disclosure condition were made before the algorithmic recommendation was provided, they serve as the No AI reference across all analyses.
“Because initial decisions were made before the algorithmic recommendation was provided, they serve as the No AI reference across all analyses.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationDecisions about whether evidence substantiates maltreatment involve contested evidence and rely heavily on professional judgment.
“Unlike domains with clear, verifiable outcomes, decisions about whether evidence substantiates maltreatment involve contested evidence and rely heavily on professional judgment (Berrick 2018;Ahn et al. 2025).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Berrick 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Ahn et al. 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAccountability is among the most widely studied mechanisms for improving judgment quality.
“Accountability, defined as the expectation of having to justify one's decisions to others, is among the most widely studied mechanisms for improving judgment quality (Lerner and Tetlock 1999).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Lerner and Tetlock 1999 source
Classified as probably not a substantial claim. May combine two claims.
Study design and conductreasoningThe controlled analogue task uses lay participants and therefore tests a broader assumption about human oversight rather than how child welfare professionals would behave in practice.
“Our controlled analogue task uses lay participants and therefore tests a broader assumption about human oversight rather than how child welfare professionals would behave in practice.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenThree child welfare experts independently assessed the risk level of all cases, and the majority decision was used as the analytical benchmark.
“To establish an analytical benchmark for evaluating participants' AI reliance, three child welfare experts independently assessed the risk level of all cases.We use the majority decision as the benchmark.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataMore effort, higher-quality written reasoning, and greater moral engagement did not increase deviation from AI when the recommendation differed from expert majority judgment.
“More effort, higher-quality written reasoning, and greater moral engagement did not increase deviation from AI when the recommendation differed from expert majority judgment.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataConsequence participants had the highest rate of alignment with expert-majority judgment, without increased decision time or a credible increase in AI following when AI and experts disagreed.
“Consequence participants had the highest rate of alignment with expert-majority judgment, without increased decision time or a credible increase in AI following when AI and experts disagreed.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductreasoningParticipants with prior child welfare experience, either professional or personal, were excluded to isolate intervention effects from variation in domain familiarity and to avoid drawing inference from participants whose lived experience could make the case material distressing.
“Among 280 participants recruited, we excluded participants with prior child welfare experience, either professional (e.g., social work, child protective services) or personal (e.g., having had a child involved with the system), to isolate intervention effects from variation in domain familiarity and to avoid drawing inference from participants whose lived experience could make the case material”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThis study was approved by the Washington University in St. Louis Institutional Review Board.
“This study was approved by the Washington University in St. Louis Institutional Review Board.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Study design and conductnone givenDecision latency is treated as a behavioral proxy for deliberation effort.
“We treat decision latency as a behavioral proxy for deliberation effort: in the No AI condition it captures unaided deliberation, while in treatment conditions it captures post-recommendation processing.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenJustification coding used AI-generated labels validated on a human-coded subset and identifies correlations rather than causes.
“Justification coding used AI-generated labels validated on a human-coded subset and identifies correlations rather than causes.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationIn child welfare, definitive ground truth is rarely available; risk assessment involves weighing ambiguous and sometimes conflicting indicators, and reasonable professionals may reach different conclusions from the same evidence.
“In child welfare, as in many high-stakes domains, definitive ground truth is rarely available; risk assessment involves weighing ambiguous and sometimes conflicting indicators, and reasonable professionals may reach different conclusions from the same evidence (Dawes, Faust, and Meehl 1989).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Dawes, Faust, and Meehl 1989 source
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationWeakly informative priors were used to regularize estimates away from extreme values.
“For each dimension, we fit Bayesian multilevel models using the brms package in R (Bürkner 2017), with weakly informative priors (Normal(0, 2) on fixed effects and Student-t(3, 0, 2.5) on variance components) to regularize estimates away from extreme values.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Bürkner 2017 source
Not checked in the free run.
May combine two claims.
Interpretationown dataAI recommendations shaped judgment across most conditions, and requiring more deliberation did not produce more discriminating reliance.
“AI recommendations shaped judgment across most conditions, and requiring more deliberation did not produce more discriminating reliance.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationWhen potential consequences are vivid and personally contingent, decision-makers may attend more to their own causal agency and less to the algorithm's assessment, disrupting anchoring by changing the object of attention before deliberation begins.
“When potential consequences are vivid and personally contingent, decisionmakers may attend more to their own causal agency and less to the algorithm's assessment (Lerner et al. 2015;Lerner and Keltner 2000), disrupting anchoring by changing the object of attention before deliberation begins.”
Quote in the paperSubstantial claim0 of 2 sources checkedMay combine two claims.
Interpretationown dataWhat people decide depends on which case they face, while how long they take reflects stable individual differences.
“The variance structure of the models further reveals a dissociation: reliance was primarily case-driven (SD case = 1.06 vs. SD pid = 0.51), while latency was primarily person-driven (SD pid = 0.53 vs. SD case = 0.21), indicating that what people decide depends on which case they face, while how long they take reflects stable individual differences.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
InterpretationreasoningProfessional expertise, institutional routines, and accountability structures may produce different reliance patterns, making replication with caseworkers essential.
“Professional expertise, institutional routines, and accountability structures may produce different reliance patterns, making replication with caseworkers essential.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenParticipants who failed an attention check, completed the study in under 180 seconds, exceeded 3 SD above the mean duration, participated in multiple conditions, or had participated in a prior pilot were excluded.
“We further excluded those who failed an attention check (n = 34), completed the study in under 180 seconds (n = 3), exceeded 3 SD above the mean duration (n = 7), participated in multiple conditions (n = 6), or had participated in a prior pilot (n = 7).”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductown dataA CPS-retained descriptive check was similar to the primary sample, but the professional-exposure subgroup was small and cannot support inferential claims.
“A CPS-retained descriptive check was similar to the primary sample, but the professional-exposure subgroup was small and cannot support inferential claims; professional replication is needed.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductown dataAn AI risk recommendation was generated for each case by a logistic regression classifier previously trained on NCANDS data, achieving 78 percent accuracy on held-out data.
“An AI risk recommendation was generated for each case by inputting case feature values into a logistic regression classifier previously trained on the National Child Abuse and Neglect Data System (NCANDS) data, achieving 78 percent accuracy on held-out data.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.
Study design and conductreasoningAlthough the design minimized demand characteristics by withholding any mention of an upcoming AI recommendation, a separate group with no AI exposure at any point would have provided a stronger baseline.
“Although the design minimized demand characteristics by withholding any mention of an upcoming AI recommendation, a separate group with no AI exposure at any point would have provided a stronger baseline.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataAccountability increased effort without reducing AI deference, and sequential disclosure did not fully preserve prior independent judgments.
“Accountability increased effort without reducing AI deference, and sequential disclosure did not fully preserve prior independent judgments.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationIncreasing the perceived cost of errors can reduce over-reliance.
“This parallels evidence that increasing the perceived cost of errors can reduce over-reliance (Vasconcelos et al. 2023).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Vasconcelos et al. 2023 source
Not checked in the free run.
May combine two claims.
InterpretationreasoningIn AI-assisted contexts, the algorithm's recommendation may function both as an anchor and as a signal of the expected conclusion, converting accountability pressure from a spur to independent thinking into a demand for rationalization.
“In AI-assisted contexts, the algorithm's recommendation may function both as an anchor and as a signal of the expected conclusion, converting accountability pressure from a spur to independent thinking into a demand for rationalization.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductnone givenThe expert benchmark had moderate agreement and represents majority judgment, not ground truth.
“The expert benchmark had moderate agreement and represents majority judgment, not ground truth.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenThe interface presented each case in a linear flow: narrative case vignette, structured assessment, AI recommendation, and user decision, ensuring participants engaged with case evidence before encountering the AI output.
“The interface presented each case in a linear flow: narrative case vignette, structured assessment, AI recommendation, and user decision, ensuring participants engaged with case evidence before encountering the AI output (Figure 1).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationUnder the risk-as-feelings hypothesis, affective responses to vivid outcome representations drive decision behavior by increasing motivational engagement.
“Under the alternative risk-as-feelings hypothesis (Loewenstein et al. 2001;Slovic et al. 2004), affective responses to vivid outcome representations drive decision behavior by increasing motivational engagement.”
Quote in the paperNot a substantial claim0 of 2 sources checkedMay combine two claims.
Study design and conductreasoningExpert majority judgment does not resolve ambiguity, and the authors do not treat it as a proxy for the correct answer.
“Expert majority judgment does not resolve this ambiguity, and we do not treat it as a proxy for the correct answer.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataParticipants were lay adults recruited online rather than child welfare professionals, so the study does not show how caseworkers would behave.
“Participants were lay adults recruited online rather than child welfare professionals, so the study does not show how caseworkers would behave.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataThe accountability requirement increased decision time but did not produce a reliance profile that differed from baseline.
“The accountability requirement increased decision time but did not produce a reliance profile that differed from baseline.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenAll models include participant- and case-specific random intercepts to separate individual tendencies from case characteristics.
“All models include participant-and case-specific random intercepts to separate individual tendencies from case characteristics, and use 4 chains of 4,000 iterations (2,000 warmup).”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationGreen and Chen (2021) provided direct support in AI contexts, showing that algorithmic risk assessments systematically altered how participants weighted risk, consistent with AI recommendations shifting the frame through which case information was evaluated.
“Green and Chen (2021) provided direct support in AI contexts, showing that algorithmic risk assessments systematically altered how participants weighted risk, consistent with AI recommendations shifting the frame through which case information was evaluated.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Green and Chen (2021) source
Not checked in the free run.
The paper relies on this without support.
Errors in child welfare carry serious consequences in both directions: failing to intervene where maltreatment has occurred risks serious harm to a child, while unwarranted intervention can disrupt families.
“Compounding this, errors carry serious consequences in both directions: failing to intervene where maltreatment has occurred risks serious harm to a child, while unwarranted intervention can disrupt families.”
The paper relies on this without support.
Experimental research on how interventions shape human-AI interaction in high-stakes contexts remains limited.
“Despite the increasing deployment of algorithmic risk assessment tools in this domain (Chouldechova et al. 2018;Vaithianathan et al. 2013), experimental research on how interventions shape human-AI interaction in such high-stakes contexts remains limited.”
The paper relies on this without support.
The conditions under which integration of AI risk scores with caseworkers' own judgment reliably preserves independent judgment in high-stakes professional contexts remain inconsistently supported across studies.
“These tools generate risk scores that caseworkers are expected to integrate with their own judgment, but the conditions under which this integration reliably preserves independent judgment in high-stakes professional contexts remain inconsistently supported across studies.”
The paper relies on this without support.
The conditions under which decision-support interface design choices preserve or further erode independent judgment remain inconsistently supported across studies.
“How decision-support interfaces are designed may matter, but the conditions under which design choices preserve or further erode independent judgment remain inconsistently supported across studies.”
The paper relies on this without support.
Expert majority judgment is a valid benchmark for evaluating decision quality.
“The expert benchmark had moderate agreement and represents majority judgment, not ground truth.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Human-coded subset is not approximately 12% of justifications
MinorThe 30 human-coded justifications are approximately 9.6% of the 312 justification texts, not 12%.
Reported 12 · recomputed 9.615385
“one human rater independently coded a randomly selected subset of 30 justifications (approximately 12% of the sample)”
Over-reliance is described as diverging from the AI
MinorThe paper defines over-reliance as following AI when experts disagree, but later describes over-reliance decisions as diverging from the AI recommendation.
“Over-reliance cases showed the highest rate of moral framing (approximately 63%), followed by Appropriate Deviation (approximately 41%) and Under-reliance (approximately 39%).Appropriate Reliance cases had the lowest rate (approximately 31%), suggesting that participants were more likely to invoke moral reasoning when their decisions diverged from what AI recommended in a harmful direction.”
“over-reliance (followed AI”
Justification coding is described as having both three and four dimensions
MinorThe coding description specifies three dimensions, but the reported agreement results include a fourth, AI reference.
“Justification content was coded on three dimensions corresponding to the theoretical frameworks introduced in the section on accountability.”
“Agreement between the human rater and the API-generated codes was high across all dimensions (reasoning quality: κ = .86;moral framing: κ = .92;case specificity: κ = .89;AI reference: κ = .83)”
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 8
- Berrick, J. D. 2018. The Impossible Imperative: Navigat- ing the Competing Principles of Child Protection. Oxford University Press.
- Chouldechova, A.; Benavides-Prado, D.; Fialko, O.; and Vaithianathan, R. 2018. A case study of algorithm-assisted decision making in child maltreatment hotline screening de- cisions. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency, volume 81 of Proceedings of Machine Learning Research, 134-148.
- Coskun, M.; and Leidner, D. 2024. Putting a human in the loop: Increasing uptake, but decreasing accuracy of auto- mated decision-making. PLOS ONE, 19(2): e0298037.
- European Parliament and Council of the European Union. 2016. Regulation (EU) 2016/679 of the European Parlia- ment and of the Council (General Data Protection Regula- tion). Official Journal of the European Union, L 119: 1-88.
- Mosier, K. L.; Skitka, L. J.; Heers, S.; and Burdick, M. 1998. Automation bias: Decision making and performance in high- tech cockpits. International Journal of Aviation Psychology, 8(1): 47-63.
- Noddings, N. 1984. Caring: A Feminine Approach to Ethics and Moral Education. University of California Press.
- Sundar, S. S. 2008. The MAIN model: A heuristic approach to understanding technology effects on credibility. In Met- zger, M. J.; and Flanagin, A. J., eds., Digital Media, Youth, and Credibility. MIT Press.
- Toulmin, S. E. 1958. The Uses of Argument. Cambridge University Press.
- Authors
- Shrestha, S., Ahn, E., & Vorobeychik, Y.
- Affiliations
- Washington University in St. Louis (US)
- University of Wisconsin–Madison (US)
- ORCID
- Ahn, Eunhye
- Venue
- Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society
- Published
- 2026-10-11
- Volume
- 9
- Issue
- 4
- Pages
- 2315-2326
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W7221461089
- Semantic Scholar
- bb212ad396d27a167effc61a32932078cd0879a1
- Cited by
- 0
- References
- 0
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (publishedVersion)
- Funders
- National Science Foundation (2403758, 2214141)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- AI-assisted decision-making, automation bias, overreliance on AI, moral reasoning, cognitive anchoring, deliberation, human-AI collaboration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-30.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-30.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
