CoChat

Algorithm aversion: People erroneously avoid algorithms after seeing them err.

Open access
Dietvorst, Berkeley J., Simmons, Joseph P., Massey, Cade · 2015 · Journal of Experimental Psychology: General

The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index; none of the 22 sources its claims cite is checked for support yet.

Abstract

Research shows that evidence-based algorithms more accurately predict the future than do human forecasters. Yet when forecasters are deciding whether to use a human forecaster or a statistical algorithm, they often choose the human forecaster. This phenomenon, which we call algorithm aversion, is costly, and it is important to understand its causes. We show that people are especially averse to algorithmic forecasters after seeing them perform, even when they see them outperform a human forecaster. This is because people more quickly lose confidence in algorithmic than human forecasters after seeing them make the same mistake. In 5 studies, participants either saw an algorithm make forecasts, a human make forecasts, both, or neither. They then decided whether to tie their incentives to the future predictions of the algorithm or the human. Participants who saw the algorithm perform were less confident in it, and less likely to choose it over an inferior human forecaster. This was true even among those who saw the algorithm outperform the human.

Overview

CoChat Summary

TL;DR

People become less willing to use a statistical algorithm after seeing it perform and make mistakes, even when it outperforms a human forecaster. This “algorithm aversion” is driven partly by people losing confidence in algorithms more quickly than in humans after observing errors.

Key findings

  • Across five studies, seeing the algorithm perform reduced participants’ likelihood of choosing it over a human forecaster. In the model-and-human conditions, 610 of 741 participants (83%) saw the model outperform the human, yet these participants were among the least likely to choose the model.
  • The model outperformed humans in all five studies. For example, bonuses if choosing model vs. human were Study 1: $1.78 vs. $1.38; Study 2: $1.77 vs. $1.09; Study 3a: $0.48 vs. $0.31; Study 3b: $0.49 vs. $0.30; Study 4: $1.79 vs. $1.38. Human forecasters produced 15–29% more error in the MBA task and 90–97% more error in the airline-passenger task.
  • Seeing the model perform significantly decreased confidence in the model’s forecasts in all four studies where this was tested, whereas seeing the human perform significantly decreased confidence in the human only in Study 4. Confidence in the model’s forecasts significantly mediated the effect of seeing the model perform on choosing the model; confidence in the human did not.
  • In Study 2, participants paid for forecasts within 20 percentiles were less likely to choose the model than those paid within 5 percentiles, χ²(1, N = 139) = 3.56, p = .059, and those paid by average absolute error, χ²(1, N = 136) = 8.56, p = .003; this effect was mediated by confidence in their own forecasts, not confidence in the model’s forecasts.

Methods

  • Five studies had participants forecast real outcomes: MBA student success (Studies 1, 2, and 4) or U.S. state rank in airline passengers (Studies 3a and 3b). Participants were assigned to control, human, model, or model-and-human conditions, then chose whether to tie incentivized bonuses to the model or the human forecaster (themselves in Studies 1–3, another participant in Study 4).
  • Final samples were Study 1: 361; Study 2: 206; Study 3a: 410; Study 3b: 1,036; Study 4: 354. Studies 1, 2, and 4 were run in the Wharton Behavioral Lab; Studies 3a and 3b used Amazon Mechanical Turk.
  • Main analyses compared forecasting performance and choice rates, measured confidence and beliefs, and used binary mediation analyses with Preacher and Hayes’s bootstrapping procedure.

Limitations

  • The paper states it did not explore all boundaries of the effect, including whether it would persist if algorithms were many times more accurate than humans, and whether results depend on features shared by the two forecasting tasks.
  • The paper states it did not explore how algorithm variations—complexity, transparency, forecaster involvement in construction, and designer expertise—affect algorithm aversion.
  • The paper states algorithm aversion is not entirely driven by seeing algorithms err, and it has little to say about how best to reduce aversion among those who have seen an algorithm err. The text also cautions that cross-experiment comparisons should be made cautiously.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1037/xge0000033

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Figures

Image 1
From the PDF
Figure 3 Studies 1-4: Participants who saw the statistical model's results were less likely to choose it. Errors bars indicate ± 1 standard error. In Study 2, 'AAE,' '5-Pct,' and '20-Pct' signify conditions in which participants were incentivized either for minimizing average absolute error, for getting within 5 percentiles of the correct answer, or for getting within 20 percentiles of the correct answer, respectively. AAE = average absolute error; Pct = percentile.
Figure 3 Studies 1-4: Participants who saw the statistical model's results were less likely to choose it. Errors bars indicate ± 1 standard error. In Study 2, 'AAE,' '5-Pct,' and '20-Pct' signify conditions in which participants were incentivized either for minimizing average absolute error, for getting within 5 percentiles of the correct answer, or for getting within 20 percentiles of the correct answer, respectively. AAE = average absolute error; Pct = percentile.From the PDF
Figure 4 Most people do not choose the statistical model unless they are more confident in the model's forecasts than in the human's forecasts. Errors bars indicate ± 1 standard error. The 'Did Not See Model Perform' line represents results from participants in the control and human conditions. The 'Saw Model Perform' line represents results from participants in the model and model-and-human conditions. Differences in confidence between the model's and human's forecasts were computed by subtracting participants' ratings of confidence in the human forecasts from their ratings of confidence in the model's forecasts (i.e., by subtracting one 5-point scale from the other). From left to right, the five x -axis categories reflect difference scores of: < -1, -1, 0, + 1, and > 1. The figure includes results from all five studies.
Figure 4 Most people do not choose the statistical model unless they are more confident in the model's forecasts than in the human's forecasts. Errors bars indicate ± 1 standard error. The 'Did Not See Model Perform' line represents results from participants in the control and human conditions. The 'Saw Model Perform' line represents results from participants in the model and model-and-human conditions. Differences in confidence between the model's and human's forecasts were computed by subtracting participants' ratings of confidence in the human forecasts from their ratings of confidence in the model's forecasts (i.e., by subtracting one 5-point scale from the other). From left to right, the five x -axis categories reflect difference scores of: < -1, -1, 0, + 1, and > 1. The figure includes results from all five studies.From the PDF

Verification

Claims the paper makes

0 Valid · 2 Not valid · 16 Not checked · 90 Own results and interpretations

75 more claims

Create a free account to see all 108 claims and to verify the 12 references still open.

See all claims — free account
  • It may state two separate claims as one. 13
  • Probably not a substantial claim. 16

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

22 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentt(360) = 4.98reported p < .001 · recomputed p < .001

    “t (360) = 4.98, p < .001”

  • Consistentt(204) = 6.26reported p < .001 · recomputed p < .001

    “t (204) = 6.26, p < .001”

  • Consistentt(405) = 7.73reported p < .001 · recomputed p < .001

    “t (405) = 7.73, p < .001”

  • Consistentt(353) = 5.11reported p < .001 · recomputed p < .001

    “t (353) = 5.11, p < .001”

  • Consistentχ²(1) = 39.94reported p < .001 · recomputed p < .001

    “χ 2 (1, N = 271) = 39.94, p < .001”

5 of 24 · the rest aren’t loaded

19 more recomputed tests

Create a free account to see every test the Stats check recomputed.

See all tests — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Self-check results appear here once an independent check has confirmed them.

Citations

Cites

Show all 23 references

Research works printed but not linked: 1

  1. Clark, D. (2012, November 7). How Nate Silver won the 2012 presidential election. Harvard Business Review Blog . Retrieved from http://blogs .hbr.org/cs/2012/11/how\_nate\_silver\_won\_the\_2012\_p.html

Cited by

See all 3,916

Metadata

Authors
Dietvorst, B. J., Simmons, J. P., & Massey, C.
Affiliations
University of Pennsylvania (US)
Venue
Journal of Experimental Psychology: General
Published
2015
Volume
144
Issue
1
Pages
114-126
ISSN
  • 0096-3445 (ISSN-L)
  • 1939-2222
OpenAlex
W3125633690
Cited by
3,232
References
22
Citations by year
2026: 1025 · 2025: 563 · 2024: 445 · 2023: 404 · 2022: 266 · 2021: 216 · 2020: 137 · 2019: 88 · 2018: 41 · 2017: 26 · 2016: 14 · 2015: 5
FWCI
13.96
Type
journal article
Language
en
Open access
Locations
  • Journal of Experimental Psychology General (publishedVersion)
  • SSRN Electronic Journal (publishedVersion)
  • PubMed (publishedVersion)
  • ScholarlyCommons (University of Pennsylvania) (submittedVersion)
Funders
  • Wharton Behavioral Laboratory
  • Wharton Risk Center Ackoff Doctoral Student Fellowship
Topics
  • Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
Keywords
algorithm aversion, trust in algorithms, prediction confidence, evidence-based algorithm, decision-making
MeSH terms
Algorithms, Avoidance Learning, Culture, Decision Making, Female, Forecasting, Humans, Male, Motivation, Comprehension, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-14, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-14 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Algorithm aversion: People erroneously avoid algorithms after seeing them err. https://app.cochat.ai/papers/10.1037/xge0000033
The paper
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General. https://doi.org/10.1037/xge0000033

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.