Slow response times undermine trust in algorithmic (but not human) predictions
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Algorithms consistently perform well on various prediction tasks, but people often mistrust their advice. Here, we demonstrate one component that affects people’s trust in algorithmic predictions: response time. In seven studies (total N = 1928 with 14,184 observations), we find that people judge slowly generated predictions from algorithms as less accurate and they are less willing to rely on them. This effect reverses for human predictions, where slowly generated predictions are judged to be more accurate. In explaining this asymmetry, we find that slower response times signal the exertion of effort for both humans and algorithms. However, the relationship between perceived effort and prediction quality differs for humans and algorithms. For humans, prediction tasks are seen as difficult and effort is therefore positively correlated with the perceived quality of predictions. For algorithms, however, prediction tasks are seen as easy and effort is therefore uncorrelated to the quality of algorithmic predictions. These results underscore the complex processes and dynamics underlying people’s trust in algorithmic (and human) predictions and the cues that people use to evaluate their quality.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
lmerTest Package: Tests in Linear Mixed Effects Models
Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017) · Journal of Statistical Software · doi:10.18637/jss.v082.i13 · Cited by 24,346
Decision aids for people facing health treatment or screening decisions
Stacey, D., Légaré, F., Lewis, K., Barry, M. J., Bennett, C. L., Eden, K. B., Holmes-Rovner, M., Llewellyn-Thomas, H., Lyddiatt, A., Thomson, R., et al. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001431.pub5 · Cited by 6,685
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Task-Dependent Algorithm Aversion
Castelo, N., Bos, M. W., & Lehmann, D. R. (2019) · Journal of Marketing Research · doi:10.1177/0022243719851788 · Cited by 1,500 · cited 1×
“Conversely, people may think that making a prediction is a relatively easy task for an al-gorithm, as it is an objective task involving the integration of multiple pieces of information or complex calculations (Castelo, Bos, & Lehmann, 2019).”
BackgroundTetlock, P. E., Kristel, O. V., Elson, S. B., Green, M. C., & Lerner, J. S. (2000) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.78.5.853 · Cited by 1,400
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018) · Management Science · doi:10.1287/mnsc.2016.2643 · Cited by 1,230
The IKEA effect: When labor leads to love
Norton, M. I., Mochon, D., & Ariely, D. (2012) · Journal of Consumer Psychology · doi:10.1016/j.jcps.2011.08.002 · Cited by 1,049 · cited 1×
“…others invest in completing relatively di ffi cult tasks, whether in the form of time, physical exertion, pain, or money, the more positive the outcome of that e ff ort is evaluated (Festinger, 1957; Kruger, Wirtz, Van Boven, & Altermatt, 2004; Labroo & Kim, 2009; Norton, Mochon, & Ariely, 2012).”
BackgroundWestfall, J., Kenny, D. A., & Judd, C. M. (2014) · Journal of Experimental Psychology: General · doi:10.1037/xge0000014 · Cited by 928
Cited by
What influences algorithmic decision-making? A systematic literature review on algorithm aversion
Mahmud, H., Islam, A. K. M. N., Ahmed, S. I., & Smolander, K. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.121390 · Cited by 601
Man Tang, P., Koopman, J., McClean, S. T., Zhang, J. H., Li, C. H., De Cremer, D., Lu, Y., & Ng, C. T. S. (2022) · Academy of Management Journal · doi:10.5465/amj.2020.1516 · Cited by 370
Yam, K. C., Tang, P. M., Jackson, J. C., Su, R., & Gray, K. (2023) · Journal of Applied Psychology · doi:10.1037/apl0001045 · Cited by 328
Why do users trust algorithms? A review and conceptualization of initial trust and trust over time
Cabiddu, F., Moi, L., Patriotta, G., & Allen, D. G. (2022) · European Management Journal · doi:10.1016/j.emj.2022.06.001 · Cited by 138
“The performance of an AI algorithm indicates the skill or expertise demonstrated by the algorithm’s ability to achieve a user’s goals is also relevant to the user’s trust over time (Efendić et al., 2020; Thiebes et al. 2021; Wang et al., 2019).”
Artificial empathy in healthcare chatbots: Does it feel authentic?
Seitz, L. (2024) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2024.100067 · Cited by 137
Metadata
- Authors
- Efendić, E., Van de Calseyde, P. P. F. M., & Evans, A. M.
- Venue
- Organizational Behavior and Human Decision Processes
- Publisher
- Elsevier BV
- Published
- 2020-03
- Volume
- 157
- Pages
- 103-114
- ISSN
- 0749-5978 (ISSN-L)
- OpenAlex
- W3004930960
- Cited by
- 98
- References
- 13
- FWCI
- 11.37
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Slow response times undermine trust in algorithmic (but not human) predictions. https://app.cochat.ai/papers/10.1016/j.obhdp.2020.01.008
- The paper
- Efendić, E., Van de Calseyde, P. P. F. M., & Evans, A. M. (2020). Slow response times undermine trust in algorithmic (but not human) predictions. Organizational Behavior and Human Decision Processes. https://doi.org/10.1016/j.obhdp.2020.01.008
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.