The Role of Environmental Predictability and Costs in Relying on Automation
Abstract
There is a growing need to understand how automated decision aids are implemented and relied upon by users. Past research has focused on factors associated with the user and automation technology to explain reliance. The purpose of the present study was determining how the predictability of the environment affects reliance. In this paper, we present the results from an experiment using a digital game where participants had access to a free environmental cue of varying predictive validity. Some participants also had access to automated advice at varying costs. We found that participants underutilized automated advice in more predictable environments and when advice was more costly; however, when costs were low and the environment was less predictable, participants tended to overutilize automated advice. These findings provide insights for a more complete model of automation use, and offer a framework for understanding automation biases by considering how automation use compares to a model of optimality.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Such behavioral measures include decision time, number of recommendation requests, and number of agreements with AI recommendations.
“Such measures include, but are not limited to measuring end-users' decision time [19,62], how many times end-users requested AI recommendation [52], and how many times they agreed with AI recommendations [7,23,58].”
From Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust (2023) · cites this paper as [52]
Cites
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879 · see citing works
Mixed-Effects Models in Sand S-PLUS
Pinheiro, J. C., & Bates, D. M. (2000) · Statistics and Computing · doi:10.1007/978-1-4419-0318-1 · Cited by 11,187 · see citing works
Mixed-Effects Models in S and S-PLUS
(2001) · Technometrics · doi:10.1198/tech.2001.s574 · Cited by 9,278 · see citing works
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 1× · see citing works
“Lee and See [12] proposed a model of the interaction between trust in and reliance on automation.”
BackgroundHumans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · see citing works
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817 · see citing works
“Clinical versus Actuarial Judgment.”
Dawes, R. M., Faust, D., & Meehl, P. E. (1989) · Science · doi:10.1126/science.2648573 · Cited by 2,298 · see citing works
Representative design and probabilistic theory in a functional psychology.
Brunswik, E. (1955) · Psychological Review · doi:10.1037/h0047470 · Cited by 1,550 · see citing works
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · see citing works
Individual choice behavior: A theoretical analysis.
Luce, R. D. (2005) · doi:10.1037/14396-000 · Cited by 1,157 · cited 1× · see citing works
“The probability of a particular orc being correct was predicted using a derivation of the multinomial logit [13] [15]: The predictive validity of the size of the orcs (theta) randomly varied between participants; thus, some participants were in environments where size was a poor diagnostic cue…”
MethodEconometric Models for Probabilistic Choice Among Products
McFadden, D. (1980) · The Journal of Business · doi:10.1086/296093 · Cited by 766 · see citing works
Cueing and Cognitive Conflict in Judge-Advisor Decision Making
Sniezek, J. A., & Buckley, T. (1995) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1995.1040 · Cited by 440 · see citing works
Why we still use our heads instead of formulas: Toward an integrative approach.
Kleinmuntz, B. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.3.296 · Cited by 432 · see citing works
Causes and effects of my disturbing little book
Meehl, P. E. (1986) · Journal of Personality Assessment · doi:10.1207/s15327752jpa5003_6 · Cited by 410 · see citing works
The Role of Representative Design in an Ecological Approach to Cognition.
Dhami, M. K., Hertwig, R., & Hoffrage, U. (2004) · Psychological Bulletin · doi:10.1037/0033-2909.130.6.959 · Cited by 405 · see citing works
The Perceived Utility of Human and Automated Aids in a Visual Detection Task
Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (2002) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/0018720024494856 · Cited by 386 · see citing works
Using advice from multiple sources to revise and improve judgments
Yaniv, I., & Milyavsky, M. (2006) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2006.05.006 · Cited by 244 · see citing works
Improving judgement with prepaid expert advice
Sniezek, J. A., Schrah, G. E., & Dalal, R. S. (2004) · Journal of Behavioral Decision Making · doi:10.1002/bdm.468 · Cited by 193 · see citing works
Yeh, M., & Wickens, C. D. (2001) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872001775898269 · Cited by 175 · see citing works
Do we listen to advice just because we paid for it? The impact of advice cost on its use
Francesca Gino, B. (2008) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2008.03.001 · Cited by 173 · see citing works
User agreement with incorrect expert system advice
Dijkstra, J. J. (1999) · Behaviour and Information Technology · doi:10.1080/014492999118832 · Cited by 124 · see citing works
Mechanisms for enhancing the credibility of an adviser: prepayment and aligned incentives
Patt, A., Bowles, H. R., & Cash, D. W. (2006) · Journal of Behavioral Decision Making · doi:10.1002/bdm.532 · Cited by 37 · see citing works
Madhavan, P., & Wiegmann, D. A. (2005) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/0018720054679489 · Cited by 32 · see citing works
Rich stimulus sampling for between-subjects designs improves model selection
Young, M. E., Cole, J. J., & Sutherland, S. C. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0133-5 · Cited by 18 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies
Vereschak, O., Bailly, G., & Caramiaux, B. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476068 · Cited by 229 · see citing works
Capable but Amoral? Comparing AI and Human Expert Collaboration in Ethical Decision Making
Tolmeijer, S., Christen, M., Kandul, S., Kneer, M., & Bernstein, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517732 · Cited by 70 · see citing works
Banovic, N., Yang, Z., Ramesh, A., & Liu, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579460 · Cited by 55 · see citing works
- Authors
- Sutherland, S. C., Harteveld, C., & Young, M. E.
- Affiliations
- Northeastern University (US)
- Kansas State University (US)
- Venue
- Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2015-04-18
- Pages
- 2535-2544
- OpenAlex
- W2010085810
- Cited by
- 12
- References
- 25
- Citations by year
- 2024: 3 · 2023: 2 · 2022: 3 · 2021: 2 · 2019: 1 · 2016: 1
- FWCI
- 0.59
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- automation reliance, environmental predictability, automated decision aids, automation bias, predictive validity, human-automation interaction, decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
