Understanding human management of automation errors
Open accessAbstract
Automation has the potential to aid humans with a diverse set of tasks and support overall system performance. Automated systems are not always reliable, and when automation errs, humans must engage in error management, which is the process of detecting, understanding, and correcting errors. However, this process of error management in the context of human-automation interaction is not well understood. Therefore, we conducted a systematic review of the variables that contribute to error management. We examined relevant research in human-automation interaction and human error to identify critical automation, person, task, and emergent variables. We propose a framework for management of automation errors to incorporate and build upon previous models. Further, our analysis highlights variables that may be addressed through design and training to positively influence error management. Additional efforts to understand the error management process will contribute to automation designed and implemented to support safe and effective system performance.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Automation complacency and bias can speed up decision making when recommendations are correct, but when the automation technology provides incorrect recommendations it can lead to omission errors and commission errors.
“Although, automation complacency and bias can speed up decision making when recommendations are correct, when the automation technology provides the incorrect recommendations it can lead to omission errors (the human does not respond to a critical situation) and commission errors (the human follows the recommendation of the automation, even though it is incorrect) (Parasuraman and Manzey, 2010;”
From Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy (2021) · cites this paper as McBride, Rogers and Fisk, 2014
Section 4.2 (page unavailable) describes automation bias as deferring to automation and reports omission and commission errors during a simulated flight; section 4.1 says complacency can reduce detection of automation errors. The copy does not establish that complacency or bias speeds decision-making when recommendations are correct. Its discussion of performance under normal conditions does not specifically report faster decisions. (LLM, conf=0.82)
Experimental — accuracy not yet measuredAutomation complacency among human operators increases with higher levels of automation and higher automation reliability.
“Automation complacency among human operators increases with higher levels of automation and higher automation reliability (McBride, Rogers and Fisk, 2014).”
From Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy (2021) · cites this paper as McBride, Rogers and Fisk, 2014
There are relevant examples where human operators successfully identified errors made by automated systems.
“There are relevant examples where human operators successfully identified errors made by automated systems [14, 54].”
From On the Quest for Effectiveness in Human Oversight: Interdisciplinary Perspectives (2024) · cites this paper as [54]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 3× · see citing works
“Functional specificity is ‘the differentiation of functions, subfunctions, and modes of automation’ (Lee and See 2004, 56).”
BackgroundInfluentialReason, J. (1990) · doi:10.1017/cbo9781139062367 · Cited by 5,108 · see citing works
Humans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · see citing works
A model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 2× · see citing works
“These technologies are designed to support overall system performance, assisting human operators with tasks such as information acquisition and processing, decision-making and action execution (Parasuraman, Sheridan, and Wickens 2000).”
BackgroundComplacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710 · cited 1× · see citing works
“Parasuraman and Manzey (2010) also argued that complacent behaviour needs to be evaluated independently of detection performance.”
BackgroundThe role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · see citing works
ENDSLEY, M. R., & KABER, D. B. (1999) · Ergonomics · doi:10.1080/001401399185595 · Cited by 1,088 · see citing works
MUIR, B. M., & MORAY, N. (1996) · Ergonomics · doi:10.1080/00140139608964474 · Cited by 993 · see citing works
Performance Consequences of Automation-Induced 'Complacency'
Parasuraman, R., Molloy, R., & Singh, I. L. (1993) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0301_1 · Cited by 875 · cited 2× · see citing works
“Although it has been shown that complacency can develop in just 20 minutes (Parasuraman, Molloy, and Singh 1993), it occurred after 20 minutes of continuous use.”
BackgroundMUIR, B. M. (1994) · Ergonomics · doi:10.1080/00140139408964957 · Cited by 873 · see citing works
Does automation bias decision-making?
SKITKA, L. J., MOSIER, K. L., & BURDICK, M. (1999) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1999.0252 · Cited by 694 · cited 2× · see citing works
“In a study by Skitka, Mosier, and Burdick (1999), participants either worked with an 88% reliable automated monitoring aid or were required to monitor the task manually without any automation.”
BackgroundFlight-deck automation: promises and problems
WIENER, E. L., & CURRY, R. E. (1980) · Ergonomics · doi:10.1080/00140138008924809 · Cited by 599 · cited 1× · see citing works
“Automation does not operate in an all-or-none fashion (Wiener and Curry 1980).”
BackgroundThe benefits of imperfect diagnostic automation: a synthesis of the literature
Wickens, C. D., & Dixon, S. R. (2007) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500370105 · Cited by 423 · cited 1× · see citing works
“Wickens and Dixon (2007) focused on modelling the role of reliability in human interaction with automated diagnostic aids.”
BackgroundHumans: Still Vital After All These Years of Automation
Parasuraman, R., & Wickens, C. D. (2008) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x312198 · Cited by 401 · see citing works
Automation Bias: Decision Making and Performance in High-Tech Cockpits
Mosier, K. L., Skitka, L. J., Heers, S., & Burdick, M. (1998) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0801_3 · Cited by 353 · see citing works
Accountability and automation bias
SKITKA, L. J., MOSIER, K., & BURDICK, M. D. (2000) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1999.0349 · Cited by 324 · cited 3× · see citing works
“With this change, Skitka, Mosier, and Burdick (2000) found that training had a significant effect on the number of commission errors committed.”
BackgroundMethodAutomation Bias and Errors: Are Crews Better Than Individuals?
Skitka, L. J., Mosier, K. L., Burdick, M., & Rosenblatt, B. (2000) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap1001_5 · Cited by 127 · see citing works
Aircrews and automation bias: the advantages of teamwork?
Mosier, K. L., Skitka, L. J., Dunbar, M., & McDonnell, L. (2001) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap1101_1 · Cited by 62 · cited 4× · see citing works
“This might explain why Mosier et al. (2001) did not find an effect of training.”
BackgroundInfluential
Flagged first, then by citations
Research works printed but not linked: 47
- Bahner JE Elepfandt MF Manzey D Misuse of diagnostic aids in process control: The effects of automation misses on complacency and automation bias Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2008a 52 19 1330 1334 doi: 10.1177/154193120805201906 · DOI
- Bahner JE Hüper A-D Manzey D Misuse of automated decision aids: Complacency, automation bias and the impact of training experience International Journal of Human-Computer Studies 2008b 66 9 688 699 doi: 10.1016/j.ijhcs.2008.06.001 · DOI · Paper page
- Bailey NR Scerbo MW Automation-induced complacency for monitoring highly reliable systems: The role of task complexity, system experience, and operator trust Theoretical Issues in Ergonomics Science 2007 8 4 321 348 doi: 10.1080/14639220500535301 · DOI · Paper page
- Bliss JP Dunn MC Behavioural implications of alarm mistrust as a function of task workload Ergonomics 2000 43 9 1283 1300 11014752
- Cohen MS Parasuraman R Freeman JT Trust in decision aids: A model and its training implications 1998 Paper presented at the 1998 Command and Control Research and Technology Symposium Monterey, CA
- Dixon SR Wickens CD Automation reliability in unmanned aerial vehicle control: A reliance-compliance model of automation dependence in high workload Human Factors 2006 48 3 474 486 17063963
- Dixon SR Wickens CD Chang D Mission control of multiple unmanned aerial vehicles: A workload analysis Human Factors 2005 47 3 479 487 doi: 10.1518/001872005774860005 16435690 · DOI
- Dixon SR Wickens CD McCarley JS On the independence of compliance and reliance: Are automation false alarms worse than misses? Human Factors 2007 49 4 564 572 doi: 10.1518/001872007x215656 17702209 · DOI · Paper page
- Ezer N Fisk AD Rogers WA Age-related differences in reliance behaviour attributable to costs within a human-decision aid system Human Factors 2008 50 6 853 863 doi: 10.1518/001872008x375018 19292009 · DOI
- Green DM Swets JA Signal detection theory and psychophysics 1988 Wiley New York, NY
- Hancock PA Jagacinski RJ Parasuraman R Wickens CD Wilson GF Kaber DB Human-automation interaction research: Past, present, and future Ergonomics in Design: The Quarterly of Human Factors Applications 2013 21 2 9 14 doi: 10.1177/1064804613477099 · DOI
- Kaber DB Endsley MR The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task Theoretical Issues in Ergonomics Science 2004 5 2 113 153 doi: 10.1080/1463922021000054335 · DOI
- Kanse L van der Schaaf T Recovery from failures in the chemical process industry International Journal of Cognitive Ergonomics 2001 5 3 199 211
- Kantowitz BH Hanowski RJ Kantowitz SC Driver acceptance of unreliable traffic information in familiar and unfamiliar settings Human Factors 1997 39 2 164 176 doi: 10.1518/001872097778543831 · DOI
- Keller D Rice S System-wide versus component-specific trust using multiple aids Journal of General Psychology 2010 137 1 114 128 doi: 10.1080/00221300903266713 20198819 · DOI
- Kontogiannis T User strategies in recovering from errors in man-machine systems Safety Science 1999 32 1 49 68 doi: 10.1016/s0925-7535(99)00010-7 · DOI
- Kontogiannis T A systems perspective of managing error recovery and tactical re-planning of operating teams in safety critical domains Journal of Safety Research 2011 42 2 73 85 doi: 10.1016/j.jsr.2011.01.003 21569889 · DOI
- Kontogiannis T Malakis S A proactive approach to human error detection and identification in aviation and air traffic control Safety Science 2009 47 5 693 706 doi: 10.1016/j.ssci.2008.09.007 · DOI
- Lee J Moray N Trust, control strategies and allocation of function in human-machine systems Ergonomics 1992 35 10 1243 1270 doi: 10.1080/00140139208967392 1516577 · DOI · Paper page
- Lorenz B Di Nocera F Röttger S Parasuraman R Automated fault-management in a simulated spaceflight micro-world Aviation, Space, and Environmental Medicine 2002 73 9 886 897 12234040
- Maltz M Shinar D Imperfect in-vehicle collision avoidance warning systems can aid drivers Human Factors 2004 46 2 357 366 15359683
- McBride SE Rogers WA Fisk AD Understanding the effect of workload on automation use for younger and older adults Human Factors 2011 53 6 672 686 doi: 10.1177/0018720811421909 22235529 PMC5816343 · DOI
- Metzger U Parasuraman R Automation in future air traffic management: Effects of decision aid reliability on controller performance and mental workload Human Factors 2005 47 1 35 49 15960085
- Molloy R Parasuraman R Monitoring an automated system for a single failure: Vigilance and task complexity effects Human Factors 1996 38 2 311 322 doi: 10.1518/001872096779048093 11536753 · DOI
- Moray N Monitoring, complacency, scepticism and eutactic behaviour International Journal of Industrial Ergonomics 2003 31 3 175 178
- Moray N Inagaki T Itoh M Adaptive automation, trust, and self-confidence in fault management of time-critical tasks Journal of Experimental Psychology-Applied 2000 6 1 44 58 doi: 10.1037//0278-7393.6.1.44 10937311 · DOI
- Nikolic MI Sarter NB Peripheral visual feedback: A powerful means of supporting effective attention allocation in event-driven, data-rich environments Human Factors 2001 43 1 30 38 doi:10.1518/001872001775992525 11474762 · DOI
- Norman DA Broadbent DE Reason JT Baddeley AD The ‘problem’ with automation: Inappropriate feedback and interaction, not ‘over-automation’ Human factors in hazardous situations 1990 137 145 Clarendon Press/Oxford University Press New York, NY
- Palmer E Degani A Electronic checklists: Evaluation of two levels of automation 1991 178 183 Proceedings of the Sixth International Aviation Psychology Symposium 29 April – 2 May 1991 Ohio State University, Department of Aviation Columbus, Ohio
- Parasuraman R de Visser E Lin M-K Greenwood PM Dopamine beta hydroxylase genotype identifies individuals less susceptible to bias in computer-assisted decision making Public Library of Science ONE 2012 7 6 1 9 doi:10.1371/journal.pone.0039675 PMC3384589 22761865 · DOI
- Reason J Human error 1990 Cambridge University Press Cambridge, UK
- Rice S Geels K Using system-wide trust theory to make predictions about dependence on four diagnostic aids Journal of General Psychology 2010 137 4 362 375 doi: 10.1080/00221309.2010.499397 21086858 · DOI
- Rice S McCarley JS Effects of response bias and judgment framing on operator use of an automated aid in a target detection task Journal of Experimental Psychology: Applied 2011 17 4 320 31 doi: 10.1037/a0024243 21707202 · DOI
- Rovira E McGarry K Parasuraman R Effects of imperfect automation on decision making in a simulated command and control task Human Factors 2007 49 1 76 87 17315845
- Sanchez J Fisk AD Rogers WA Rovira E Understanding reliance on automation: Effects of error type, error distribution, age and experience Theoretical Issues in Ergonomics Science 2011 Advance online publication. doi:10.1080/1463922X.2011.611269 PMC4307024 25642142 · DOI
- Sarter N Investigating mode errors on automated flight decks: Illustrating the problem-driven, cumulative, and interdisciplinary nature of human factors research Human Factors 2008 50 3 506 510 doi: 10.1518/001872008x312233 18689060 · DOI
- Sarter NB Mumaw RJ Wickens CD Pilots’ monitoring strategies and performance on automated flight decks: An empirical study combining behavioral and eye-tracking data Human Factors 2007 49 3 347 357 doi:10.1518/001872007X196685 17552302 · DOI
- Sarter NB Woods DD Pilot interaction with cockpit automation II: An experimental study of pilots’ model and awareness of the flight management and guidance system International Journal of Aviation Psychology 1994 4 1 1 28
- Sarter NB Woods DD Billings CE Salvendy G Automation surprises Handbook of human factors and ergonomics 1997 1926 1943 Wiley New York, NY 2nd ed
- Sheridan TB Risk, human error, and system resilience: Fundamental ideas Human Factors 2008 50 3 418 426 doi: 10.1518/001872008x250773 18689048 · DOI
- Singh IL Molloy R Parasuraman R Automation-induced ‘complacency’: Development of the complacency-potential rating scale International Journal of Aviation Psychology 1993a 3 2 111 122 doi: 10.1207/s15327108ijap0302_2 · DOI
- Singh IL Molloy R Parasuraman R Individual differences in monitoring failures of automation Journal of General Psychology 1993b 120 3 357
- Skjerve A Skraaning G The quality of human-automation cooperation in human-system interface for nuclear power plants International Journal of Human-Computer Studies 2004 61 5 649 677 doi:10.1016/j.ijhcs.2004.06.001 · DOI
- Tsang PS Vidulich MA Salvendy G Mental workload and situation awareness Handbook of human factors and ergonomics 2006 243 268 John Wiley & Sons, Inc. Hoboken, NJ 3rd ed
- Wickens CD Carswell CM Salvendy G Information processing Handbook of human factors and ergonomics 2006 1570 1596 John Wiley & Sons, Inc. Hoboken, NJ 3rd ed
- Wickens CD Li H Santamaria A Sebok A Sarter NB Stages and levels of automation: An integrated meta-analysis 2010 54 4 389 393 Proceedings of the Human Factors and Ergonomics Society 54th Annual Meeting
- Wilkison B Fisk AD Rogers WA Effects of mental model quality on collaborative system performance 2007 51 22 1506 1510 Proceedings of the Human Factors and Ergonomics Society 51st Annual Meeting
Cited by
Sorted by citations
Dwivedi, Y. K., Hughes, L., Ismagilova, E., Aarts, G., Coombs, C., Crick, T., Duan, Y., Dwivedi, R., Edwards, J., Eirug, A., et al. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.08.002 · Cited by 4,611 · see citing works
“Automation compla-cency among human operators increases with higher levels of automation and higher automation reliability (McBride et al., 2014).”
BackgroundInfluentialAlmost human: Anthropomorphism increases trust resilience in cognitive agents.
de Visser, E. J., Monfort, S. S., McKendrick, R., Smith, M. A. B., McKnight, P. E., Krueger, F., & Parasuraman, R. (2016) · Journal of Experimental Psychology: Applied · doi:10.1037/xap0000092 · Cited by 614 · see citing works
On the Quest for Effectiveness in Human Oversight: Interdisciplinary Perspectives
Sterz, S., Baum, K., Biewer, S., Hermanns, H., Lauber-Rönsberg, A., Meinel, P., & Langer, M. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3659051 · Cited by 91 · see citing works
“Discussing them is beyond the scope of this paper; however, prima facie , there appears to be no inherent conflict between our proposal and these objectives.”
BackgroundInfluentialChen, Y., Zahedi, F. M., Abbasi, A., & Dobolyi, D. (2021) · Information & Management · doi:10.1016/j.im.2020.103394 · Cited by 53 · see citing works
Reviewing Speech Input with Audio: Differences between Blind and Sighted Users
Hong, J., Vaing, C., Kacorri, H., & Findlater, L. (2020) · ACM Transactions on Accessible Computing · doi:10.1145/3382039 · Cited by 11 · see citing works
- Authors
- McBride, S. E., Rogers, W. A., & Fisk, A. D.
- Affiliations
- Georgia Institute of Technology (US)
- Corresponding author
- McBride, Sara E.
- Rogers, Wendy A.
- ORCID
- Rogers, Wendy A.
- Venue
- Theoretical Issues in Ergonomics Science
- Publisher
- Informa UK Limited
- Published
- 2013-08-07
- Volume
- 15
- Issue
- 6
- Pages
- 545-577
- ISSN
- 1463-922X (ISSN-L)
- 1464-536X
- PMC
- PMC4221095
- OpenAlex
- W2077623198
- Cited by
- 82
- References
- 18
- Citations by year
- 2026: 16 · 2025: 4 · 2024: 13 · 2023: 4 · 2022: 7 · 2021: 8 · 2020: 12 · 2019: 5 · 2018: 3 · 2017: 2 · 2016: 6 · 2015: 2
- FWCI
- 4.00
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Theoretical Issues in Ergonomics Science (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute on Aging (P01AG017211)
- Topics
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Risk and Safety Analysis
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- error management, human-automation interaction, human error, human factors, error detection, error correction, automation design, human training, safety
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
