On the Independence of Compliance and Reliance: Are Automation False Alarms Worse Than Misses?
Abstract
Objective Participants performed a tracking task and system monitoring task while aided by diagnostic automation. The goal of the study was to examine operator compliance and reliance as affected by automation failures and to clarify claims regarding independence of these two constructs. Background Background data revealed a trend toward nonindependence of the compliance-reliance constructs. Method Thirty-two undergraduate students performed the simulation that presented the visual display while dependent measures were collected. Results False alarm-prone automation hurt overall performance more than miss-prone automation. False alarm-prone automation also clearly affected both operator compliance and reliance, whereas miss-prone automation appeared to affect only operator reliance. Conclusion Compliance and reliance do not appear to be entirely independent of each other. Application False alarms appear to be more damaging to overall performance than misses, and designers must take the compliance-reliance constructs into consideration.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Responses to textual alerts in EHRs are similar to previously documented responses to audible and visual alarms.
“Our findings suggest that responses to textual alerts in EHRs are similar to previously documented responses to audible and visual alarms. [16–18]”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [17]
Some studies suggest that a high rate of false alarms can also reduce overall performance on tasks due to increased cognitive burden.
“Furthermore, some studies suggest that a high rate of false alarms can also reduce overall performance on tasks, due to increased cognitive burden as the individual’s attention becomes divided between relevant and irrelevant information [17].”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [17]
False alarms reduce responsiveness to alarms and may also reduce overall performance on tasks interrupted by alarms.
“it is well-established in the human factors literature that false alarms reduce responsiveness to alarms and may also reduce overall performance on tasks interrupted by alarms. [16–18]”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [17]
Uninformative alerts are similar to false alarms.
“Uninformative alerts are similar to false alarms”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [17]
The human factors literature in other settings shows that high false alarm rates reduce responsiveness as the individual learns that the alarm is unreliable.
“The human factors literature in other settings shows that high false alarm rates reduce responsiveness as the individual learns, consciously or unconsciously, that the alarm is unreliable [16–18].”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [17]
In other settings, low alarm informativeness and increased cognitive workload predict poor alarm response.
“In other settings, low alarm informativeness and increased cognitive workload predict poor alarm response.”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [17]
Cites
The benefits of imperfect diagnostic automation: a synthesis of the literature
Wickens, C. D., & Dixon, S. R. (2007) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500370105 · Cited by 423 · cited 1× · see citing works
“This is particularly true of diagnostic automation, which must make diagnoses and long-range predictions in a world of imperfect probabilistic information (Wickens & Dixon, 2007).”
BackgroundSelf-citation
Flagged first, then by citations
Cited by
Sorted by citations
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817 · see citing works
with the HITEC Investigators, Ancker, J. S., Edwards, A., Nosal, S., Hauser, D., Mauer, E., & Kaushal, R. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0430-8 · Cited by 709 · see citing works
Almost human: Anthropomorphism increases trust resilience in cognitive agents.
de Visser, E. J., Monfort, S. S., McKendrick, R., Smith, M. A. B., McKnight, P. E., Krueger, F., & Parasuraman, R. (2016) · Journal of Experimental Psychology: Applied · doi:10.1037/xap0000092 · Cited by 614 · see citing works
Humans: Still Vital After All These Years of Automation
Parasuraman, R., & Wickens, C. D. (2008) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x312198 · Cited by 401 · see citing works
Self-citationEvaluating Effects of User Experience and System Transparency on Trust in Automation
Yang, X. J., Unhelkar, V. V., Li, K., & Shah, J. A. (2017) · Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2909824.3020230 · Cited by 208 · see citing works
Complacency and Automation Bias in the Use of Imperfect Automation
Wickens, C. D., Clegg, B. A., Vieane, A. Z., & Sebok, A. L. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815581940 · Cited by 176 · see citing works
Self-citationHuman Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks
Lu, Z., & Yin, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445562 · Cited by 131 · see citing works
Merritt, S. M., Lee, D., Unnerstall, J. L., & Huber, K. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814561675 · Cited by 83 · see citing works
Prospective memory in an air traffic control simulation: External aids that signal when to act.
Loft, S., Smith, R. E., & Bhaskara, A. (2011) · Journal of Experimental Psychology: Applied · doi:10.1037/a0022845 · Cited by 40 · see citing works
The relationship of trust and dependence
Patton, C. E., & Wickens, C. D. (2024) · Ergonomics · doi:10.1080/00140139.2024.2342436 · Cited by 44 · see citing works
Self-citationContinuous Calibration of Trust in Automated Systems
Merritt, S. M., Huber, K., LaChapell-Unnerstall, J., & Lee, D. (2014) · doi:10.21236/ada606748 · Cited by 20 · see citing works
Kim, J. Y., Lester, C., & Yang, X. J. (2025) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208251326795 · Cited by 21 · see citing works
- Authors
- Dixon, S. R., Wickens, C. D., & McCarley, J. S.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Corresponding author
- Dixon, Stephen R.
- Venue
- Human Factors: The Journal of the Human Factors and Ergonomics Society
- Publisher
- SAGE Publications
- Published
- 2007-08
- Volume
- 49
- Issue
- 4
- Pages
- 564-572
- ISSN
- 0018-7208 (ISSN-L)
- 1547-8181
- OpenAlex
- W2101876245
- Cited by
- 265
- References
- 1
- Citations by year
- 2026: 16 · 2025: 14 · 2024: 13 · 2023: 5 · 2022: 11 · 2021: 13 · 2020: 14 · 2019: 9 · 2018: 11 · 2017: 12 · 2016: 26 · 2015: 8 · 2014: 13 · 2013: 12 · 2012: 13
- FWCI
- 24.25
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Human Factors The Journal of the Human Factors and Ergonomics Society (publishedVersion)
- PubMed (publishedVersion)
- Defense Technical Information Center (DTIC) (submittedVersion)
- Topics
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Risk and Safety Analysis
- Keywords
- automated diagnosis, false alarms, automation failure, system monitoring, human-automation interaction, tracking task
- MeSH terms
- Accidents, Aviation, Automation, Computer Simulation, Equipment Failure, Humans, Ergonomics, Illinois, Task Performance and Analysis, User-Computer Interface, Workload
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
