The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
The benefits of imperfect diagnostic automation: a synthesis of the literature
Abstract
This review of the literature examines, in a quantitative fashion, how the level of imperfection or unreliability of diagnostic automation affects the performance of the human operator who is jointly consulting that automation and the raw data itself. The data from 20 different studies were used to generate 35 different data points that compared performance with varying levels of unreliability, with that of a non-automated baseline condition. A regression analysis of benefits/costs relative to baseline was carried out, and revealed a strong linear function of benefits with reliability. The analysis revealed that a reliability of 0.70 was the ‘crossover point’ below which unreliable automation was worse than no automation at all. The analysis also revealed that performance was more strongly affected by reliability in high workload conditions, implicating the role of workload-imposed automation dependence in producing this relationship, and suggesting that humans tend to protect performance of concurrent tasks from imperfection of diagnostic automation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This observation supports prior research suggesting that both over-reliance and under-reliance are closely linked to perceived AI competence and actual system performance (Lee & See, 2004;Wickens & Dixon, 2007).”
From Investigating appropriate reliance on AI-Based decision support systems: the role of expertise, trust, and self-confidence (2025) · cites this paper as Wickens & Dixon, 2007
In the citing paper
“Previous research has revealed consistently a positive relationship between automation reliability and the (snapshot) trust in automation measured at the end of an experiment”
From Toward Quantifying Trust Dynamics: How People Adjust Their Trust After Moment-to-Moment Interaction With Automation (2023) · cites this paper as Wickens & Dixon, 2007
In the citing paper
“70% is the threshold above which using automation improves task performance and below which such use may harm performance”
From Toward Quantifying Trust Dynamics: How People Adjust Their Trust After Moment-to-Moment Interaction With Automation (2023) · cites this paper as Wickens & Dixon, 2007
In the citing paper
“On the contrary, we find that automation reliability significantly predicts the (snapshot) trust in automation measured at the end of an experiment, which is in line with previous research (de Visser & Parasuraman, 2011; Du et al., 2020; Wickens & Dixon, 2007).”
From Toward Quantifying Trust Dynamics: How People Adjust Their Trust After Moment-to-Moment Interaction With Automation (2023) · cites this paper as Wickens & Dixon, 2007
Citations
Cites
The magical number seven, plus or minus two: Some limits on our capacity for processing information.
Miller, G. A. (1956) · Psychological Review · doi:10.1037/h0043158 · Cited by 17,812
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 3×
“Numerous contextual factors such as human operator knowledge and payoffs may well moderate this value considerably (Lee and See 2004).”
BackgroundResultInfluentialA model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 2×
“( 2000 ), diagnostic aiding refers both to stage 1 automation (e.”
BackgroundSelf-citationPerformance Consequences of Automation-Induced 'Complacency'
Parasuraman, R., Molloy, R., & Singh, I. L. (1993) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0301_1 · Cited by 875 · cited 2×
“In both of these cases, the automation benefits are likely to increase, to the extent that the single task to be automated is more difficult (Maltz and Shinar 2003, Dixon and Wickens 2006), and/or that concurrent task load is imposed (Parasuraman et al. 1993).”
BackgroundDoes automation bias decision-making?
SKITKA, L. J., MOSIER, K. L., & BURDICK, M. (1999) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1999.0252 · Cited by 694
Yeh, M., & Wickens, C. D. (2001) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872001775898269 · Cited by 175
Self-citation
Research works printed but not linked: 29
- 10.1016/j.apergo.2003.07.003 · DOI
- Davison HJ, 2001, Proceedings of the 11th International Symposium on Aviation Psychology
- 10.1518/001872097778543930 · DOI
- 10.1518/001872005774860005 · DOI
- 10.1518/001872006778606822 · DOI
- Dzindolet MT, 2001, Proceedings of the 45th Annual Meeting of the Human Factors and Ergonomics Society, 532
- Galster SM, 2001, Proceedings of the 45th Annual Meeting of the Human Factor Society, 321
- 10.1037/1076-898X.1.1.19 · DOI
- Lee JD, 2004, Human Factors, 46, 50, 10.1518/hfes.46.1.50.30392 · DOI · Paper page
- 10.1016/S0925-7535(00)00043-6 · DOI
- 10.1518/001872006777724408 · DOI · Paper page
- 10.1518/001872001775900931 · DOI
- 10.1518/hfes.45.2.281.27239 · DOI
- 10.1518/hfes.46.2.357.37348 · DOI
- 10.1518/0018720053653802 · DOI
- 10.1518/001872001775870395 · DOI
- 10.1518/001872096779048093 · DOI
- Sheridan T, 2002, Humans and Automation: System Design and Research Issues
- Sheridan, T and Parasurman, R. 2006. Human automation interaction. Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting. 2006. Edited by: Nickerson, R. Santa Monica, CA: Human Factors and Ergonomics Society.
- 10.1207/s15327051hci0101_2 · DOI
- John M, 2002, Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting
- Swets JA, 1982, The Evaluation of Diagnostic Systems
- Thomas LC, 2003, Proceedings of the 47th Annual Human Factors Ergonomics Society Conference
- Wickens CD, 2006, Human Factors of Remotely Operated Vehicles
- 10.1207/STHF0202_01 · DOI
- Wiegmann DA, 2003, Human Factors
- Xu X, Ergonomics
- 10.1518/0018720024497925 · DOI
- 10.1518/hfes.45.3.390.27249 · DOI
Cited by
Sorted by citations
Items 401–408 of 408
Responses to Alerts and Subjective Reports
Kedar, G., Meyer, J., & Bereby-Meyer, Y. (2013) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213571033
“Numerous studies have shown that moderate to highly valid cues greatly help improve decision makers’ performance (e.g., Wickens & Dixon, 2007; Botzer, Meyer & Parmet, 2013).”
BackgroundGraves, W. J., Stauble, M. R., Rice, S. C., & Hottman, S. (2013) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213571350
“…if not most, of the automation studies in the human factors literature has focused on trust in automation (e.g. Lee & Moray, 1994; Lee & See, 2004; Muir, 1987; 1994) and human-automation performance (e.g. Dixon, Wickens & Chang, 2005; Parasuraman, Sheridan & Wickens, 2000; Wickens & Dixon, 2007).”
BackgroundBustamante, E. A. (2010) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193121005400309
Liu, J., Zhong, C., Meng, F., Shank, D. B., & Wunsch, D. C. (2026) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2026.103140
Creation of Tonal and Speech Alarm Efficacy Scales
Lange, R. A., Rice, S., Severin, C., Chiu, J. S. L., & Ruskin, K. (2022) · College Aviation Review International · doi:10.22488/okstate.22.100260
Effects of Transparency and Haze on Trust and Performance During a Full Motion Video Analysis Task
Leibner, S. C. (2020) · doi:10.25777/7sba-p044
“These models agree with previous studies that automation will result in reduced performance when reliability falls below 70%, at least when the reliability level is consistent (Acharya et al., 2018; Wang, Zhang, & Yang, 2018; Wickens & Dixon, 2007).”
ResultMohamed, S. H. (2015) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/62933
“…typically, the minimum perceived reliability before uses would rely on an imperfect automated system lies around 60% to 70% (Kantowitz et al, 1997; Wickens & Dixon, 2007; Wiegmann, Rich & Zhang, 2001; Yeh & Wickens, 2001) Validity refers to the proximity between the aid’s estimate and the…”
Background
Metadata
- Authors
- Wickens, C. D., & Dixon, S. R.
- Corresponding author
- Wickens, Christopher D.
- Venue
- Theoretical Issues in Ergonomics Science
- Publisher
- Informa UK Limited
- Published
- 2007-05
- Volume
- 8
- Issue
- 3
- Pages
- 201-212
- ISSN
- 1463-922X (ISSN-L)
- 1464-536X
- OpenAlex
- W2019160338
- Cited by
- 423
- References
- 0
- Citations by year
- 2026: 31 · 2025: 25 · 2024: 32 · 2023: 26 · 2022: 27 · 2021: 28 · 2020: 29 · 2019: 19 · 2018: 24 · 2017: 18 · 2016: 26 · 2015: 19 · 2014: 22 · 2013: 27 · 2012: 18
- FWCI
- 15.16
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Theoretical Issues in Ergonomics Science (publishedVersion)
- Funders
- University of Illinois at Urbana-Champaign
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Risk and Safety Analysis
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Keywords
- automated diagnosis, automation reliability, operator performance, human-automation interaction, workload, crossover point, dual-task performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
