The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
The benefits of imperfect diagnostic automation: a synthesis of the literature
Abstract
This review of the literature examines, in a quantitative fashion, how the level of imperfection or unreliability of diagnostic automation affects the performance of the human operator who is jointly consulting that automation and the raw data itself. The data from 20 different studies were used to generate 35 different data points that compared performance with varying levels of unreliability, with that of a non-automated baseline condition. A regression analysis of benefits/costs relative to baseline was carried out, and revealed a strong linear function of benefits with reliability. The analysis revealed that a reliability of 0.70 was the ‘crossover point’ below which unreliable automation was worse than no automation at all. The analysis also revealed that performance was more strongly affected by reliability in high workload conditions, implicating the role of workload-imposed automation dependence in producing this relationship, and suggesting that humans tend to protect performance of concurrent tasks from imperfection of diagnostic automation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This observation supports prior research suggesting that both over-reliance and under-reliance are closely linked to perceived AI competence and actual system performance (Lee & See, 2004;Wickens & Dixon, 2007).”
From Investigating appropriate reliance on AI-Based decision support systems: the role of expertise, trust, and self-confidence (2025) · cites this paper as Wickens & Dixon, 2007
In the citing paper
“Previous research has revealed consistently a positive relationship between automation reliability and the (snapshot) trust in automation measured at the end of an experiment”
From Toward Quantifying Trust Dynamics: How People Adjust Their Trust After Moment-to-Moment Interaction With Automation (2023) · cites this paper as Wickens & Dixon, 2007
In the citing paper
“70% is the threshold above which using automation improves task performance and below which such use may harm performance”
From Toward Quantifying Trust Dynamics: How People Adjust Their Trust After Moment-to-Moment Interaction With Automation (2023) · cites this paper as Wickens & Dixon, 2007
In the citing paper
“On the contrary, we find that automation reliability significantly predicts the (snapshot) trust in automation measured at the end of an experiment, which is in line with previous research (de Visser & Parasuraman, 2011; Du et al., 2020; Wickens & Dixon, 2007).”
From Toward Quantifying Trust Dynamics: How People Adjust Their Trust After Moment-to-Moment Interaction With Automation (2023) · cites this paper as Wickens & Dixon, 2007
Citations
Cites
The magical number seven, plus or minus two: Some limits on our capacity for processing information.
Miller, G. A. (1956) · Psychological Review · doi:10.1037/h0043158 · Cited by 17,812
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 3×
“Numerous contextual factors such as human operator knowledge and payoffs may well moderate this value considerably (Lee and See 2004).”
BackgroundResultInfluentialA model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 2×
“( 2000 ), diagnostic aiding refers both to stage 1 automation (e.”
BackgroundSelf-citationPerformance Consequences of Automation-Induced 'Complacency'
Parasuraman, R., Molloy, R., & Singh, I. L. (1993) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0301_1 · Cited by 875 · cited 2×
“In both of these cases, the automation benefits are likely to increase, to the extent that the single task to be automated is more difficult (Maltz and Shinar 2003, Dixon and Wickens 2006), and/or that concurrent task load is imposed (Parasuraman et al. 1993).”
BackgroundDoes automation bias decision-making?
SKITKA, L. J., MOSIER, K. L., & BURDICK, M. (1999) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1999.0252 · Cited by 694
Yeh, M., & Wickens, C. D. (2001) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872001775898269 · Cited by 175
Self-citation
Research works printed but not linked: 29
- 10.1016/j.apergo.2003.07.003 · DOI
- Davison HJ, 2001, Proceedings of the 11th International Symposium on Aviation Psychology
- 10.1518/001872097778543930 · DOI
- 10.1518/001872005774860005 · DOI
- 10.1518/001872006778606822 · DOI
- Dzindolet MT, 2001, Proceedings of the 45th Annual Meeting of the Human Factors and Ergonomics Society, 532
- Galster SM, 2001, Proceedings of the 45th Annual Meeting of the Human Factor Society, 321
- 10.1037/1076-898X.1.1.19 · DOI
- Lee JD, 2004, Human Factors, 46, 50, 10.1518/hfes.46.1.50.30392 · DOI · Paper page
- 10.1016/S0925-7535(00)00043-6 · DOI
- 10.1518/001872006777724408 · DOI · Paper page
- 10.1518/001872001775900931 · DOI
- 10.1518/hfes.45.2.281.27239 · DOI
- 10.1518/hfes.46.2.357.37348 · DOI
- 10.1518/0018720053653802 · DOI
- 10.1518/001872001775870395 · DOI
- 10.1518/001872096779048093 · DOI
- Sheridan T, 2002, Humans and Automation: System Design and Research Issues
- Sheridan, T and Parasurman, R. 2006. Human automation interaction. Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting. 2006. Edited by: Nickerson, R. Santa Monica, CA: Human Factors and Ergonomics Society.
- 10.1207/s15327051hci0101_2 · DOI
- John M, 2002, Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting
- Swets JA, 1982, The Evaluation of Diagnostic Systems
- Thomas LC, 2003, Proceedings of the 47th Annual Human Factors Ergonomics Society Conference
- Wickens CD, 2006, Human Factors of Remotely Operated Vehicles
- 10.1207/STHF0202_01 · DOI
- Wiegmann DA, 2003, Human Factors
- Xu X, Ergonomics
- 10.1518/0018720024497925 · DOI
- 10.1518/hfes.45.3.390.27249 · DOI
Cited by
Sorted by citations
Complacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710
“Wickens and Dixon (2007) proposed that this cutoff value is approximately 70% (with a standard error of ± 14%), on the basis of studies examining the beneficial effects of automation support.”
BackgroundFrom Here to Autonomy: Lessons Learned From Human–Automation Research
Endsley, M. R. (2017) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720816681350 · Cited by 898
“The presence of competing tasks has also been found to increase trust in automation in a metaanalysis of studies (Wickens & Dixon, 2007), thus shifting operator attention away from the automation and decreasing SA of the functions being automated (Carsten, Lai, Barnard, Jamson, & Merat, 2012; Kaber…”
BackgroundParasuraman, R., Sheridan, T., & Wickens, C. (2008) · Journal of Cognitive Engineering and Decision Making · doi:10.1518/155534308x284417 · Cited by 729
Self-citationHuman Performance Consequences of Stages and Levels of Automation: An Integrated Meta-Analysis
Onnasch, L., Wickens, C. D., Li, H., & Manzey, D. (2014) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720813501549 · Cited by 524
“It is also in line with other authors who also departed from the classic approach for similar reasons (e.g., Hutchins, Wickens, Carolan, & Cumming, 2013; Wickens & Dixon, 2007; Wickens et al., 2013; Wickens, Hooey, Gore, Sebok, & Koe-nicke, 2009).”
ResultSelf-citationAutomation bias and verification complexity: a systematic review
Lyell, D., & Coiera, E. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw105 · Cited by 451
Metadata
- Authors
- Wickens, C. D., & Dixon, S. R.
- Corresponding author
- Wickens, Christopher D.
- Venue
- Theoretical Issues in Ergonomics Science
- Publisher
- Informa UK Limited
- Published
- 2007-05
- Volume
- 8
- Issue
- 3
- Pages
- 201-212
- ISSN
- 1463-922X (ISSN-L)
- 1464-536X
- OpenAlex
- W2019160338
- Cited by
- 423
- References
- 0
- Citations by year
- 2026: 31 · 2025: 25 · 2024: 32 · 2023: 26 · 2022: 27 · 2021: 28 · 2020: 29 · 2019: 19 · 2018: 24 · 2017: 18 · 2016: 26 · 2015: 19 · 2014: 22 · 2013: 27 · 2012: 18
- FWCI
- 15.16
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Theoretical Issues in Ergonomics Science (publishedVersion)
- Funders
- University of Illinois at Urbana-Champaign
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Risk and Safety Analysis
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Keywords
- automated diagnosis, automation reliability, operator performance, human-automation interaction, workload, crossover point, dual-task performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
