The benefits of imperfect diagnostic automation: a synthesis of the literature

“Numerous contextual factors such as human operator knowledge and payoffs may well moderate this value considerably (Lee and See 2004).”

“( 2000 ), diagnostic aiding refers both to stage 1 automation (e.”

“In both of these cases, the automation benefits are likely to increase, to the extent that the single task to be automated is more difficult (Maltz and Shinar 2003, Dixon and Wickens 2006), and/or that concurrent task load is imposed (Parasuraman et al. 1993).”

“In broad terms, automation introduction generally improves the human-machine system performances in regular circumstances, but it can also tarnish performances in case of automation failures (see Wickens and Dixon 2007 for a literature review).”

“%) was close to the estimated “crossover point”, below which unreliable automation is typically worse than no automation at all; Wickens and Dixon, 2005).”

“These levels of accuracy were chosen based on prior work identifying 70% as the accuracy at which automation is considered better than the lack thereof [41].”

“Third, these fi ctitious AFRS had unchar-acteristically low accuracy, which can reduce the operator ’ s reliance on a decision-aid (Wickens & Dixon, 2007), and limits the generalization of these fi ndings to operational settings with more accurate systems.”

“Similarly, Wickens and Dixon [23] question the notions of complacency or reduced vigilance as explanations of automation bias.”

“37 In this context, Wickens and Dixon (2007) 38 reviewed the literature in search of indirect Figure 3.”

“even when it is imperfectly reliable, given that its reliability is greater than the threshold of automation assistance between 0.70 and 0.75 (Wickens & Dixon, 2007).”

“…that a reliability below 70% is the “ crossover point, ” where unreliable automation can be worse than no automation at all (Dixon and Wickens, 2006; Wickens and Dixon, 2007), future studies should further test the differences of conversational indicators of trust with a gradient level of…”

“According to a previous study, task performance can worsen by using an automated rather than a manual procedure when the reliability of automation is <70% [20].”

“The accuracy in each reliability condition was chosen based on prior work identifying 70% as the level at which automation is considered reliable [43].”

“Wolfe and Van Wert’s (2010) Multiple Decision Model (2010) proposed that under LP conditions false alarms should decrease.”

“Employing data from Metzger and Parasuraman as well as others in a metaanalysis, Wickens and Dixon (2007) suggested that compared to the manual condition people can perform detection tasks better with the automation whose reliability is as low as around 70%.”

“According to previous studies (e.g. Dixon et al., 2007; Dixon and Wickens; 2006; Rovira et al., 2007; Wickens and Dixon, 2007; Wickens et al., 2005) we expected that, compared to higher reliability levels, lower alarm reliabilities should result in significant performance decrements in the…”

“…& McCarley, 2007; Dzindo-let, Pierce, Beck, & Dawe, 1999; Mosier, Skitka, Heers, & Burdick, 1998; Rice, 2009; Rice & Hunt, 2009; Rice & Keller, 2009; Rice, Keller, Hunt, & Trafimow, 2009; Rice, Trafimow, Clayton, & Hunt, in press; Skitka, Mosier, Burdick, & Rosenblatt, 2000; Wickens & Dixon, 2007).”

“In this context, an unreliable warning may be allocated less attention, resulting in increased response delays (see Wickens & Dixon, 2005, for a discussion about the influence of reliability under high and low workload).”

“This was based on Wickens and Dixon’s finding that diagnostic aids with less than 70% reliability were worse for performance than no aid at all [46].”

“This trend would confirm a prediction based on an analysis of 22 studies that indicated a reliability level of 70% as the point below which unreliable assistance would be worse than no assistance at all (Wickens and Dixon 2007).”

“This is consistent with previous research that suggests that as long as automation reliability is greater than 70% and the task is not too easy, performance improves (Wickens & Dixon, 2007).”

“According to Wickens and Dixon [2007] below a level of around 70% accuracy, providing automation, assistance or guidance is worse than not providing it at all.”

“(2001) and others (Wickens and Dixon 2007; Ma and Kaber 2007; Chavaillaz et al.”

“Therefore, a cut-off value of 70% was chosen (Wickens & Dixon, 2007) in order to use a cut-off level where imperfection is perceived but not leading to an impression of complete faultiness.”

“Similarly, Wickens, Dixon and Johnson [12] demonstrated a greater cost associated with FAP automation (than with MP automation) that affected both the automated and concurrent tasks.”

“To the extent that automation is reliable and trusted (Wickens & Dixon, 2007), humanautomation performance should exceed human manual performance.”

“Research in other domains has shown that if the reliability rate of an alert is lower than 70%, professionals would perform better, overall, if they ignored the alert entirely and relied on their own detection and decision-making abilities (Wickens & Dixon, 2005).”

“This reliability is above the baseline threshold level of around 70% to be perceived as useful (Wickens & Dixon, 2007).”

“In their meta-analysis, Wickens and Dixon (2007) also noticed that trust in automation was able to modulate the 70% automation reliability threshold.”

“This also sheds light on Wickens and Dixon [31] findings that automation with below 70% reliability may not be beneficial.”

“In addition, previous research has indicated that 70% is a critical threshold for human trust (Wickens & Dixon, 2007).”

“The effect of automation error type on human performance (Wickens & Dixon, 2007) appears to depend on the task being performed.”

“Some automated systems can even harm performance making it worse than at baseline (Wickens & Dixon, 2007).”

“At 79% accuracy, our model is above the 0.70 reliability threshold wherein imperfect automation is preferable to no automation [32].”

“SAGAT queries were used to capture drivers’ situation awareness (SA) of their environment [35].”

“If reliability is below 70 per cent, it is better having no automation at all (Wickens & Dixon, 2007).”

“…perhaps unsurprising given previous research which has indicated that automation needs to be at least 70% reliable to be useful to human operators (Wickens & Dixon, 2007); however; the results demonstrate that for automation that increases in reliability, the 70% reliability level cannot simply…”

“This reliability exceeds the 70% reliability threshold which increases the probability that humans rely on automation (Wickens & Dixon, 2007), while providing enough data points for statistical power.”

“[47], but a level of imperfect reliability that is nevertheless helpful in aiding the human-automation team performance [48].”

“The selected reliability percentages were informed by prior research detailing critical threshold points of reliability induced automation complacency (Wickens & Dixon, 2007).”

“The observed percentages of unresolved hypoxemic episodes were compared with 70%, the lower limit of an acceptable true alarm rate for clinical utility (see DISCUSSION).(13) First, analyses were performed including all hypoxemic episodes, and then second, limited to 1 randomly selected episode from each case.”

“Wickens and Dixon (2007) explained that under a reliability level of 70%, the use of automation should be to ponder.”

“As most diagnoses are forced paced in the MCRs (Rouse 1978), the time pressure and interface design influence the human performance in digitalised task environments (Nikolic and Sarter 2007; Wickens and Dixon 2007).”

“A review of twenty automation reliability studies suggests that operator reliance on imperfect automation is greater under high task demands because the operator’s limited resources are being depleted (Wickens and Dixon, 2005).”

“Based on the analysis of 20 studies, Wickens and Dixon (2007) focused only on the information stages (stages 1 and 2); therefore, it is also necessary to investigate this threshold using action automation with a range of reliability levels.”

“The 60% reliability level was chosen on the basis of findings of a meta-analysis by Wickens and Dixon (2005) that operators often ignore alerts with reliability of less than 70% (which the authors called the “cross-over” point, below which unreliable automation was worse than no automation).”

“The goal of human-automation performance is to exceed that of the automation—a goal that the literature reveals to be quite difficult to attain (Wickens & Dixon, 2007; Dixon, Wickens & McCarley, 2007).”

“…falls below the critical reliability threshold of 70% identified in prior research as necessary for effective automation support (Onnasch, 2015; Wickens & Dixon, 2007), the reduced accuracy served a critical methodological purpose: it introduced sufficient disagreement between participants’…”

“This notion is acknowledged by Wickens and Dixon (2007) who state, in reference to their 70% performance threshold, that “ in addition to the fact that the constant is only an estimate (which was surrounded by a 95% confidence interval of roughly 0.14) … we acknowledge the potential impact of many…”

“Numerous other experiments have produced similar results (e.g., Bartlett & McCarley, 2017; Corcoran, Dennett, & Carpenter, 1970; Meyer, Wiczorek, & Gün-zler, 2013; for a review, see Wickens & Dixon, 2007).”

“…review has provided ample evidence for the multiple effects of reduced system reliability, two meta-analyses have suggested that low system reliability does not only have incremental effects but it may also involve a more qualitative change (Wickens and Dixon 2007; Onnasch, Ruff, and Manzey 2014).”

“…time for a discrepancy in Indicated Air Speed was not identified within 2 minutes (Dijk, Merwe, & Zon, 2011), suggesting that a longer period is required to identify contradictory information, particularly concerning systems that have proven to be reliable in the past (Wickens & Dixon, 2007).”

“Operators must rely on automation to help them overcome the complexity of these systems as they cannot safely and efficiently monitor and interact with these complex systems by themselves (Lee & See, 2004; Wickens & Dixon, 2007).”

“that the reliability below 70% is considered worse than no automation [66].”

“Given the review of studies from Wickens and Dixon (2007), it is interesting that we find a 70% threshold in some instances.”

“Although some studies have tested 60% reliability against various other levels, results have been mixed (Wickens & Dixon, 2005).”

“Previous research indicates that when automation has near perfect reliability, the human-automation team performs better than if the automation has degraded reliability (de Visser & Parasuraman, 2011; Dixon & Wickens, 2006; Dixon et al., 2006; Endsley, 2015; Wickens & Dixon, 2007).”

“Operators often fail to use an aid appropriately and therefore benefit less than possible from the aid’s assistance (Parasuraman & Riley, 1997; Wickens & Dixon, 2007).”

“In the field of aviation, automation has been progressively taking on a larger portion of the workload, while reliability in automation has been shown to improve task performance (Wickens & Dixon, 2007).”

“Considering the inherent uncertainty of automation, assuming perfect assistance from automation is unrealistic (Wickens & Dixon, 2005).”

“Researchers have explored the relationship between emergency signal reliability and task operator behavior extensively (see Wickens & Dixon, 2007).”

“Since a few human operators are usually employed in security surveillance, human-factors related issues also need to be carefully addressed, including cognitive ergonomics in human-machine interaction [6].”

“The AI accuracy was set as 82% to ensure that the AI would be perceived as useful while providing sufficient misses and false alarms [55].”

“It often involves the level of agreement with automation, regardless of the true state of the task (i.e. reliance and compliance; see Wickens & Dixon, 2007; Parasuraman & Manzey, 2010, for reviews).”

“This creates the potential for operators to erroneously act on incorrect advice (misuse; Lee & See, 2004) or disregard correct advice (disuse),1 although this is less likely with reliable automation (Wickens & Dixon, 2007).”

“The literature suggests that performers tend to have greater trust in, and utilize more often, those systems with the highest reported, experienced, or perceived reliability (Bahner, Huper, and Manzey 2008; Wickens and Dixon 2007; Skitka 1999; Bolstad and Endsley 1998; Levine and Samet 1973).”

“It might be the case that EID provides sufficient overview of the domain to allow information gathering to be performed with no effect of automation reliability [3], [14]–[17].”

“In the literatures of human – automation interaction (HAI), human – computer interaction, and human factors engineering, dozens of studies offer evidence of the negative influences of unreliable and imperfect automation and of automation failures (e.g., Chen et al., 2021; Wickens & Dixon, 2007).”