Flight-deck automation: promises and problems

“System operators working with automation have been found to have a diminished ability to detect system errors and subsequently perform tasks manually in the face of automation failures as compared with manual performance on the same tasks (Billings, 1991; Moray, 1986; Wickens, 1992a; Wiener and Curry, 1980).”

“Automated warning systems can also lead 242-June 1997 to reliance on warning signals as the primary indicator of system malfunctions rather than as secondary checks (Wiener & Curry, 1980).”

“unbalanced mental workload, reduced situation awareness, complacency, or skill loss may nevertheless be associated with other human performance problems that ultimately impact on system performance, including mode confusion and low operator trust in automation [1]–[10], [45]–[48].”

“This is not an aspect of human performance to be aimed for in computers, as it can raise problems with monitoring for failure (e.g. Wiener and Curry, 1980); automatic systems should fail obviously.”

“…automated and decision support systems constitutes an important area of inquiry in human factors and ergonomics (Bainbridge, 1983; Lee & Seppelt, 2009; Mosier, 2002; Parasuraman, 2000; R. Parasuraman, Sheridan, & Wickens, 2000; Rasmussen, 1986; Sheridan, 2002; Wiener & Curry, 1980; Woods, 1996).”

“The out-of-the-loop performance problem has been widely documented as a potential negative consequence of automation (Billings, 1991; Moray, 1986; Wickens, 1992; Wiener and Curry, 1980).”

“There is well documented evidence of the degradation of manual flying skills due to a lack of opportunity to practice in highly automated aircraft (e.g. Casner, Geven, Recker, & Schooler, 2014; Haslbeck & Hoermann, 2016; Wiener & Curry, 1980).”

“The flight management systems used in many aircraft are an example of this, with even well-trained pilots expressing surprise at system behaviors (McClumpha & James, 1994; Wiener & Curry, 1980).”

“At this point in time, serious problems associated with automation and numerous human–machine system errors have been documented (cf. Wiener and Curry 1980, Moray 1986, Billings 1991, Sarter and Woods 1995).”

“Automated systems have not always been used by human operators in the ways that designers intended ( Wiener & Curry, 1980; Woods, 1996).”

“…that leave operators with poor SA, slow to detect and to diagnose automation problems and failures, have been highly documented as a critical performance problem in many systems (Endsley & Kaber, 1996; Kaber & Endsley, 1997b, 2004; Parasuraman & Riley, 1997; Wiener & Curry, 1980; Young, 1969).”

“With the current level of technology, much is possible, but as Wiener and Curry (1980) already recognized, ''…the question is no longer whether one or another function can be automated, but, rather, whether it should be.''”

“It has been long known that automation can both hurt and benefit human performance (e.g., Bainbridge, 1983; Ephrath & Young, 1981; Kessel & Wickens, 1982; Rasmussen & Rouse, 1981; Sheridan, 2002; Wickens & Kes-sel, 1979, 1981; Wickens, Mavor, Parasuraman, & McGee, 1998; Wiener & Curry, 1980).”

“For instance, a machine is perceived as having unique properties such as invariance and perseverance (Wiener and Curry 1980, Connors et al. 1994) or the ability to perform consistently across situations.”

“Much of the impetus for understanding human interaction with automated systems stems from several accidents that have involved automation, an issue first raised in the aviation domain in a seminal paper by E. L. Wiener and Curry (1980).”

“…from reducing the human role in system control and problem solving (Amalberti, 1999; Bainbridge, 1983; Billings, 1997; Lewis, 1998; Parasuraman & Riley, 1997; Parasuraman et al., 2000; Rasmussen, 1986; Sarter et al., 1997; Satchell, 1998; Sheridan, 1992; Wickens et al., 1998; Wiener & Curry, 1980).”

“A seminal article by Wiener and Curry (1980) discussed these developments, posing many questions still being examined today: Does automation reduce human workload?”

“At the same time, problems in human–automation interaction have also been well documented, both in the laboratory and the field (Wiener and Curry 1980, Bainbridge 1983, Rasmussen 1986, Sheridan 1992, Billings 1997, Parasuraman and Riley 1997, Sarter et al. 1997, Lewis 1998, Satchell 1998,…”

“However, the increased prevalence and complexity of these automated systems has generated a corresponding increase in the demand for operators to monitor automated systems for failures or unanticipated states (Wiener and Curry 1980, Sarter and Woods 1995).”

“…when workload was already low (e.g., autopilot during the cruise phase) and increased mental workload in phases of flight when workload was already high (e.g., reprogramming the flight management system during final approach), a phenomenon referred to as clumsy automation (Wiener & Curry, 1980).”

“…human factors in automation over the past 50 years indicates that human factors issues are a key hurdle to overcome (Bainbridge, 1983; Billings, 1988; Endsley & Kiris, 1995; Lee, Wickens, Liu, & Boyle, 2017; Parasuraman & Riley, 1997; Sarter & Woods, 1995; Sheridan, 1992; Wiener & Curry, 1980).”

“Such a hypothesis is supported in a number of scientific publications , conference proceedings and incident reports (see for example, Weiner and Curry, 1980) and need not be considered further in detail here.”

“…and Human Factors Branch, 4676 Columbia Parkway, MS C24, Cincinnati, OH 45226-1998, USA. e-mail: ekh2@cdc.gov operator and system performance, new error forms and degraded performance have also been noted (Wiener and Curry 1980, Billings 1997, Parasuraman and Riley 1997, Sarter et al. 1997).”

“The changing role of the human operator from active controller to passive monitor, as exemplified by the modern airline pilot, has not eliminated the vigilance problem but merely changed it (Wiener and Curry, 1980).”

“…historic threads that reach back through Billings’s (1991) elegant work on human-centered automation, through early empirical and theoretical work on accidents induced by flight deck automation (Wiener & Curry, 1980) and the broader realm of other industrial environments (Rasmussen & Rouse, 1981).”

“The rapid digitalization of cockpits designed in the 1970s brought what Wiener and Curry (1980) called "promises and problems."”

“With the hindsight of over three decades, the influence of both Wiener and Curry’s [2] and Bainbridge’s [1] papers on contemporary thinking on automation is evident; but Bainbridge’s appears to have had a greater impact, if by no other measure than the number of their respective citations in…”

“It has been long known that automation can both hurt and benefit human performance (e.g., Bainbridge, 1983; Wiener & Curry, 1980; Sheridan, 2002).”

“There was little question as to whether or not automation would have an increased role in aviation, but the impact of that automation on crew performance was largely unsupported by research ( Wiener, 1989 ; Wiener & Curry 1980 ).”

“Furthermore, it is well known that for a fully automated system, there exists an “out-of-the-loop” problem [2], meaning that the driver cannot adequately supervise the system since he is no longer sufficiently involved in the driving task.”

“…areas of process control; the major handicap being that the operator’s knowledge and specifically skilled expertise is impoverished due to the lack of practice in manual procedures (in aeronautics, see Wiener & Curry, 1980; for control processes see de Keyser, 1986; Vicente & Rasmussen, 1987).”

“Detrimental effects of support systems have also been reported (primarily in aviation), such as complacency [46] and skill degradation [47], [48].”

“The central concept of the LOA is that automation is not an all-or-none phenomenon but instead can be implemented at various levels (e.g., Billings,1991; Lorenz & Parasuraman, 2007; Parasuraman, Sheridan, & Wickens, 2000; Wiener & Curry, 1980).”

“Such consequences were documented more than a decade before “Humans and Automation” in papers such as “Flight-Deck Automation: Promises and Problems” (Wiener & Curry, 1980) and “Ironies of Automation” (Bainbridge, 1983).”

“Much of the ideology behind driving automation has been around for some time in aviation systems, and there is considerable ergonomics knowledge about the promises and problems that can be extrapolated from the aviation domain (Billings, 1993; Stanton & Marsden, 1996; Wiener & Curry, 1980).”

“Problems with loss of skills needed for manual performance and decision-making have been reported in aviation (Jacobson 2010; National Transportation Safety Board 2010; Wiener and Curry 1980), information automation (Volz et al. 2016), and vehicle automation (Nordhoff et al. 2023), among others.”

“Similarly, research on the design of cognitive aids also shows a frequent lack of attention paid to preparatory activities (e.g., [ 37 ], [38]).”

“The increase in both quantity and complexity of advanced automated systems has also resulted in an increased demand for operators to monitor systems for failures or malfunctions (Wiener & Curry, 1980).”

“…in which humans are assigned to monitor automation or to monitor the environment for conditions beyond which the automation can operate (Bainbridge, 1983; Wiener & Curry, 1980), despite consistent findings that humans are ineffective at this task (Lee & Moray, 1992; Molloy & Parasuraman, 1996).”

“Rather, the success of modern technology depends to a large extent on its ability to function as a “team player” with human practitioners (e.g., Billings, 1996; Parasuraman & Mouloua, 1996; Wiener & Curry, 1980).”

“Among other weaknesses, such as the potential for programming errors (Wickens, 1992; Wiener and Curry, 1980), automation can lack the flexibility of human operators in the face of novel situations and thus difficulties can be encountered when the designers attempt to replace human problem solving abilities with automa…”

“Finally, we explore how automation such as GenAI can paradoxically lead to easy tasks being made easier and hard tasks made harder, a phenomenon we refer to as “task-complexity polarization” (also known as “clumsy automation” in Human Factors research (Wiener & Curry (1980); Section 3.4)).”

“Studies that revealed how pilots’ attention naturally drifted when automated systems were used prompted an industry-wide focus on the “out-of-the-loop phenomenon” (Endsley & Kiris, 1995; Wiener & Curry, 1980).”

“Indeed, con-flicts do not solely occur between humans operators, but also involve pilots and automation [12, 39].”

“Given that humans are not good at monitoring systems (Wiener and Curry, 1980; Bainbridge, 1983), the so-called “out-of-the-loop” (OOTL) performance problem can lead to new accident mechanisms (Endsley and Kirris, 1995).”

“Furthermore, it is well-documented in the aviation literature [11] that there is inevitable skill erosion with automation making human intervention to fly the aircraft a dicey proposition at best.”

“” Wiener and Curry [4] and Flad et al.”

“…responsibility and authority—can result in several adverse outcomes, one manifestation is a requirement for humans to monitor automation (Bainbridge, 1983; Wiener & Curry, 1980), despite consistent findings that humans are ineffective at this task (Lee & Molloy, 1992; Molloy & Parasuraman, 1996).”

“For commercial flight, where a one percent fuel savings across the entire fleet represents more than $100 million in annual fuel costs [19], the performance benefits of advanced technology can be significant.”

“Automation does not operate in an all-or-none fashion (Wiener and Curry 1980).”

“However, despite the apparent advantages of keeping drivers’ heads up and their eyes on the road [35], HUDs can also monopolize drivers’ attention through cognitive capture [70], potentially diverting them from from the broader context and leading to primary/secondary task inversion [73], where an operator begins to r…”

“There certainly is debate about whether the operator is a better monitor when actively in the loop or not (Wiener and Curry, 1980) and what are the best strategies for intervention (Wiener, 1993).”

“…domain is that humans are poor monitors (Hancock and Parasuraman, 1992; Kibler, 1965; Harris, 2002; Pritchett and Lewis, 2010; Sheridan, 1996; Wiener and Curry, 1980), or as Wiener (1985) put it: “ After three decades of highly proli fi c research on human vigilance (Mackie, 1977), we are…”

“…typically comes with the cost of alternate forms of human work-load (e.g. cognitive vs. visual motor) generated by the demands of automation for supervision in function coordination and performance to a goal (see Wiener and Curry (1980) for examples in the automated aircraft fl ight-deck).”

“Despite the benefits, Wiener and Curry (1980) suggested that automation has limitations, which will contribute to accidents.”

“Early warnings were raised by Wiener and Curry (1980) even prior to having broader data sets to examine this anticipated threat.”

“, 1980), at Chernobyl (though we may never know the full story there), and in highly automated transport aircraft (Wiener, 1985; Wiener and Curry, 1980) have demonstrated all too dramatically that it is time to rethink the old doctrines.”

“For instance, Wiener and Curry (1980) revealed that many pilots turned off the autopilot frequently to maintain their manual fl ying skills because of the fear of losing pro fi ciency.”