Implementing lumberjacks and black swans into model-based tools to support human-automation interaction
The DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index.
Abstract
Objective The objectives were to (a) implement theoretical perspectives regarding human-automation interaction (HAI) into model-based tools to assist designers in developing systems that support effective performance and (b) conduct validations to assess the ability of the models to predict operator performance. Background Two key concepts in HAI, the lumberjack analogy and black swan events, have been studied extensively. The lumberjack analogy describes the effects of imperfect automation on operator performance. In routine operations, an increased degree of automation supports performance, but in failure conditions, increased automation results in more significantly impaired performance. Black swans are the rare and unexpected failures of imperfect automation. Method The lumberjack analogy and black swan concepts have been implemented into three model-based tools that predict operator performance in different systems. These tools include a flight management system, a remotely controlled robotic arm, and an environmental process control system. Results Each modeling effort included a corresponding validation. In one validation, the software tool was used to compare three flight management system designs, which were ranked in the same order as predicted by subject matter experts. The second validation compared model-predicted operator complacency with empirical performance in the same conditions. The third validation compared model-predicted and empirically determined time t…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The risk of negative consequences associated with AI-enabled automation errors or failure increases with increasing levels of automation and in the latter functional stages of information processing (Sebok and Wickens, 2017).”
From Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy (2021) · cites this paper as Sebok and Wickens, 2017
In the citing paper
“some failures cannot be foreseen in advance and may not be included in the robot's planner (i.e., Black Swans; Sebok and Wickens, 2017).”
From Understanding and Resolving Failures in Human-Robot Interaction: Literature Review and Model Development (2018) · cites this paper as Sebok and Wickens, 2017
In the citing paper
“an increase in automation during failure condition reduces operator performance (the “lumberjack analogy”; Sebok and Wickens, 2017).”
From Understanding and Resolving Failures in Human-Robot Interaction: Literature Review and Model Development (2018) · cites this paper as Sebok and Wickens, 2017
Citations
Cites
A model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 1×
“Based on the original classic work of Sheridan and Verplank (1979), Parasuraman et al. (2000) considered that each stage within the taxonomy could be characterized by different levels of implementation.”
BackgroundSelf-citationThe black swan: the impact of the highly improbable
(2007) · Choice Reviews Online · doi:10.5860/choice.45-1430 · Cited by 3,454
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936 · cited 1×
“Examples of this dichotomy may include, in medicine, the distinction between automated diagnostic aids and treatment recommendations (Garg et al., 2005; Morrow, Wickens, & North, 2006).”
BackgroundTrust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817 · cited 2×
“• Capitalize on the meaning of the “loss of SA curve” and carefully craft intuitive displays of dynamic processes that can improve SA without imposing significant mental effort on the operator, in the spirit of ecological displays (Burns et al., 2008; Hoff & Bashir, 2015; Mercado et al., 2016).”
BackgroundComplacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710 · cited 1×
“Although there are many ways any given automation system can fail, based on the work of Mosier, Skitka, Heers, and Burdick (1998) and Parasuraman and Manzey (2010), an important dichotomy built into our S-PRINT model of automation failure detection was whether automation decision aid failed because…”
MethodThe Out-of-the-Loop Performance Problem and Level of Control in Automation
Endsley, M. R., & Kiris, E. O. (1995) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095779064555 · Cited by 1,406 · cited 1×
“Such a finding is also consistent with the general understanding that being out of the loop degrades awareness of what automation is doing (Endsley & Kiris, 1995; Kaber & Endsley, 2004), with a progressively greater loss as automation does more work and the human does less work.”
ResultHuman and Computer Control of Undersea Teleoperators
Sheridan, T. B., & Verplank, W. L. (1978) · doi:10.21236/ada057655 · Cited by 1,227 · cited 1×
“Based on the original classic work of Sheridan and Verplank (1979), Parasuraman et al. (2000) considered that each stage within the taxonomy could be characterized by different levels of implementation.”
BackgroundRensink, R. A. (2002) · Annual Review of Psychology · doi:10.1146/annurev.psych.53.100901.135125 · Cited by 850
Kaber, D. B., & Endsley, M. R. (2004) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922021000054335 · Cited by 789 · cited 1×
“Such a finding is also consistent with the general understanding that being out of the loop degrades awareness of what automation is doing (Endsley & Kiris, 1995; Kaber & Endsley, 2004), with a progressively greater loss as automation does more work and the human does less work.”
ResultSchaefer, K. E., Chen, J. Y. C., Szalma, J. L., & Hancock, P. A. (2016) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720816634228 · Cited by 768
Research works printed but not linked: 13
- Allender L., 2000, Proceedings of the Military, Government and Aerospace Simulation Symposium, 2000 Advanced Simulation Technologies Conference, 139
- Boehm-Davis D. A., 2002, Proceedings of the Twenty-Fourth Annual Conference of the Cognitive Science, 27
- Boorman D. J., 2004, Paper presented at the International Conference on Human-Computer Interaction in Aeronautics
- Degani A., 2013, The Oxford handbook of cognitive engineering, 433
- Foyle D., 2008, Human performance models in aviation
- Gore B. F., 2010, Human modelling in assisted transportation, 43
- Lüdtke A., 2010, Paper presented at the Human Computer Interaction Aero Conference 2010
- Lüdtke A., 2012, Paper presented at the Embedded Real Time Software and Systems Conference
- Lüdtke A., 2009, Proceedings of 7th IFIP WG 13.5 Working Conference, HESSD 2009, 54
- Morrow D. G., 2006, Annual review of human factors and ergonomics, 1, 254
- Mumaw R., 2006, Paper presented at HCI-Aero 2006, International Conference on Human Computer Interaction
- Pew R., 1997, Modeling cognitive and organizational performance
- Wickens C. D., Sebok A., Keller J., Peters S., Small R., Hutchins S., Algarín L., Gore B. F., Hooey B. L., Foyle D. C. (2013). Modeling and evaluating pilot performance in NextGen: Review of and recommendations regarding pilot modeling efforts, architectures, and validation studies (NASA/TM 2013–216504). Washington, DC: NASA.
Cited by
Dwivedi, Y. K., Hughes, L., Ismagilova, E., Aarts, G., Coombs, C., Crick, T., Duan, Y., Dwivedi, R., Edwards, J., Eirug, A., et al. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.08.002 · Cited by 4,611
“The risk of negative consequences associated with AI-enabled automation errors or failure increases with increasing levels of automation and in the latter functional stages of information processing (Sebok & Wickens, 2017).”
BackgroundHonig, S., & Oron-Gilad, T. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00861 · Cited by 338
“In turn, cognitive resources devoted to the failure have an impact on the task: an increase in automation during failure condition reduces operator performance (the “lumberjack analogy”; Sebok and Wickens, 2017).”
BackgroundThe “Out-of-the-Loop” concept in automated driving: proposed definition, measures and implications
Merat, N., Seppelt, B., Louw, T., Engström, J., Lee, J. D., Johansson, E., Green, C. A., Katazaki, S., Monk, C., Itoh, M., et al. (2019) · Cognition, Technology & Work · doi:10.1007/s10111-018-0525-8 · Cited by 282
Mueller, S. T., Hoffman, R. R., Clancey, W., Emrey, A., & Klein, G. (2019) · arXiv · doi:10.48550/arxiv.1902.01876 · Cited by 215
“…Miller, 1988; Muir, 1994; Muramatsu and Pratt, 2001; Pirolli and Card, 1999, 2005, Prietula et al., 2000, 2000; Roth, Butterworth, and Loftus, 1985; Sebok and Wickens, 2017; Strauch, 2017; Tullio et al., 2007; Vermeulen, Luyten, van den Hoven, and Coninx, 2013; Wallace and Freuder, 2001; Wille,…”
Ironies of Automation: Still Unresolved After All These Years
Strauch, B. (2018) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2017.2732506 · Cited by 139
Ironies of artificial intelligence
Endsley, M. R. (2023) · Ergonomics · doi:10.1080/00140139.2023.2243404 · Cited by 99
“…2017b) In related work, Chris Wickens described the lum-berjack effect which states that while automation can create better performance in routine situations, it also can lead to more catastrophic outcomes in non-routine situations where automation is often ill suited (Sebok and Wickens 2017).”
BackgroundMental Workload: Assessment, Prediction and Consequences
Wickens, C. D. (2017) · Communications in computer and information science · doi:10.1007/978-3-319-61061-0_2 · Cited by 95
“But here, explicitly, choosing to rely upon automation at all, and choosing to rely on higher rather than lower degrees of automation, is a decision that incurs progressively greater risks (lower expected value) if the automation should unexpectedly fail [46].”
BackgroundSelf-citationAvetisyan, L., Ayoub, J., & Zhou, F. (2022) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2022.07.010 · Cited by 84
Automation Stages & Levels, 20 Years After
Wickens, C. D. (2017) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343417727438 · Cited by 72
“That is, people can agree on the four stage-defined functions above, as applied to any existing or proposed automation system, bearing in mind that a given system may accomplish more than one of the Source: Sebok and Wickens (2017).”
BackgroundSelf-citationInfluence of human-machine interactions and task demand on automation selection and use
Navarro, J., Heuveline, L., Avril, E., & Cegarra, J. (2018) · Ergonomics · doi:10.1080/00140139.2018.1501517 · Cited by 53
“Moreover, the lumberjack analogy postulates that the higher the degree of automation, the higher the performances impairment in case of automation failure (see Onnasch et al. 2014 for a meta-analysis; Sebok and Wickens 2017).”
BackgroundThe Absence of Degree of Automation Trade-Offs in Complex Work Settings
Jamieson, G. A., & Skraaning, G. (2019) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720819842709 · Cited by 40
“Wickens and colleagues have captured hypothesized DOA trade-offs in the lumberjack or routine-failure trade-off effect (see Figure 2, Onnasch, Wickens, Li, & Manzey, 2014; Sebok & Wickens, 2017; Wickens et al., 2010).”
BackgroundIMPACTS: a trust model for human-autonomy teaming
Hou, M., Ho, G., & Dunwoody, D. (2021) · Human-Intelligent Systems Integration · doi:10.1007/s42454-020-00023-x · Cited by 44
Strickland, L. J. G., Boag, R. J., Heathcote, A., Bowden, V. K., & Loft, S. D. (2023) · Journal of Experimental Psychology Applied · doi:10.1037/xap0000463 · Cited by 35
Sargent, R., Walters, B. A., & Wickens, C. D. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35596-7_16 · Cited by 29
Self-citationThe Effects of Continuous Driving-Related Feedback on Drivers’ Response to Automation Failures
Cohen-Lazry, G., Borowsky, A., & Oron-Gilad, T. (2017) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213601974 · Cited by 32
“Several authors (e.g., Sebok & Wickens, 2016) have suggested that in order to improve drivers’ performance and reaction times, it is vital to increase the levels of their Situation Awareness (SA; Endsley, 1995).”
BackgroundTenhundfeld, N. L., Demir, M., & de Visser, E. J. (2022) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434221096261 · Cited by 30
“This risk can be exacerbated by the rarity of failures and the level or degree of automation being used (Onnasch et al., 2014; Parasuraman et al., 2000; Sebok & Wickens, 2017).”
Warden, A. C., Wickens, C. D., Rehberg, D., Ortega, F. R., & Clegg, B. A. (2023) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2023.3302152 · Cited by 30
“This can be seen as a specific manifestation of the more general “Lumberjack” tradeoff principle of HAI: the more effective automation is when it is correct, the greater the consequences to the HATs performance when it is wrong [38], [59].”
BackgroundSelf-citationReacting and responding to rare, uncertain and unprecedented events
Hancock, P. A. (2022) · Ergonomics · doi:10.1080/00140139.2022.2095443 · Cited by 25
“As elaborated below, there are reasons to believe that increasing levels of technical systems’ reliability will foster the relative increase of the frequency of such blue swan appearances (and see Sebok and Wickens 2017).”
BackgroundThe Failure to Grasp Automation Failure
Skraaning, G., & Jamieson, G. A. (2023) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434231189375 · Cited by 21
“Sebok & Wickens, 2017, p. 191 (emphasis added) This liberal de fi nition classi fi es any operator misconception about automation design intentions as an automation failure.”
Lyu, M., Li, F., Lee, C., & Chen, C. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112086 · Cited by 23
Wickens, C. D., Onnasch, L., Sebok, A., & Manzey, D. (2020) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720820901957 · Cited by 21
Self-citationXu, X., Yu, R., & Yuan, M. (2023) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2023.103424 · Cited by 19
Interruption Management in the Context of Take-Over-Requests in Conditional Driving Automation
Borowsky, A., Zangi, N., & Oron-Gilad, T. (2022) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2022.3194006 · Cited by 18
Detecting automation failures in a simulated supervisory control environment
Foroughi, C. K., Sibley, C. M., Brown, N. L., Rovira, E. M., Pak, R., & Coyne, J. T. (2019) · Ergonomics · doi:10.1080/00140139.2019.1629639 · Cited by 17
“One possible caveat to this design approach is the challenge in human detection of very rare and unexpected automation failures, known as ‘ black swans ’ (Molloy and Parasuraman 1996; Sebok and Wickens 2017; Wickens et al. 2009).”
BackgroundPrinzel, L. J., Krois, P. A., Ellis, K. K. E., Vincent, M. J., Stephens, C. L., Oza, N. C., Chancey, E. T., Davies, M., Mah, R. W., Ackerson, J., et al. (2024) · AIAA SciTech Forum · doi:10.2514/6.2024-1603 · Cited by 25
Gillmore, S. C., & Tenhundfeld, N. L. (2020) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181320641020 · Cited by 16
“…issues of the loss of SA (Endsley, 2001), performance decrements once regaining control (Stanton et al., 1997), and even out-of-the-loop unfamiliarity (OOTLUF) over long periods of use (Sebok & Wickens, 2017), each of which could result in potentially disastrous consequences for the user or others.”
BackgroundMode Awareness Interfaces in Automated Vehicles, Robotics, and Aviation: A Literature Review
Özkan, Y. D., Mirnig, A. G., Meschtscherjakov, A., Demir, C., & Tscheligi, M. (2021) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3409118.3475125 · Cited by 15
Automated Vehicle Failure: The First Pedestrian Fatality and Public Perception
Tapiro, H., Wyman, A., Borowsky, A., Petzoldt, T., Wang, X., & Hurwitz, D. (2022) · Transportation Research Record Journal of the Transportation Research Board · doi:10.1177/03611981221083297 · Cited by 13
Understanding Automation Failure
Endsley, M. R. (2024) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434231222059 · Cited by 14
“Based on this history, Wickens (Onnasch et al., 2014; Sebok & Wickens, 2017) coined the term “ lumberjack effect ” to refer to the general fi nding that while automation may work well most of the time in typical or normal situations, the performance problems that occur in novel or unexpected…”
Human factors considerations for the context-aware design of adaptive autonomous teammates
Hauptman, A. I., Mallick, R., Flathmann, C., & McNeese, N. J. (2024) · Ergonomics · doi:10.1080/00140139.2024.2380341 · Cited by 15
“…workload, situational awareness, and human performance (Onnasch et al. 2014). in these instances, the proposed solutions extend mainly towards automation and include suggestions for enhanced interfaces that boost situation awareness (Wickens et al. 2010; sebok and Wickens 2017; Morgan 2022).”
Wickens, C. D., Fitzgerald, N. J., Clegg, B. A., Smith, C. A. P., Orth, D., & Kincaid, K. (2020) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181320641470 · Cited by 12
Self-citationHancock, P. A., Cruit, J. K., Kochan, J. A., Kaplan, A. D., Diaz, Y., & Pruchnicki, S. (2022) · The International Journal of Aerospace Psychology · doi:10.1080/24721840.2022.2090944 · Cited by 15
Yiu, C. Y., Ng, K. K. H., Li, Q., & Yuan, X. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3567987 · Cited by 11
Jamieson, G. A., & Skraaning, G. (2020) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720820904623 · Cited by 11
“…et al. (2019) offer five critiques of Jamieson and Skraaning (2019), which we meet by (a) referring to precedents set by Onnasch et al. (2014) and Sebok and Wickens (2017), (b) appropriately weighing evidence from subjective measures, and (c) providing additional methodological and statistical…”
BackgroundMethodOvercoming the Lumberjack Effect Through Adaptive Autonomy
Hauptman, A. I., & McNeese, N. J. (2022) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181322661372 · Cited by 11
Trapsilawati, F., Chen, C., Wickens, C. D., & Qu, X. (2021) · Journal of Navigation · doi:10.1017/s0373463320000703 · Cited by 11
“The basis of this prediction lies in the logic proposed by Sebok and Wickens (2017).”
BackgroundSelf-citationMeasuring Automation Bias and Complacency in an X-Ray Screening Task
Davis, J. R., Atchley, A., Barr, H. M., Simon, H., & Tenhundfeld, N. L. (2020) · Systems and Information Engineering Design Symposium · doi:10.1109/sieds49339.2020.9106670 · Cited by 13
“The likelihood of complacency to occur may be increased when the system is perceived to be reliable, as these systems lead to increased trust [16]–[18].”
BackgroundZhang, Y., Angell, L., & Bao, S. (2021) · Transportation Research Interdisciplinary Perspectives · doi:10.1016/j.trip.2021.100337 · Cited by 12
Return-to-Manual Performance can be Predicted Before Automation Fails
Griffiths, N., Bowden, V. K., Wee, S., & Loft, S. D. (2022) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208221147105 · Cited by 10
“These points notwithstanding, automation in safety-critical work settings such as ATC is likely to be highly reliable, with automation failures rare and unexpected ( ‘ Black Swans ’ , Sebok & Wickens, 2017).”
BackgroundThe Effects of Driver Coupling and Automation Impedance on Emergency Steering Interventions
Bhardwaj, A., Lu, Y., Pan, S., Sarter, N. B., & Gillespie, R. B. (2020) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc42975.2020.9282961 · Cited by 9
“These results are consistent with the lumberjack analogy [24]; when automation worked properly, the increased degree of automation could improve performance, but in the failure conditions, higher levels”
ResultIs my User Impaired? Designing Adaptive Automation that Monitors the User’s State
Cotter, J. E., Atchley, A., Banz, B. C., & Tenhundfeld, N. L. (2021) · Systems and Information Engineering Design Symposium · doi:10.1109/sieds52267.2021.9483731 · Cited by 10
“This is additionally compounded by the fact that with greater levels of automation, the consequences are more severe [19], and the more compromised is the driver’s ability to intervene in the appropriate manner [20].”
BackgroundTenhundfeld, N. L., Forsyth, J. B., Sprague, N. R., El-Tawab, S., Cotter, J. E., & Vangsness, L. (2023) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434231206422 · Cited by 8
“…and fail to provide appropriate supervisory control over the system (Merritt et al., 2019; Wickens et al., 2015), which can lead to degraded situation awareness, failure to detect system errors, and worse overall human – machine team performance (Manzey et al., 2012; Sebok & Wickens, 2017).”
BackgroundTowards a Theory on Process Automation Effects
Vu, H., Haase, J., Leopold, H., & Mendling, J. (2023) · Lecture notes in business information processing · doi:10.1007/978-3-031-41623-1_17 · Cited by 9
“The lumberjack analogy points out that as the level of automation increases, the performance of routine tasks improves, but the monitoring and reactivity of the human to failure scenarios significantly decreases [39].”
Lee, B. C., Park, J., Jeong, H., & Park, J. (2020) · Applied Sciences · doi:10.3390/app10041288 · Cited by 7
“This trade-off is analogous to the “lumber jack effect” in which “the higher trees in the forest are, the farther they fall” [3,11].”
BackgroundAutomation and the imbrication of human and material agency: A sociomaterial perspective
Viktorelius, M., MacKinnon, S. N., & Lundh, M. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102538 · Cited by 7
Wang, S., Liu, Y., Wang, X., Liu, Z., You, X., & Li, Y. (2024) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208241311808 · Cited by 6
“This result validates the lumberjack effect, a key concept in human-automation interaction (Sebok & Wickens, 2017).”
Engineering Interactive Systems Embedding AI Technologies
Dix, A. J., Mayer, S., Palanque, P. A., Panizzi, E., & Spano, L. D. (2023) · ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS) · doi:10.1145/3596454.3597195 · Cited by 6
“Even though having larger automation might induce larger failures (known as the lumberjack analogy [16]), integrating such AI-related technologies can be performed at various levels, from micro to macro, requiring different (and maybe conflicting) engineering approaches.”
BackgroundComparing Coupled and Decoupled Steering Interface Designs for Emergency Obstacle Evasion
Bhardwaj, A., Lu, Y., Pan, S., Sarter, N. B., & Gillespie, R. B. (2021) · IEEE Access · doi:10.1109/access.2021.3106225 · Cited by 6
“If the automation system is unreliable, giving automation more authority precludes the driver from covering for automation faults [36].”
MethodINDUSTRY 4.0:SOCIAL CHALLENGES AND RISKS
Marzano, G., & Sigüencia, L. O. (2021) · Environment Technology Resources Proceedings of the International Scientific and Practical Conference · doi:10.17770/etr2021vol2.6546 · Cited by 5
“Moreover, according to [17], many types of automation failures, including software bugs and hardware failures, lie in how automation performs as the designer intended but not as the user intended.”
BackgroundAdaptive and Adaptable Automation in manufacturing: a bibliometric review
Vigoroso, L., Caffaro, F., Tronci, M., & Fargnoli, M. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.01.164 · Cited by 4
Measuring Safety and Performance in Human–Automation Systems: Special Issue Commentary
Márquez, J. J., & Gore, B. F. (2017) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720817696300 · Cited by 4
“Sebok and Wickens’s (2017) contribution addresses two types of human–machine system failures: the “lumberjack” and the “black swan.” The authors used three model-based tools to evaluate these types of failures in different automated systems. The methods Sebok and Wickens describe are found to be useful for predicting…”
BackgroundMethodInfluentialHuman and machine: Evaluating whether action automation influences visual perception
Tenhundfeld, N. L., & Witt, J. K. (2020) · Attention Perception & Psychophysics · doi:10.3758/s13414-020-02037-8 · Cited by 4
Stumbling Towards a Shared Apprehension of Automation Failure
Jamieson, G. A., & Skraaning, G. (2024) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434241292400 · Cited by 3
“But why, then, would we refrain from embracing Sebok and Wickens ’ (2017) broad perspective on automation failure and count any unexpected automation behavior as a failure mechanism?”
Reconceptualizing Trust: Defining, Modeling, and Measuring Trust
Erchov, S. (2017) · doi:10.13021/mars/5458 · Cited by 3
Bowden, V. K., Gegoff, I., Kilpatrick, P., & Loft, S. D. (2025) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208251335536 · Cited by 2
“This is known as the “ lumberjack ” effect, where increased DOA improves routine performance, but impairs performance under automation failure conditions (Sebok & Wickens, 2017).”
Quandt, R. P. (2022) · Frontiers in Physics · doi:10.3389/fphy.2022.941824 · Cited by 2
A New Framework for an Old Idea: Overhauling Reliability to Meet Current and Future Needs
Cotter, J. E., Atchley, A., Barr, H. M., Weger, K., Mesmer, B. L., Menon, V., Gholston, S. E., & Tenhundfeld, N. L. (2023) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/21695067231192935 · Cited by 2
“…than errors which occur later on (Rossi et al., 2017), which can in turn impact workload (Bailey & Scerbo, 2007; de Visser & Parasuraman, 2011), situation awareness (Parasuraman et al., 2008), performance (Sebok & Wickens, 2017), and reliance strategies (Du et al., 2019; Dzindolet et al., 2003).”
BackgroundA Critical Investigation of Rationalities in Automation with BPM
Klessascheck, F., Bein, L., Haase, J., & Pufahl, L. (2024) · IEEE Conference on Business Informatics · doi:10.1109/cbi62504.2024.00014 · Cited by 2
Modernizing CI/CD Pipelines: Migrating from Legacy Tools to GitLab for Enterprise Applications
Koneru, N. M. K. (2021) · International Journal of Science and Research Archive · doi:10.30574/ijsra.2021.1.2.0027 · Cited by 2
Sebok, A. L., Walters, B. A., & Wickens, C. D. (2017) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213601970 · Cited by 2
“This performance decrement is referred to as the Lumberjack Analogy (Onnasch et al., 2013; Sebok & Wickens, 2016), and is related to the OOTL phenomenon described earlier.”
MethodSelf-citationWu, M., Zhang, L., Li, W., Wan, L., Lu, N., & Zhang, J. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00292 · Cited by 2
“, 2017), and many other backgrounds (Vicente et al., 2004; Wickens et al., 2009; Sebok and Wickens, 2016).”
Designing visualization and interaction for industrial control rooms of the future
Domova, V. (2020) · Linköping studies in science and technology. Dissertations · doi:10.3384/diss.diva-165625 · Cited by 4
Festschrift in honour of Professor Neville Stanton: A lone crusader in a world of driving simulators
Walker, G. H., Eriksson, A., Clark, J. R., & Young, M. S. (2021) · Applied Ergonomics · doi:10.1016/j.apergo.2021.103594 · Cited by 2
The Human‐Automation Behavioral Interaction Task (HABIT) analysis framework
Baird, I., Fendley, M. E., & Warren, R. (2022) · Human Factors and Ergonomics in Manufacturing & Service Industries · doi:10.1002/hfm.20963 · Cited by 2
“Extensive human factors research agrees that automation can impair the human operator's situation awareness and cause OOTL performance problems and system failures (Lee, 2006; Onnasch et al., 2014; Parasuraman & Manzey, 2010; Parasuraman et al., 1993; Sebok & Wickens, 2017).”
BackgroundHuman-Machine Function Allocation Method Based on a Non-Cooperative Game for the Manned Submersible
Liao, W., Chen, D., Qiao, Y., Wang, H., Gou, Z., & Yang, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3315603 · Cited by 2
INTO THE BLACK BOX: DESIGNING FOR TRANSPARENCY IN ARTIFICIAL INTELLIGENCE
Vorm, E. S. (2019) · IUScholarWorks (Indiana University) · doi:10.13140/rg.2.2.12257.33120 · Cited by 3
Human As Automation Failsafe: Concept, Implications, Guidelines and Innovations
Miller, C., Shively, J., Brandt, S. L., Wauck, H. C., Sarathy, V., & Freedman, R. G. (2022) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181322661014 · Cited by 1
Individual differences in multi-tasking ability moderate the benefits of using low-degree automation
Greenwell-Barnden, J. N., Loft, S. D., Bowden, V. K., Bender, A. D., Whitney, S. J., Lipp, O. V., & Visser, T. A. W. (2025) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2024.2446848 · Cited by 2
“Thus, we suggest the lack of an interaction between multi-tasking and automation chiefly reflects the inconsistent benefits of automation on SA (Onnasch, Wickens, and Manzey 2014; Sebok and Wickens 2017).”
Identifying Human Out-of-the-Loop in Cruising Flights Using EEG Spectral Features with Deep Learning
Yiu, C. Y., Ng, K. K. H., Li, Q., & 中原, 信. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-94153-5_12 · Cited by 1
Dickerson, K., Watkins, H., Sparks, D., Green, N. H., & Morrow, S. L. (2025) · Energies · doi:10.3390/en19010220 · Cited by 1
Greenwell-Barnden, J. N., Visser, T. A. W., Loft, S. D., Whitney, S. J., & Bowden, V. K. (2026) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208261431968 · Cited by 1
“Alternatively, automation may work as intended by the designer but not as the operator expects (Sebok & Wickens, 2017).”
Transparency and Conflict Resolution Automation Reliability in Air Trffic Control
Trapsilawati, F., Wickens, C. D., Chen, C., & Qu, X. (2017) · Journal of Bioresource Management · Cited by 1
Self-citationAlerts on the NextGen Flight Deck
Sebok, A. L., Wickens, C. D., Walters, B. A., & Fennell, K. (2017) · CORE Scholar (Wright State University) · Cited by 1
Self-citationBehavioral Adaptation to Driving Automation Systems: Guidance for Consumer Education
Noble, A. M. (2020) · VTechWorks (Virginia Tech) · Cited by 3
Cognition and Interaction: From Computers to Smart Objects and Autonomous Agents
Amon Rapp, M. T., Tirassa, M., & Ziemke, T. (2019) · Frontiers research topics · doi:10.3389/978-2-88963-002-8
“In turn, cognitive resources devoted to the failure have an impact on the task: an increase in automation during failure condition reduces operator performance (the “lumberjack analogy”; Sebok and Wickens, 2017).”
BackgroundAn Approach to Modeling Simulated Military Human-agent Teaming
Hidalgo, M. (2020) · Journal of International Crisis and Risk Communication Research · Cited by 1
Towards a Comprehensive Model of Military Human-Agent Teaming
Hidalgo, M., & Jones, L.-R. (2023) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/21695067231192436
“Research shows that the level of automation assigned to the agent is beneficial for routine task performance, although it may also lead to problems with human take-over qualities and situation awareness (Sebok & Wickens, 2017).”
BackgroundEffects of Automated Diagnostic Tools on Human Performance during Failure Management
Ung, K. (2025) · PolyPublie (École Polytechnique de Montréal) · Cited by 2
Fourth Workshop on Engineering Interactive Systems Embedding AI Technologies
Campos, J. C., Fayollas, C., Luyten, K., Palanque, P. A., Panizzi, E., Pascher, M., Paternò, F., & Spano, L. D. (2026) · ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS) · doi:10.1145/3807968.3822519
PILOT EXPERIENCES AND PERCEPTIONS REGARDING TRAINING FOR UNEXPECTED EVENTS
Sebok, A. L., & Ku, H. (2026) · Performance Improvement Quarterly · doi:10.56811/piq-25-006
Self-citation
Metadata
- Authors
- Sebok, A., & Wickens, C. D.
- Affiliations
- Alion Science and Technology (United States) (US)
- Consultant, Fort Collins, Colorado
- Corresponding author
- Sebok, Angelia
- Venue
- Human Factors: The Journal of the Human Factors and Ergonomics Society
- Publisher
- SAGE Publications
- Published
- 2016-09-27
- Volume
- 59
- Issue
- 2
- Pages
- 189-203
- ISSN
- 0018-7208 (ISSN-L)
- 1547-8181
- OpenAlex
- W2515673966
- Semantic Scholar
- be58d7b73cd79b3114c721dac2786d6975903b6b
- Cited by
- 75
- References
- 59
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 7 · 2023: 12 · 2022: 12 · 2021: 9 · 2020: 12 · 2019: 4 · 2018: 3 · 2017: 8
- FWCI
- 7.88
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Human Factors The Journal of the Human Factors and Ergonomics Society (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Aeronautics and Space Administration
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Risk and Safety Analysis
- Social Sciences › Human Factors and Ergonomics › Ergonomics and Human Factors
- Keywords
- human-automation interaction, black swan events, operator performance prediction, flight management system, automation design
- MeSH terms
- Automation, Humans, Man-Machine Systems, Models, Theoretical, Task Performance and Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.