Implementing lumberjacks and black swans into model-based tools to support human-automation interaction
The DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index.
Abstract
Objective The objectives were to (a) implement theoretical perspectives regarding human-automation interaction (HAI) into model-based tools to assist designers in developing systems that support effective performance and (b) conduct validations to assess the ability of the models to predict operator performance. Background Two key concepts in HAI, the lumberjack analogy and black swan events, have been studied extensively. The lumberjack analogy describes the effects of imperfect automation on operator performance. In routine operations, an increased degree of automation supports performance, but in failure conditions, increased automation results in more significantly impaired performance. Black swans are the rare and unexpected failures of imperfect automation. Method The lumberjack analogy and black swan concepts have been implemented into three model-based tools that predict operator performance in different systems. These tools include a flight management system, a remotely controlled robotic arm, and an environmental process control system. Results Each modeling effort included a corresponding validation. In one validation, the software tool was used to compare three flight management system designs, which were ranked in the same order as predicted by subject matter experts. The second validation compared model-predicted operator complacency with empirical performance in the same conditions. The third validation compared model-predicted and empirically determined time t…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The risk of negative consequences associated with AI-enabled automation errors or failure increases with increasing levels of automation and in the latter functional stages of information processing (Sebok and Wickens, 2017).”
From Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy (2021) · cites this paper as Sebok and Wickens, 2017
In the citing paper
“some failures cannot be foreseen in advance and may not be included in the robot's planner (i.e., Black Swans; Sebok and Wickens, 2017).”
From Understanding and Resolving Failures in Human-Robot Interaction: Literature Review and Model Development (2018) · cites this paper as Sebok and Wickens, 2017
In the citing paper
“an increase in automation during failure condition reduces operator performance (the “lumberjack analogy”; Sebok and Wickens, 2017).”
From Understanding and Resolving Failures in Human-Robot Interaction: Literature Review and Model Development (2018) · cites this paper as Sebok and Wickens, 2017
Citations
Cites
A model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 1×
“Based on the original classic work of Sheridan and Verplank (1979), Parasuraman et al. (2000) considered that each stage within the taxonomy could be characterized by different levels of implementation.”
BackgroundSelf-citationThe black swan: the impact of the highly improbable
(2007) · Choice Reviews Online · doi:10.5860/choice.45-1430 · Cited by 3,454
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936 · cited 1×
“Examples of this dichotomy may include, in medicine, the distinction between automated diagnostic aids and treatment recommendations (Garg et al., 2005; Morrow, Wickens, & North, 2006).”
BackgroundTrust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817 · cited 2×
“• Capitalize on the meaning of the “loss of SA curve” and carefully craft intuitive displays of dynamic processes that can improve SA without imposing significant mental effort on the operator, in the spirit of ecological displays (Burns et al., 2008; Hoff & Bashir, 2015; Mercado et al., 2016).”
BackgroundComplacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710 · cited 1×
“Although there are many ways any given automation system can fail, based on the work of Mosier, Skitka, Heers, and Burdick (1998) and Parasuraman and Manzey (2010), an important dichotomy built into our S-PRINT model of automation failure detection was whether automation decision aid failed because…”
MethodThe Out-of-the-Loop Performance Problem and Level of Control in Automation
Endsley, M. R., & Kiris, E. O. (1995) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095779064555 · Cited by 1,406 · cited 1×
“Such a finding is also consistent with the general understanding that being out of the loop degrades awareness of what automation is doing (Endsley & Kiris, 1995; Kaber & Endsley, 2004), with a progressively greater loss as automation does more work and the human does less work.”
ResultHuman and Computer Control of Undersea Teleoperators
Sheridan, T. B., & Verplank, W. L. (1978) · doi:10.21236/ada057655 · Cited by 1,227 · cited 1×
“Based on the original classic work of Sheridan and Verplank (1979), Parasuraman et al. (2000) considered that each stage within the taxonomy could be characterized by different levels of implementation.”
BackgroundRensink, R. A. (2002) · Annual Review of Psychology · doi:10.1146/annurev.psych.53.100901.135125 · Cited by 850
Kaber, D. B., & Endsley, M. R. (2004) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922021000054335 · Cited by 789 · cited 1×
“Such a finding is also consistent with the general understanding that being out of the loop degrades awareness of what automation is doing (Endsley & Kiris, 1995; Kaber & Endsley, 2004), with a progressively greater loss as automation does more work and the human does less work.”
ResultSchaefer, K. E., Chen, J. Y. C., Szalma, J. L., & Hancock, P. A. (2016) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720816634228 · Cited by 768
Research works printed but not linked: 13
- Allender L., 2000, Proceedings of the Military, Government and Aerospace Simulation Symposium, 2000 Advanced Simulation Technologies Conference, 139
- Boehm-Davis D. A., 2002, Proceedings of the Twenty-Fourth Annual Conference of the Cognitive Science, 27
- Boorman D. J., 2004, Paper presented at the International Conference on Human-Computer Interaction in Aeronautics
- Degani A., 2013, The Oxford handbook of cognitive engineering, 433
- Foyle D., 2008, Human performance models in aviation
- Gore B. F., 2010, Human modelling in assisted transportation, 43
- Lüdtke A., 2010, Paper presented at the Human Computer Interaction Aero Conference 2010
- Lüdtke A., 2012, Paper presented at the Embedded Real Time Software and Systems Conference
- Lüdtke A., 2009, Proceedings of 7th IFIP WG 13.5 Working Conference, HESSD 2009, 54
- Morrow D. G., 2006, Annual review of human factors and ergonomics, 1, 254
- Mumaw R., 2006, Paper presented at HCI-Aero 2006, International Conference on Human Computer Interaction
- Pew R., 1997, Modeling cognitive and organizational performance
- Wickens C. D., Sebok A., Keller J., Peters S., Small R., Hutchins S., Algarín L., Gore B. F., Hooey B. L., Foyle D. C. (2013). Modeling and evaluating pilot performance in NextGen: Review of and recommendations regarding pilot modeling efforts, architectures, and validation studies (NASA/TM 2013–216504). Washington, DC: NASA.
Cited by
Dwivedi, Y. K., Hughes, L., Ismagilova, E., Aarts, G., Coombs, C., Crick, T., Duan, Y., Dwivedi, R., Edwards, J., Eirug, A., et al. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.08.002 · Cited by 4,611
“The risk of negative consequences associated with AI-enabled automation errors or failure increases with increasing levels of automation and in the latter functional stages of information processing (Sebok & Wickens, 2017).”
BackgroundHonig, S., & Oron-Gilad, T. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00861 · Cited by 338
“In turn, cognitive resources devoted to the failure have an impact on the task: an increase in automation during failure condition reduces operator performance (the “lumberjack analogy”; Sebok and Wickens, 2017).”
BackgroundThe “Out-of-the-Loop” concept in automated driving: proposed definition, measures and implications
Merat, N., Seppelt, B., Louw, T., Engström, J., Lee, J. D., Johansson, E., Green, C. A., Katazaki, S., Monk, C., Itoh, M., et al. (2019) · Cognition, Technology & Work · doi:10.1007/s10111-018-0525-8 · Cited by 282
Mueller, S. T., Hoffman, R. R., Clancey, W., Emrey, A., & Klein, G. (2019) · arXiv · doi:10.48550/arxiv.1902.01876 · Cited by 215
“…Miller, 1988; Muir, 1994; Muramatsu and Pratt, 2001; Pirolli and Card, 1999, 2005, Prietula et al., 2000, 2000; Roth, Butterworth, and Loftus, 1985; Sebok and Wickens, 2017; Strauch, 2017; Tullio et al., 2007; Vermeulen, Luyten, van den Hoven, and Coninx, 2013; Wallace and Freuder, 2001; Wille,…”
Ironies of Automation: Still Unresolved After All These Years
Strauch, B. (2018) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2017.2732506 · Cited by 139
Metadata
- Authors
- Sebok, A., & Wickens, C. D.
- Affiliations
- Alion Science and Technology (United States) (US)
- Consultant, Fort Collins, Colorado
- Corresponding author
- Sebok, Angelia
- Venue
- Human Factors: The Journal of the Human Factors and Ergonomics Society
- Publisher
- SAGE Publications
- Published
- 2016-09-27
- Volume
- 59
- Issue
- 2
- Pages
- 189-203
- ISSN
- 0018-7208 (ISSN-L)
- 1547-8181
- OpenAlex
- W2515673966
- Semantic Scholar
- be58d7b73cd79b3114c721dac2786d6975903b6b
- Cited by
- 75
- References
- 59
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 7 · 2023: 12 · 2022: 12 · 2021: 9 · 2020: 12 · 2019: 4 · 2018: 3 · 2017: 8
- FWCI
- 7.88
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Human Factors The Journal of the Human Factors and Ergonomics Society (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Aeronautics and Space Administration
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Risk and Safety Analysis
- Social Sciences › Human Factors and Ergonomics › Ergonomics and Human Factors
- Keywords
- human-automation interaction, black swan events, operator performance prediction, flight management system, automation design
- MeSH terms
- Automation, Humans, Man-Machine Systems, Models, Theoretical, Task Performance and Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Implementing lumberjacks and black swans into model-based tools to support human-automation interaction. https://app.cochat.ai/papers/10.1177/0018720816665201
- The paper
- Sebok, A., & Wickens, C. D. (2017). Implementing lumberjacks and black swans into model-based tools to support human-automation interaction. Human Factors: The Journal of the Human Factors and Ergonomics Society. https://doi.org/10.1177/0018720816665201
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.