From Here to Autonomy: Lessons Learned From Human–Automation Research
Abstract
As autonomous and semiautonomous systems are developed for automotive, aviation, cyber, robotics and other applications, the ability of human operators to effectively oversee and interact with them when needed poses a significant challenge. An automation conundrum exists in which as more autonomy is added to a system, and its reliability and robustness increase, the lower the situation awareness of human operators and the less likely that they will be able to take over manual control when needed. The human-autonomy systems oversight model integrates several decades of relevant autonomy research on operator situation awareness, out-of-the-loop performance problems, monitoring, and trust, which are all major challenges underlying the automation conundrum. Key design interventions for improving human performance in interacting with autonomous systems are integrated in the model, including human-automation interface features and central automation interaction paradigms comprising levels of automation, adaptive automation, and granularity of control approaches. Recommendations for the design of human-autonomy interfaces are presented and directions for future research discussed.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Endsley (2017) adds that highly automated environments create out-of-the-loop performance problems, weakening the human operator's situation awareness and ability to intervene.
“Endsley (2017) adds that highly automated environments create out-of-the-loop performance problems, weakening the human operator's situation awareness and ability to intervene.”
From From Human Oversight to Cognitive Sovereignty: A Process-Based Governance Standard for AI-Assisted Legal Reasoning (2026) · cites this paper as Endsley (2017)
Human-AI interaction research diagnoses automation bias, inappropriate reliance and out-of-the-loop performance problems.
“Human-AI interaction research diagnoses automation bias, inappropriate reliance and out-of-the-loop performance problems (Lee & See, 2004;Hoff & Bashir, 2015;Endsley, 2017).”
From From Human Oversight to Cognitive Sovereignty: A Process-Based Governance Standard for AI-Assisted Legal Reasoning (2026) · cites this paper as Endsley, 2017
Trust calibration, automation bias, and out-of-the-loop performance problems are persistent human factors risks.
“Trust calibration, automation bias, and out-of-the-loop performance problems are persistent human factors risks (Lee & See, 2004;Hoff & Bashir, 2015;Endsley, 2017).”
From From Human Oversight to Cognitive Sovereignty: A Process-Based Governance Standard for AI-Assisted Legal Reasoning (2026) · cites this paper as Endsley, 2017
The format of data visualization affects how users interpret the underlying data and thereby influences their decision-making process.
“The format of data visualization affects how users interpret the underlying data and thereby influences their decision-making process (Endsley, 2017; Sosulski, 2018).”
From Check the box! How to deal with automation bias in AI-based personnel selection (2023) · cites this paper as Endsley, 2017
Presenting too little or highly aggregated information can decrease transparency and limit critical elaboration of the data.
“Conversely, presenting too little information, or information that is highly aggregated, can decrease transparency and limit critical elaboration of the data”
From Check the box! How to deal with automation bias in AI-based personnel selection (2023) · cites this paper as Endsley, 2017
Highlighting information and visualizing it in a highly aggregated form can be problematic because users tend to focus on highlighted information while ignoring contradictory information.
“Moreover, highlighting information and visualizing this information in a highly aggregated form can be problematic, as users tend to strongly focus on the highlighted information while ignoring contradictory information (Alberdi et al., 2009; Endsley, 2017).”
From Check the box! How to deal with automation bias in AI-based personnel selection (2023) · cites this paper as Endsley, 2017
The amount and format of represented data can impact how users process information and decision quality.
“Drawing from the field of visual analytics, the amount and format of the represented data of an AI-based system, such as a dashboard, can have a significant impact on how users process the information and how good the jointly reached decisions are”
From Check the box! How to deal with automation bias in AI-based personnel selection (2023) · cites this paper as Endsley, 2017
The presentation of strongly aggregated data might induce a more peripheral information processing.
“According to the ELM, the presentation of strongly aggregated data might induce a more peripheral information processing, as presenting only specific parts of the data might lead users to pay less attention to the entire underlying data (Endsley, 2017).”
From Check the box! How to deal with automation bias in AI-based personnel selection (2023) · cites this paper as Endsley, 2017
Cites
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 2× · see citing works
“Trust can be defined as “the attitude that an agent will help achieve an individual’s goals in a situation characterized by uncertainty and vulnerability” (Lee & See, 2004, p. 54).”
BackgroundHumans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · see citing works
A model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 1× · see citing works
“Parasuraman et al. (2000) presented a similar stage of automation taxonomy with the information monitoring factor divided into two levels, information filtering and information integration or inference, and with option generation and action selection combined in their action selection category.”
MethodA Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., & Parasuraman, R. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811417254 · Cited by 2,051 · cited 6× · see citing works
“There are also many individual differences that can be at play affecting cognitive constructs, such as trust (Hancock et al., 2011; Hoff & Bashir, 2015), SA (Endsley, 1995a, 2006; Endsley & Bolstad, 1994), and workload (Hancock & Meshkati, 1988), which in turn will create individual differences in…”
BackgroundMethodInfluentialThe power of testing memory: Basic research and implications for educational practice
Roediger, H. L., & Karpicke, J. D. (2006) · Perspectives on Psychological Science · doi:10.1111/j.1745-6916.2006.00012.x · Cited by 2,037 · see citing works
ENDSLEY, M. R., & KABER, D. B. (1999) · Ergonomics · doi:10.1080/001401399185595 · Cited by 1,088 · see citing works
Self-citationMUIR, B. M., & MORAY, N. (1996) · Ergonomics · doi:10.1080/00140139608964474 · Cited by 993 · see citing works
Performance Consequences of Automation-Induced 'Complacency'
Parasuraman, R., Molloy, R., & Singh, I. L. (1993) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0301_1 · Cited by 875 · see citing works
Flight-deck automation: promises and problems
WIENER, E. L., & CURRY, R. E. (1980) · Ergonomics · doi:10.1080/00140138008924809 · Cited by 599 · cited 2× · see citing works
“The flight management systems used in many aircraft are an example of this, with even well-trained pilots expressing surprise at system behaviors (McClumpha & James, 1994; Wiener & Curry, 1980).”
BackgroundTen challenges for making automation a ‘team player’ in joint human-agent activity
Klein, G., Woods, D. D., Bradshaw, J. M., Hoffman, R. R., & Feltovich, P. J. (2004) · IEEE Intelligent Systems · doi:10.1109/mis.2004.74 · Cited by 590 · see citing works
The benefits of imperfect diagnostic automation: a synthesis of the literature
Wickens, C. D., & Dixon, S. R. (2007) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500370105 · Cited by 423 · cited 1× · see citing works
“The presence of competing tasks has also been found to increase trust in automation in a metaanalysis of studies (Wickens & Dixon, 2007), thus shifting operator attention away from the automation and decreasing SA of the functions being automated (Carsten, Lai, Barnard, Jamson, & Merat, 2012; Kaber…”
BackgroundManzey, D., Reichenbach, J., & Onnasch, L. (2012) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343411433844 · Cited by 277 · see citing works
Sklar, A. E., & Sarter, N. B. (1999) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872099779656716 · Cited by 268 · see citing works
adaptive aiding for human/computer control
Rouse, W. B. (1988) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/001872088803000405 · Cited by 252 · see citing works
Imposing limits on autonomous systems
Hancock, P. A. (2017) · Ergonomics · doi:10.1080/00140139.2016.1190035 · Cited by 162 · see citing works
Status or Recommendation: Selecting the Type of Information for Decision Aiding
Crocoll, W. M., & Coury, B. G. (1990) · Proceedings of the Human Factors Society Annual Meeting · doi:10.1177/154193129003401922 · Cited by 68 · see citing works
Olson, W. A., & Sarter, N. B. (1999) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193129904300313 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Shneiderman, B. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1741118 · Cited by 1,743 · see citing works
Langer, M., Oster, D., Speith, T., Hermanns, H., Kästner, L., Schmidt, E., Sesing, A., & Baum, K. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103473 · Cited by 652 · see citing works
Xu, W., Dainoff, M. J., Ge, L., & Gao, Z. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2041900 · Cited by 363 · see citing works
Kraus, J., Scholz, D., Stiegemeier, D., & Baumann, M. (2020) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720819853686 · Cited by 276 · see citing works
Abbass, H. A. (2019) · Cognitive Computation · doi:10.1007/s12559-018-9619-0 · Cited by 200 · see citing works
Kunze, A., Summerskill, S. J., Marshall, R., & Filtness, A. J. (2019) · Ergonomics · doi:10.1080/00140139.2018.1547842 · Cited by 165 · see citing works
Algorithmic Decision-Making and the Control Problem
Zerilli, J., Knott, A., Maclaurin, J., & Gavaghan, C. (2019) · Minds and Machines · doi:10.1007/s11023-019-09513-7 · Cited by 145 · see citing works
Psychological factors influencing technology adoption: A case study from the oil and gas industry
Roberts, R., Flin, R., Millar, D., & Corradi, L. (2021) · Technovation · doi:10.1016/j.technovation.2020.102219 · Cited by 121 · see citing works
Salwei, M. E., & Carayon, P. (2022) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434221097357 · Cited by 112 · see citing works
Ironies of Generative AI: Understanding and Mitigating Productivity Loss in Human-AI Interaction
Simkute, A., Tankelevitch, L., Kewenig, V., Scott, A. E., Sellen, A., & Rintel, S. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2405782 · Cited by 89 · see citing works
Effective human–AI work design for collaborative decision-making
Jain, R., Garg, N., & Khera, S. N. (2023) · Kybernetes · doi:10.1108/k-04-2022-0548 · Cited by 85 · see citing works
The Impact of Training on Human–Autonomy Team Communications and Trust Calibration
Johnson, C. J., Demir, M., McNeese, N. J., Gorman, J. C., Wolff, A. T., & Cooke, N. J. (2023) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208211047323 · Cited by 68 · see citing works
Check the box! How to deal with automation bias in AI-based personnel selection
Kupfer, C., Prassl, R., Fleiß, J., Malin, C., Thalmann, S., & Kubicek, B. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1118723 · Cited by 59 · see citing works
“Drawing from the field of visual analytics, the amount and format of the represented data of an AI-based system, such as a dashboard, can have a significant impact on how users process the information and how good the jointly reached decisions are (Endsley, 2017; Sosulski, 2018).”
BackgroundMethodInfluentialProduct liability for defective AI
Buiten, M. C. (2024) · European Journal of Law and Economics · doi:10.1007/s10657-024-09794-z · Cited by 41 · see citing works
Unveiling Shadows: Harnessing Artificial Intelligence for Insider Threat Detection
Yilmaz, E., & Can, O. (2024) · Engineering, Technology & Applied Science Research · doi:10.48084/etasr.6911 · Cited by 32 · see citing works
Toward a science of human–AI teaming for decision making: a complementarity framework
Gonzalez, C., Donahue, K., Goldstein, D. G., Heidari, H., Jalali, M. S., Schelble, B., Singh, A., & Woolley, A. W. (2026) · PNAS Nexus · doi:10.1093/pnasnexus/pgag030 · Cited by 33 · see citing works
How Do We Assess the Trustworthiness of AI? Introducing the Trustworthiness Assessment Model (TrAM)
Schlicker, N., Baum, K., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2022) · doi:10.31234/osf.io/qhwvx · Cited by 79 · see citing works
A Computational Model of Coupled Human Trust and Self-confidence Dynamics
Williams, K. J., Yuh, M. S., & Jain, N. (2023) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3594715 · Cited by 18 · see citing works
Reduced autonomy workspace (RAW)—an interaction design approach for human-automation cooperation
Nylin, M., Johansson Westberg, J., & Lundberg, J. (2022) · Cognition, Technology & Work · doi:10.1007/s10111-022-00695-2 · Cited by 16 · see citing works
Shukla, S. (2026) · AI, Law, Politics · doi:10.5709/alp-02.01.2026-04 · Cited by 2 · see citing works
Mahy, T., & Li, H. (2026) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2026.2623216 · Cited by 2 · see citing works
TAN, K. H. (2026) · Open Access Journal of Multidisciplinary Research · doi:10.47760/oajmr.2026.v02i09.001 · Cited by 1 · see citing works
- Authors
- Endsley, M. R.
- Affiliations
- SA Technologies (United States) (US)
- Corresponding author
- Endsley, Mica R.
- ORCID
- Endsley, Mica R.
- Venue
- Human Factors: The Journal of the Human Factors and Ergonomics Society
- Publisher
- SAGE Publications
- Published
- 2016-12-15
- Volume
- 59
- Issue
- 1
- Pages
- 5-27
- ISSN
- 0018-7208 (ISSN-L)
- 1547-8181
- OpenAlex
- W2574287525
- Cited by
- 898
- References
- 17
- Citations by year
- 2026: 220 · 2025: 124 · 2024: 100 · 2023: 97 · 2022: 96 · 2021: 91 · 2020: 71 · 2019: 50 · 2018: 28 · 2017: 18 · 2016: 1 · 2015: 2
- FWCI
- 5.09
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Human Factors The Journal of the Human Factors and Ergonomics Society (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Engineering › Biomedical Engineering › Ergonomics and Human Factors
- Keywords
- situational awareness, trust in automation, levels of automation, adaptive automation, human performance, monitoring
- MeSH terms
- Automation, Humans, Man-Machine Systems, Task Performance and Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
