Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 695Human-Automation Interaction and SafetyTeam Dynamics and PerformanceSocial Robot Interaction and HRI

The DOI matches its Crossref record; 155 of the 155 references it cites were found in a registry or an index; none of the 252 sources its claims cite is checked for support yet.

Human–Autonomy Teaming: A Review and Analysis of the Empirical Literature

Open access
O’Neill, Thomas, McNeese, Nathan, Barron, Amy, Schelble, Beau·2022·Human Factors: The Journal of the Human Factors and Ergonomics Society

Abstract

Objective: We define human–autonomy teaming and offer a synthesis of the existing empirical research on the topic. Specifically, we identify the research environments, dependent variables, themes representing the key findings, and critical future research directions. Background: Whereas a burgeoning literature on high-performance teamwork identifies the factors critical to success, much less is known about how human–autonomy teams (HATs) achieve success. Human–autonomy teamwork involves humans working interdependently toward a common goal along with autonomous agents. Autonomous agents involve a degree of self-government and self-directed behavior (agency), and autonomous agents take on a unique role or set of tasks and work interdependently with human team members to achieve a shared objective. Method: We searched the literature on human–autonomy teaming. To meet our criteria for inclusion, the paper needed to involve empirical research and meet our definition of human–autonomy teaming. We found 76 articles that met our criteria for inclusion. Results: We report on research environments and we find that the key independent variables involve autonomous agent characteristics, team composition, task characteristics, human individual differences, training, and communication. We identify themes for each of these and discuss the future research needs. Conclusion: There are areas where research findings are clear and consistent, but there are many opportunities for future research. Particularly important will be research that identifies mechanisms linking team input to team output variables.

Overview

CoChat Summary

TL;DR

This integrative review synthesizes 76 empirical articles on human–autonomy teaming (HAT), defines HATs as interdependent human(s) and autonomous agent(s) occupying distinct team roles, and maps findings using an I-M-O framework. It finds generally positive effects of higher agent autonomy, reliability, interdependence, and training, but human–human teams usually outperform HATs and transparency/communication quantity show mixed effects.

Key findings

  • The review identified 76 empirical articles; all were simulation/lab studies, none field-based. Of these, 57 were military- or emergency-related simulations, 17 were nonmilitary/emergency simulations, and 2 could not be classified; the most represented testbeds were CERTT-UAS-STE (18 articles) and MIX (15 articles).
  • Agent autonomy was coded as high in 26 articles, partial in 31, and multiple levels in 15. Higher agent autonomy generally had positive effects (e.g., easier to work with, higher communication efficiency, stronger coordination, higher performance, lower workload), though some studies found a moderate level ideal; reliability was consistently positive, while transparency had mixed effects (e.g., improved performance/trust/understanding but also complacency, conflict, and workload).
  • Human–human teams generally outperformed HATs; interdependence led to positive outcomes; higher task difficulty led to negative outcomes such as higher workload and lower performance; training produced consistently positive results; communication quality was associated with stronger team functioning, but communication quantity findings were mixed.
  • Dependent variables studied most often were team and individual performance (70 studies), workload (39), trust (24), situation awareness (23), team coordination (15), and shared mental models (6); little research addressed Hackman’s team viability and individual learning/growth outcomes.

Methods

  • Narrative/integrative literature review with four-step search. Inclusion required empirical research, sufficient description to code agent autonomy, and at least one autonomous agent and one human working interdependently toward a common goal; the agent had to reach at least Level 5 on Parasuraman et al.’s (2000) automation continuum.
  • Searches covered ACM Digital Library, IEEE Xplore, and Sage Journals: initial search returned 1,942 articles, reduced to 476 after full-text “team” filtering, yielding 33 included; author-focused search returned 930 potential articles, yielding 31 included; Google Scholar search returned 152 articles, yielding 8 included; adding “human–automation interaction” yielded 4 more; total unique articles = 76.
  • Coding was inductive and iterative, organized within the I-M-O model of team effectiveness. After calibration, 30% of remaining articles were randomly selected for independent coding; inter-coder agreement was 89% overall.

Limitations

  • Stated: the search may have missed relevant articles; using “teamwork” as a search term may have constrained coverage; a disproportionate number of papers came from two laboratories/testbeds (CERTT-UAS-STE 24%, MIX 20%); it is unclear whether humans perceived the autonomous agents as team members; and HRI and autonomous vehicle literatures were omitted.
  • Evident from the text: all included studies were simulation/lab-based, most involved short-term “zero-history” teams, often single human–single agent dyads, limited communication, and mostly partial rather than high autonomy, limiting generalizability.
  • Evident from the text: no studies formally reported mediation analyses, so mechanisms linking team inputs to outputs remain largely untested.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1177/0018720820960865

Cites

155 of 155 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Inputs are theorized to influence mediating mechanisms, which in turn affect multi-level human–autonomy teaming outcomes. Most individual studies in the current review only considered a single path in the I-M-O chain (i.e., did not test mediating mechanisms). Source. Adapted from Kazi et al. (in press).
Figure 1 Inputs are theorized to influence mediating mechanisms, which in turn affect multi-level human–autonomy teaming outcomes. Most individual studies in the current review only considered a single path in the I-M-O chain (i.e., did not test mediating mechanisms). Source. Adapted from Kazi et al. (in press).CC BY-NC · PubMed Central
Figure 2 Histogram of the number of publications in the current review at each year.
Figure 2 Histogram of the number of publications in the current review at each year.CC BY-NC · PubMed Central

Verification

Claims the paper makes

0 Valid · 10 Not valid · 200 Not checked · 136 Own results and interpretations

179 more claims

Create a free account to see all 346 claims and to verify the 150 references still open.

See all claims — free account
  • Probably not a substantial claim. 118
  • It may state two separate claims as one. 80

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 155 references

Research works printed but not linked: 27

  1. Abadi M., 2016, 12th Symposium on Operating Systems Design and Implementation, 16, 265
  2. *Boyce M. W., 2015, Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts (pp. 179–180).
  3. Byrne D. E, 1971, The attraction paradigm
  4. *Chen J. Y., 2013, [Conference session]. 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 103–104)
  5. *Chen J. Y., 2016, Companion Publication of the 21st International Conference on Intelligent User Interfaces (IUI) (pp. 28–31).
  6. *Chien S., 2016, In 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (pp.2884–2889).
  7. *Demir M., 2016, 2016 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA) (pp. 28–34). IEEE
  8. *Demir M., 2019, 2019 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA) (pp. 122–126).
  9. Endsley M. R., 2012, Designing for situation awareness: An approach to human-centered design
  10. Endsley M. R., 1995, Situation awareness global assessment technique (SAGAT) TRACON air traffic control version user guide
  11. Fitts P. M, 1951, Human engineering for an effective air-navigation and traffic-control system
  12. Hackman J. R, 1990, Groups that work and those that don't (No E10 H123)
  13. *Hafızoğlu F. M., 2018, Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS) (pp. 514–522).
  14. *Hanna N., 2014, Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems (pp. 1485–1486).
  15. Hanna N., 2014, Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems (pp.1369–1370).
  16. *Hanna N., 2015, Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems (pp. 1777–1778).
  17. Kazi S., Human Factors.
  18. Kidd C. D., 2005, Proceeding of AISB, 5, 141
  19. 10.1037/0021-9010.61.4.395 · doi:10.1037/0021-9010.61.4.395 · Paper page
  20. Powers A., 2007, Proceedings of the Acm/ieee International Conference on Human-Robot Interaction (pp. 145–152).
  21. Reig S., 2019, 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 742–744).
  22. Sheridan T. B, 1980, Technology Review, 83, 60
  23. 10.1177/014920639902500205 · doi:10.1177/014920639902500205 · Paper page
  24. Woods D. D, 1985, Conference Record for 1985 IEEE Third Conference on Human Factors and Nuclear Safety
  25. Woods D. D, 1994, Proceedings of the 16th Annual Conference of the Cognitive Science Study. Hillsdale, NJ: Lawrence Erlbaum (pp. 961–971).
  26. Woods D. D. (1996). Decomposing automation: Apparent simplicity, real complexity (pp. 3–17). Theory and applications.
  27. You S., 2018, International Journal of Robotic Engineering, (IJRE), 2

Cited by

Items 501–600 of 694

Metadata

Authors
O’Neill, T., McNeese, N., Barron, A., & Schelble, B.
Affiliations
  • University of Calgary, Calgary, AB, Canada
  • Clemson University (US)
  • University of Waterloo (CA)
Venue
Human Factors: The Journal of the Human Factors and Ergonomics Society
Published
2020-10-22
Volume
64
Issue
5
Pages
904-938
ISSN
  • 0018-7208 (ISSN-L)
  • 1547-8181
OpenAlex
W3093840653
Cited by
695
References
155
Citations by year
2026: 225 · 2025: 174 · 2024: 120 · 2023: 88 · 2022: 63 · 2021: 23 · 2020: 2
FWCI
8.19
Type
journal article
Language
en
Open access
Locations
  • Human Factors The Journal of the Human Factors and Ergonomics Society (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • TigerPrints (Clemson University) (submittedVersion)
  • Sage Journals Data (cc-by)
Funders
Social Sciences and Humanities Research Council of Canada
Topics
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Social Sciences › Psychology › Social Psychology › Team Dynamics and Performance
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
human-autonomy teaming, autonomous agents, team composition, task characteristics, individual differences, team training, team communication, team performance, human agency
MeSH terms
Communication, Humans

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-14, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-14 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.