Who/What Is My Teammate? Team Composition Considerations in Human–AI Teaming
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
There are many unknowns regarding the characteristics and dynamics of human-AI teams, including a lack of understanding of how certain human-human teaming concepts may or may not apply to human-AI teams and how this composition affects team performance. This article outlines an experimental research study that investigates essential aspects of human-AI teaming such as team performance, team situation awareness, and perceived team cognition in various mixed composition teams (human-only, human-human-AI, human-AI-AI, and AI-only) through a simulated emergency response management scenario. Results indicate dichotomous outcomes regarding perceived team cognition and performance metrics, as perceived team cognition was not predictive of performance. Performance metrics like team situational awareness and team score showed that teams composed of all human participants performed at a lower level than mixed human-AI teams, with the AI-only teams attaining the highest performance. Perceived team cognition was highest in human-only teams, with mixed composition teams reporting perceived team cognition 58% below the all-human teams. These results inform future mixed teams of the potential performance gains in utilizing mixed teams' over human-only teams in certain applications, while also highlighting mixed teams' adverse effects on perceived team cognition.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
The emerging conceptualization of groups as information processors.
Hinsz, V. B., Tindale, R. S., & Vollrath, D. A. (1997) · Psychological Bulletin · doi:10.1037/0033-2909.121.1.43 · Cited by 1,428
Human–Autonomy Teaming: A Review and Analysis of the Empirical Literature
O’Neill, T., McNeese, N., Barron, A., & Schelble, B. (2022) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720820960865 · Cited by 695
Self-citationTen challenges for making automation a ‘team player’ in joint human-agent activity
Klein, G., Woods, D. D., Bradshaw, J. M., Hoffman, R. R., & Feltovich, P. J. (2004) · IEEE Intelligent Systems · doi:10.1109/mis.2004.74 · Cited by 590
Cooke, N. J., Gorman, J. C., Myers, C. W., & Duran, J. L. (2013) · Cognitive Science · doi:10.1111/cogs.12009 · Cited by 522
Teaming With a Synthetic Teammate: Insights into Human-Autonomy Teaming
McNeese, N. J., Demir, M., Cooke, N. J., & Myers, C. (2018) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720817743223 · Cited by 380
Self-citationLearning with Opponent-Learning Awareness
Foerster, J., Chen, R. Y., Al-Shedivat, M., Whiteson, S., Abbeel, P., & Mordatch, I. (2018) · International Joint Conference on Autonomous Agents and Multiagent Systems · doi:10.65109/hgwa8807 · Cited by 215
On the Utility of Learning about Humans for Human-AI Coordination
Carroll, M., Shah, R., Ho, M. K., Griffiths, T. L., Seshia, S. A., Abbeel, P., & Dragan, A. (2019) · arXiv · doi:10.48550/arxiv.1910.05789 · Cited by 168
Creating high performance teamwork in organizations
O'Neill, T. A., & Salas, E. (2018) · Human Resource Management Review · doi:10.1016/j.hrmr.2017.09.001 · Cited by 108
Common Metrics to Benchmark Human-Machine Teams (HMT): A Review
Damacharla, P., Javaid, A. Y., Gallimore, J. J., & Devabhaktuni, V. K. (2020) · arXiv · doi:10.48550/arxiv.2008.04855 · Cited by 97
Cited by
Lee, H.-P. (., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713778 · Cited by 490
“(select all that apply) (i) Informally, at school or university (e.g., learnt from peers, or picked it up over time) (ii) Through formal training at school or university (e.g., took a course) (iii) Informally, at my workplace (e.g., learnt from colleagues, or picked it up over time)”
Enhancing AI-Assisted Group Decision Making through LLM-Powered Devil's Advocate
Chiang, C.-W., Lu, Z., Li, Z., & Yin, M. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645199 · Cited by 142
Adapt and overcome: Perceptions of adaptive autonomous agents for human-AI teaming
Hauptman, A. I., Schelble, B. G., McNeese, N. J., & Madathil, K. C. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107451 · Cited by 110
Self-citationAI Knowledge: Improving AI Delegation through Human Enablement
Pinski, M., Adam, M., & Benlian, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580794 · Cited by 62
Artificial Intelligence and Groups: Effects of Attitudes and Discretion on Collaboration
Bezrukova, K., Griffith, T. L., Spell, C., Rice, V., & Yang, H. E. (2023) · Group & Organization Management · doi:10.1177/10596011231160574 · Cited by 51
Metadata
- Authors
- McNeese, N. J., Schelble, B. G., Canonico, L. B., & Demir, M.
- Venue
- IEEE Transactions on Human-Machine Systems
- Published
- 2021-08
- Volume
- 51
- Issue
- 4
- Pages
- 288-299
- ISSN
- 2168-2291 (ISSN-L)
- 2168-2305
- OpenAlex
- W3165597437
- Cited by
- 101
- References
- 11
- FWCI
- 13.22
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.