You Complete Me: Human-AI Teams and Complementary Expertise
The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
People consider recommendations from AI systems in diverse domains ranging from recognizing tumors in medical images to deciding which shoes look cute with an outfit. Implicit in the decision process is the perceived expertise of the AI system. In this paper, we investigate how people trust and rely on an AI assistant that performs with different levels of expertise relative to the person, ranging from completely overlapping expertise to perfectly complementary expertise. Through a series of controlled online lab studies where participants identified objects with the help of an AI assistant, we demonstrate that participants were able to perceive when the assistant was an expert or non-expert within the same task and calibrate their reliance on the AI to improve team performance. We also demonstrate that communicating expertise through the linguistic properties of the explanation text was effective, where embracing language increased reliance and distancing language reduced reliance on AI.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This evolution may also profoundly alter human cognition and selfperception [55], risking deskilling [202] or over-reliance [84, 104, 107], yet simultaneously offering strong augmentation to human [45].”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [202]
In the citing paper
“These parameters are commonly adapted in literature to capture the level of reliance within the human-AI interaction context.”
From Dealing with Uncertainty: Understanding the Impact of Prognostic Versus Diagnostic Tasks on Trust and Reliance in Human-AI Decision Making (2024) · cites this paper as [141]
In the citing paper
“Prior work has highlighted that appropriate trust and reliance play a critical role in achieving complementary human-AI team performance [58, 90, 139, 141].”
From Dealing with Uncertainty: Understanding the Impact of Prognostic Versus Diagnostic Tasks on Trust and Reliance in Human-AI Decision Making (2024) · cites this paper as [141]
In the citing paper
“They found that both factors affected user reliance, highlighting the importance of carefully considering the language used to express (un)certainty.”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as Zhang et al. [116]
In the citing paper
“This is consistent with prior findings that first-person messages increase recipients' involvement and engagement [24, 116] compared with general or third-person messages.”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [116]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736
Baier, A. (1986) · Ethics · doi:10.1086/292745 · Cited by 2,567
Trust, control strategies and allocation of function in human-machine systems
LEE, J., & MORAY, N. (1992) · Ergonomics · doi:10.1080/00140139208967392 · Cited by 1,459
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
MUIR, B. M., & MORAY, N. (1996) · Ergonomics · doi:10.1080/00140139608964474 · Cited by 993
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · cited 6×
“In addition to graphical representations of confidence (e.g., [5]) future studies could manipulate the bot’s appearance; the bot could be made interactive and mimic human gestures, facial expressions, or other mannerisms that indicate confidence [37, 48, 49].”
BackgroundResultInfluentialDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 1×
“Along with many other HCI studies [38, 51, 52], we follow a similar paradigm in our experiments.”
Result
Research works printed but not linked: 14
- Alexandra Chouldechova , Diana Benavides-Prado , Oleksandr Fialko , and Rhema Vaithianathan . 2018 . A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions . In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol. 81) , Sorelle A Friedler and Christo Wilson (Eds.). PMLR, 134–148. Alexandra Chouldechova, Diana Benavides-Prado, Oleksandr Fialko, and Rhema Vaithianathan. 2018. A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol. 81), Sorelle A Friedler and Christo Wilson (Eds.). PMLR, 134–148.
- Maartje M A de Graaf and Bertram F Malle . 2017 . How People Explain Action (and Autonomous Intelligent Systems Should Too) . In 2017 AAAI Fall Symposium Series. Maartje M A de Graaf and Bertram F Malle. 2017. How People Explain Action (and Autonomous Intelligent Systems Should Too). In 2017 AAAI Fall Symposium Series.
- Shi Feng and Jordan Boyd-Graber . 2019 . What can AI do for me? evaluating machine learning interpretations in cooperative play . In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19) . Association for Computing Machinery, New York, NY, USA, 229–239. Shi Feng and Jordan Boyd-Graber. 2019. What can AI do for me? evaluating machine learning interpretations in cooperative play. In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19). Association for Computing Machinery, New York, NY, USA, 229–239.
- Shi Feng and Jordan Boyd-Graber . 2019 . What can AI do for me? evaluating machine learning interpretations in cooperative play . In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19) . Association for Computing Machinery, New York, NY, USA, 229–239. Shi Feng and Jordan Boyd-Graber. 2019. What can AI do for me? evaluating machine learning interpretations in cooperative play. In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19). Association for Computing Machinery, New York, NY, USA, 229–239.
- Maaike Harbers , Karel van den Bosch, and John-Jules Ch Meyer . 2009 . A Study into Preferred Explanations of Virtual Agent Behavior. In Intelligent Virtual Agents. Springer Berlin Heidelberg , 132–145. Maaike Harbers, Karel van den Bosch, and John-Jules Ch Meyer. 2009. A Study into Preferred Explanations of Virtual Agent Behavior. In Intelligent Virtual Agents. Springer Berlin Heidelberg, 132–145.
- Rafal Kocielnik , Saleema Amershi , and Paul N Bennett . 2019 . Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-user Expectations of AI Systems . In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery , New York, NY, USA, 1–14. Rafal Kocielnik, Saleema Amershi, and Paul N Bennett. 2019. Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-user Expectations of AI Systems. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–14.
- Moritz Körber , Eva Baseler , and Klaus Bengler . 2018. Introduction matters: Manipulating trust in automation and reliance in automated driving. Appl. Ergon. 66 (Jan . 2018 ), 18–31. Moritz Körber, Eva Baseler, and Klaus Bengler. 2018. Introduction matters: Manipulating trust in automation and reliance in automated driving. Appl. Ergon. 66 (Jan. 2018), 18–31.
- Vivian Lai , Chacha Chen , Q Vera Liao , Alison Smith-Renner , and Chenhao Tan . 2021. Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies. (Dec . 2021 ). arxiv:2112.11471 [cs.AI] Vivian Lai, Chacha Chen, Q Vera Liao, Alison Smith-Renner, and Chenhao Tan. 2021. Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies. (Dec. 2021). arxiv:2112.11471 [cs.AI] · arXiv
- Nira Liberman , Yaacov Trope , Elena Stephan , and Others. 2007. Psychological distance. Social psychology: Handbook of basic principles 2 ( 2007 ), 353–383. Nira Liberman, Yaacov Trope, Elena Stephan, and Others. 2007. Psychological distance. Social psychology: Handbook of basic principles 2 (2007), 353–383.
- Bertram F Malle . 2006. How the mind explains behavior: Folk explanations, meaning, and social interaction . MIT press . Bertram F Malle. 2006. How the mind explains behavior: Folk explanations, meaning, and social interaction. MIT press.
- Tim Miller . 2019. Explanation in artificial intelligence: Insights from the social sciences. Artif. Intell. 267 (Feb . 2019 ), 1–38. Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artif. Intell. 267 (Feb. 2019), 1–38.
- Frederik Naujoks , Yannick Forster , Katharina Wiedemann , and Alexandra Neukum . 2017. A Human-Machine Interface for Cooperative Highly Automated Driving . In Advances in Human Aspects of Transportation . Springer International Publishing , 585–595. Frederik Naujoks, Yannick Forster, Katharina Wiedemann, and Alexandra Neukum. 2017. A Human-Machine Interface for Cooperative Highly Automated Driving. In Advances in Human Aspects of Transportation. Springer International Publishing, 585–595.
- 10.1145/3301275.3302308 · DOI · Paper page
- Kun Yu , Shlomo Berkovsky , Ronnie Taib , Jianlong Zhou , and Fang Chen . 2019 . Do I trust my machine teammate? an investigation from perception to decision . In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19) . Association for Computing Machinery, New York, NY, USA, 460–468. Kun Yu, Shlomo Berkovsky, Ronnie Taib, Jianlong Zhou, and Fang Chen. 2019. Do I trust my machine teammate? an investigation from perception to decision. In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19). Association for Computing Machinery, New York, NY, USA, 460–468.
Cited by
Mehrotra, S., Degachi, C., Vereschak, O., Jonker, C. M., & Tielman, M. L. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3696449 · Cited by 144
Complementarity in human-AI collaboration: concept, sources, and evidence
Hemmer, P., Schemmer, M., Kühl, N., Vössing, M., & Satzger, G. (2025) · European Journal of Information Systems · doi:10.1080/0960085x.2025.2475962 · Cited by 237
Three Challenges for AI-Assisted Decision-Making
Steyvers, M., & Kumar, A. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916231181102 · Cited by 217
Position: Towards Bidirectional Human-AI Alignment
Shen, H., Knearem, T., Ghosh, R., Alkiek, K., Krishna, K., Yachuan, L., Petridis, S., Peng, Y.-H., Qiwei, L., Si, C., et al. (2025) · neural information processing systems · doi:10.52202/085713-4186 · Cited by 210
Gomez, C., Cho, S. M., Ke, S., Huang, C.-M., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 209
Metadata
- Authors
- Zhang, Q., Lee, M. L., & Carter, S.
- Affiliations
- University of Michigan (US)
- Toyota Research Institute (US)
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-27
- Pages
- 1-28
- OpenAlex
- W4225159586
- Semantic Scholar
- 584633908e7b9a69ceab4bcddf5d2c104e016fc3
- Cited by
- 85
- References
- 40
- Citations by year
- 2026: 18 · 2025: 29 · 2024: 19 · 2023: 18 · 2022: 1
- FWCI
- 32.30
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- human-AI teaming, trust in AI, AI reliance, object identification, explanation text, language style, team performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: You Complete Me: Human-AI Teams and Complementary Expertise. https://app.cochat.ai/papers/10.1145/3491102.3517791
- The paper
- Zhang, Q., Lee, M. L., & Carter, S. (2022). You Complete Me: Human-AI Teams and Complementary Expertise. CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517791
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.