CoChat

You Complete Me: Human-AI Teams and Complementary Expertise

Zhang, Qiaoning, Lee, Matthew L, Carter, Scott · 2022 · CHI Conference on Human Factors in Computing Systems

The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.

Abstract

People consider recommendations from AI systems in diverse domains ranging from recognizing tumors in medical images to deciding which shoes look cute with an outfit. Implicit in the decision process is the perceived expertise of the AI system. In this paper, we investigate how people trust and rely on an AI assistant that performs with different levels of expertise relative to the person, ranging from completely overlapping expertise to perfectly complementary expertise. Through a series of controlled online lab studies where participants identified objects with the help of an AI assistant, we demonstrate that participants were able to perceive when the assistant was an expert or non-expert within the same task and calibrate their reliance on the AI to improve team performance. We also demonstrate that communicating expertise through the linguistic properties of the explanation text was effective, where embracing language increased reliance and distancing language reduced reliance on AI.

Overview

Integrity signals

Registry

Crossref · 10.1145/3491102.3517791

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 41 references

Research works printed but not linked: 14

  1. Alexandra Chouldechova , Diana Benavides-Prado , Oleksandr Fialko , and Rhema Vaithianathan . 2018 . A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions . In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol. 81) , Sorelle A Friedler and Christo Wilson (Eds.). PMLR, 134–148. Alexandra Chouldechova, Diana Benavides-Prado, Oleksandr Fialko, and Rhema Vaithianathan. 2018. A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol. 81), Sorelle A Friedler and Christo Wilson (Eds.). PMLR, 134–148.
  2. Maartje M A de Graaf and Bertram F Malle . 2017 . How People Explain Action (and Autonomous Intelligent Systems Should Too) . In 2017 AAAI Fall Symposium Series. Maartje M A de Graaf and Bertram F Malle. 2017. How People Explain Action (and Autonomous Intelligent Systems Should Too). In 2017 AAAI Fall Symposium Series.
  3. Shi Feng and Jordan Boyd-Graber . 2019 . What can AI do for me? evaluating machine learning interpretations in cooperative play . In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19) . Association for Computing Machinery, New York, NY, USA, 229–239. Shi Feng and Jordan Boyd-Graber. 2019. What can AI do for me? evaluating machine learning interpretations in cooperative play. In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19). Association for Computing Machinery, New York, NY, USA, 229–239.
  4. Shi Feng and Jordan Boyd-Graber . 2019 . What can AI do for me? evaluating machine learning interpretations in cooperative play . In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19) . Association for Computing Machinery, New York, NY, USA, 229–239. Shi Feng and Jordan Boyd-Graber. 2019. What can AI do for me? evaluating machine learning interpretations in cooperative play. In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19). Association for Computing Machinery, New York, NY, USA, 229–239.
  5. Maaike Harbers , Karel van den Bosch, and John-Jules Ch Meyer . 2009 . A Study into Preferred Explanations of Virtual Agent Behavior. In Intelligent Virtual Agents. Springer Berlin Heidelberg , 132–145. Maaike Harbers, Karel van den Bosch, and John-Jules Ch Meyer. 2009. A Study into Preferred Explanations of Virtual Agent Behavior. In Intelligent Virtual Agents. Springer Berlin Heidelberg, 132–145.
  6. Rafal Kocielnik , Saleema Amershi , and Paul N Bennett . 2019 . Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-user Expectations of AI Systems . In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery , New York, NY, USA, 1–14. Rafal Kocielnik, Saleema Amershi, and Paul N Bennett. 2019. Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-user Expectations of AI Systems. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–14.
  7. Moritz Körber , Eva Baseler , and Klaus Bengler . 2018. Introduction matters: Manipulating trust in automation and reliance in automated driving. Appl. Ergon. 66 (Jan . 2018 ), 18–31. Moritz Körber, Eva Baseler, and Klaus Bengler. 2018. Introduction matters: Manipulating trust in automation and reliance in automated driving. Appl. Ergon. 66 (Jan. 2018), 18–31.
  8. Vivian Lai , Chacha Chen , Q Vera Liao , Alison Smith-Renner , and Chenhao Tan . 2021. Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies. (Dec . 2021 ). arxiv:2112.11471 [cs.AI] Vivian Lai, Chacha Chen, Q Vera Liao, Alison Smith-Renner, and Chenhao Tan. 2021. Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies. (Dec. 2021). arxiv:2112.11471 [cs.AI] · arXiv
  9. Nira Liberman , Yaacov Trope , Elena Stephan , and Others. 2007. Psychological distance. Social psychology: Handbook of basic principles 2 ( 2007 ), 353–383. Nira Liberman, Yaacov Trope, Elena Stephan, and Others. 2007. Psychological distance. Social psychology: Handbook of basic principles 2 (2007), 353–383.
  10. Bertram F Malle . 2006. How the mind explains behavior: Folk explanations, meaning, and social interaction . MIT press . Bertram F Malle. 2006. How the mind explains behavior: Folk explanations, meaning, and social interaction. MIT press.
  11. Tim Miller . 2019. Explanation in artificial intelligence: Insights from the social sciences. Artif. Intell. 267 (Feb . 2019 ), 1–38. Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artif. Intell. 267 (Feb. 2019), 1–38.
  12. Frederik Naujoks , Yannick Forster , Katharina Wiedemann , and Alexandra Neukum . 2017. A Human-Machine Interface for Cooperative Highly Automated Driving . In Advances in Human Aspects of Transportation . Springer International Publishing , 585–595. Frederik Naujoks, Yannick Forster, Katharina Wiedemann, and Alexandra Neukum. 2017. A Human-Machine Interface for Cooperative Highly Automated Driving. In Advances in Human Aspects of Transportation. Springer International Publishing, 585–595.
  13. 10.1145/3301275.3302308 · DOI · Paper page
  14. Kun Yu , Shlomo Berkovsky , Ronnie Taib , Jianlong Zhou , and Fang Chen . 2019 . Do I trust my machine teammate? an investigation from perception to decision . In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19) . Association for Computing Machinery, New York, NY, USA, 460–468. Kun Yu, Shlomo Berkovsky, Ronnie Taib, Jianlong Zhou, and Fang Chen. 2019. Do I trust my machine teammate? an investigation from perception to decision. In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19). Association for Computing Machinery, New York, NY, USA, 460–468.

Cited by

See all 153

Metadata

Authors
Zhang, Q., Lee, M. L., & Carter, S.
Affiliations
  • University of Michigan (US)
  • Toyota Research Institute (US)
Venue
CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2022-04-27
Pages
1-28
OpenAlex
W4225159586
Cited by
85
References
40
Citations by year
2026: 18 · 2025: 29 · 2024: 19 · 2023: 18 · 2022: 1
FWCI
32.30
Type
conference paper
Language
en
Open access
closed
Locations
CHI Conference on Human Factors in Computing Systems (publishedVersion)
Topics
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
human-AI teaming, trust in AI, AI reliance, object identification, explanation text, language style, team performance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: You Complete Me: Human-AI Teams and Complementary Expertise. https://app.cochat.ai/papers/10.1145/3491102.3517791
The paper
Zhang, Q., Lee, M. L., & Carter, S. (2022). You Complete Me: Human-AI Teams and Complementary Expertise. CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517791

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.