Understanding the Evolvement of Trust Over Time within Human-AI Teams
Open accessThe DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Abstract
The success of human-AI teams (HATs) requires humans to work with AI teammates in trustful ways over a certain time period. However, how trust evolves and changes dynamically in response to human-AI team interactions is generally understudied. This work explores the evolvement of trust in HATs over time by analyzing 45 participants' experiences of trust or distrust in an AI teammate prior to, during, and after collaborating with AI in a three-member HAT. Our findings highlight that humans' expectations of AI's ability, integrity, benevolence, and adaptability influence their initial trust in AI before collaboration. However, this initial trust can be maintained or revised through the development of situational trust during collaboration in response to the AI teammate's communication behaviors. Further, the trust developed through collaboration can impact individuals' subsequent expectations of AI's ability and their collaborations with AI. Our findings also reveal the similarities and differences in the temporal dimensions of trust for AI and human teammates. We contribute to CSCW community by offering one of the first empirical investigations into the dynamic and temporal dimension of trust evolvement in HATs. Our work yields insights into the pathways to expanding the methodological toolkit for investigating the development of trust in HATs, formulating theories of trust for the HAT context. These insights further inform the effective design of AI teammates and provide guidance on the timing, content, and methods for calibrating trust in future human-AI collaboration contexts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Artificial intelligence in service
Huang, M.-H., & Rust, R. T. (2018) · Journal of Service Research · doi:10.1177/1094670517752459 · Cited by 3,978 · cited 2×
“HATs combine human emotional and intuitive qualities with the precision and rapid processing abilities of AI technologies [46].”
Rempel, J. K., Holmes, J. G., & Zanna, M. P. (1985) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.49.1.95 · Cited by 3,146
Trust And Distrust: New Relationships and Realities
Lewicki, R. J., McAllister, D. J., & Bies, R. J. (1998) · The Academy of Management Review · doi:10.5465/amr.1998.926620 · Cited by 3,091
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921 · cited 1×
“Additionally, immediacy behaviors, such as proactive and responsive actions, help build cognitive trust by creating a sense of closeness and setting high expectations for AI performance [37].”
Communication and Trust in Global Virtual Teams
Jarvenpaa, S. L., & Leidner, D. E. (1999) · Organization Science · doi:10.1287/orsc.10.6.791 · Cited by 2,903
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
An integrative model of organizational trust: past, present, and future
Schoorman, F. D., Mayer, R. C., & Davis, J. H. (2007) · Academy of Management Review · doi:10.5465/amr.2007.24348410 · Cited by 2,794
Baier, A. (1986) · Ethics · doi:10.1086/292745 · Cited by 2,567
THE DYNAMIC NATURE OF CONFLICT: A LONGITUDINAL STUDY OF INTRAGROUP CONFLICT AND GROUP PERFORMANCE.
Jehn, K. A., & Mannix, E. A. (2001) · Academy of Management Journal · doi:10.2307/3069453 · Cited by 2,487
Swift Trust and Temporary Groups
Meyerson, D., Weick, K. E., & Kramer, R. M. (1996) · Trust in Organizations: Frontiers of Theory and Research · doi:10.4135/9781452243610.n9 · Cited by 1,982
Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Shneiderman, B. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1741118 · Cited by 1,743 · cited 1×
“The nature of tasks assigned to AI further contributes to trust development [95]; AI that is effectively used for tasks aligning with its capabilities is more likely to be trusted [39].”
Interpersonal trust, trustworthiness, and gullibility.
Rotter, J. B. (1980) · American Psychologist · doi:10.1037/0003-066x.35.1.1 · Cited by 1,694
Generative AI and ChatGPT: applications, challenges, and AI-human collaboration
Fui-Hoon Nah, F., Zheng, R., Cai, J., Siau, K., & Chen, L. (2023) · Journal of Information Technology Case and Application Research · doi:10.1080/15228053.2023.2233814 · Cited by 1,363
McDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206
Trust in Management and Performance: Who Minds the Shop While the Employees Watch the Boss?
Mayer, R. C., & Gavin, M. B. (2005) · Academy of Management Journal · doi:10.5465/amj.2005.18803928 · Cited by 1,200
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
Trust and team performance: A meta-analysis of main effects, moderators, and covariates.
De Jong, B. A., Dirks, K. T., & Gillespie, N. (2016) · Journal of Applied Psychology · doi:10.1037/apl0000110 · Cited by 616
Generating scholarly content with ChatGPT: Ethical challenges for medical publishing
Liebrenz, M., Schleifer, R., Buadze, A., Bhugra, D., & Smith, A. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00019-5 · Cited by 594
Dynamic nature of trust in virtual teams
Kanawattanachai, P., & Yoo, Y. (2002) · The Journal of Strategic Information Systems · doi:10.1016/s0963-8687(02)00019-7 · Cited by 594 · cited 2×
“For instance, initial trust can be easily destroyed if team members fail to live up to others’ expectations, whereas it can also be maintained or even increased if the team has better performance throughout the collaboration [52].”
Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
Jacovi, A., Marasović, A., Miller, T., & Goldberg, Y. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445923 · Cited by 545
Madaio, M. A., Stark, L., Wortman Vaughan, J., & Wallach, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376445 · Cited by 436 · cited 1×
“The integrity principles could be presented as a fairness checklist as suggested by [47, 66].”
Breuer, C., Hüffmeier, J., & Hertel, G. (2016) · Journal of Applied Psychology · doi:10.1037/apl0000113 · Cited by 375 · cited 1×
“On the contrary, high trust contributes to team outcomes such as team satisfaction, team information processing and team performance [8].”
Adaptive trust calibration for human-AI collaboration
Okamura, K., & Yamada, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0229132 · Cited by 239 · cited 1×
“…as monitoring behaviors, and juxtapose such cues against the actual accuracy of AI to determine if the human is over-trusting, under-trusting or applying appropriate trust toward the AI teammate, thereby dynamically display calibration cues (along the lines of [16, 81]) appropriate at the moment.”
Zhang (2021) · doi:10.1145/3432945 · Cited by 225 · cited 2×
“This echoes recent explorations that AI is expected to be an ideal version of a human [113] and more trusted than humans [110].”
Planning with Trust for Human-Robot Collaboration
Chen, M., Nikolaidis, S., Soh, H., Hsu, D., & Srinivasa, S. (2018) · Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3171221.3171264 · Cited by 221
Kunze, A., Summerskill, S. J., Marshall, R., & Filtness, A. J. (2019) · Ergonomics · doi:10.1080/00140139.2018.1547842 · Cited by 165
Wischnewski, M., Krämer, N., & Müller, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581197 · Cited by 146
Zhang, R., Duan, W., Flathmann, C., McNeese, N., Freeman, G., & Williams, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610072 · Cited by 134
Self-citationHuman Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks
Lu, Z., & Yin, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445562 · Cited by 131
Overtrust in External Cues of Automated Vehicles: An Experimental Investigation
Holländer, K., Wintersberger, P., & Butz, A. (2019) · Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications · doi:10.1145/3342197.3344528 · Cited by 109
Rechkemmer, A., & Yin, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501967 · Cited by 97
Trust in Human-AI Interaction: Scoping Out Models, Measures, and Methods
Ueno, T., Sawa, Y., Kim, Y., Urakami, J., Oura, H., & Seaborn, K. (2022) · Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491101.3519772 · Cited by 91
Trust Evolvement in Hybrid Team Collaboration: A Longitudinal Case Study
Cheng, X., Yin, G., Azadegan, A., & Kolfschoten, G. (2016) · Group Decision and Negotiation · doi:10.1007/s10726-015-9442-x · Cited by 87 · cited 1×
“The pattern of trust development has also been found to vary based on team composition, levels of familiarization, closeness in coordination and different cultural values [17].”
Do You Still Trust Me? Human-Robot Trust Repair Strategies
Esterwood, C., & Robert, L. P. (2021) · 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN) · doi:10.1109/ro-man50785.2021.9515365 · Cited by 75 · cited 1×
“…of AI trust calibration strategies have focused on AI’s reliability [106] or do not differentiate these sub-constructs [106], despite a few exceptions (e.g., [32]) Our work has demonstrated that besides AI’s reliability, its integrity, benevolence, and adaptability aspects also warrant calibration.”
Do I trust my machine teammate?: an investigation from perception to decision
Yu, K., Berkovsky, S., Taib, R., Zhou, J., & Chen, F. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302277 · Cited by 71
Trust Repair Strategies with Self-Driving Vehicles: An Exploratory Study
Kohn, S. C., Quinn, D., Pak, R., de Visser, E. J., & Shaw, T. H. (2018) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931218621254 · Cited by 64 · cited 2×
“This is similar to the well-studied concept of trust repair [53, 55, 93].”
Bobko, P., Hirshfield, L., Eloy, L., Spencer, C., Doherty, E., Driscoll, J., & Obolsky, H. (2023) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2022.2086644 · Cited by 33 · cited 2×
“While the prevailing body of research primarily focuses on calibrating trust in AI solely based on its reliability (e.g., [7, 85, 106]), our study highlights the necessity to also calibrate trust concerning the AI’s integrity, benevolence, and adaptability, particularly in team contexts.”
Cited by
Exploring automation bias in human–AI collaboration: a review and implications for explainable AI
Romeo, G., & Conti, D. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02422-7 · Cited by 209
CreAItive Collaboration? Users' Misjudgment of AI-Creativity Affects Their Collaborative Performance
Bangerl, M. M., Disch, L., David, T., & Pammer-Schindler, V. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713886 · Cited by 13
“Users’ familiarity with AI, which we understand to encompass general, practical, and community-based knowledge [16] is linked to perceived usefulness and trust in AI and leads to a more accurate assessment of AI strengths and weaknesses [6, 27, 98].”
From testbeds to high-stakes work: a review of human-AI teaming domains and teaming factors
Kargarnovin, S., Hernandez, C. I., Reiners, D., Cruz-Neira, C., Bochenek, G., & Karwowski, W. (2026) · Frontiers in Robotics and AI · doi:10.3389/frobt.2026.1733942 · Cited by 2
“More broadly, people arrive with expectations about where AI will succeed or fail, and trust or distrust can carry over to other AIs more readily than it does between humans, making trust diffusion a system-level consideration in multi-AI environments (Duan et al., 2024).”
Panchanadikar, R., & Freeman, G. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790869 · Cited by 3
Self-citation
Metadata
- Authors
- Duan, W., Zhou, S., Scalia, M. J., Yin, X., Weng, N., Zhang, R., Freeman, G., McNeese, N., Gorman, J., & Tolston, M.
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2024-11-07
- Volume
- 8
- Issue
- CSCW2
- Pages
- 1-31
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W4404172944
- Cited by
- 45
- References
- 37
- FWCI
- 21.34
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.