The Trust Recovery Journey. The Effect of Timing of Errors on the Willingness to Follow AI Advice.
Open accessThe DOI matches its Crossref record; 53 of the 54 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Complementing human decision-making with AI advice offers substantial advantages. However, humans do not always trust AI advice appropriately and are overly sensitive to incidental AI errors, even in cases with overall good performance. Today’s research still needs to uncover the underlying aspects of trust decline and recovery over time in repeated human-AI interactions. Our work investigates the consequences of incidental AI error on (self-reported) trust and participants’ reliance on AI advice. Results from our experiment, where 208 participants evaluated 14 legal cases before and after receiving algorithmic advice, showed that trust significantly decreased after early and late errors but was rapidly restored in both scenarios. Reliance significantly dropped only for early errors but not for late errors. In both scenarios, reliance was able to be restored. Results suggest that late (compared to early) errors are less drastic in trust loss and allow quicker recovery. These findings align with an interpretation in which humans can build up trust over time if a system is performing well, making them more tolerant of incidental AI errors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 1 Schematic reliance (A, left) and trust (B, right) trajectories based on our three experimental conditions, Baseline, Early Error, and Late Error. The vertical thick lines represent the early (trial 4) and late (trial 11) errors. The trajectories represent assumed progressions of reliance and trust based on previous research, for example, trust increases over time [32, 68] where reliance rather stabilizes ([32], including expected reliance and trust drops, and expected recovery. We note that we omit any realistic fluctuations by using straight lines.](https://figures.cochat.ai/paper-figures/dbdd7ae94eaf4eefa4bb3e0598498099/the-trust-recovery-journey-the-effect-of-timing-of-errors-on-the-willingness-to-figure-1.png)




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · cited 1×
“…the resulting manifestation of what and how a person thinks about the system’s competencies (labelled “trust”); the second is based on the behavior elicited by the AI advice and is defined by the extent to which the AI advice influences a person’s decision-making (labelled “reliance”) [26, 35, 44].”
BackgroundTrust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 2×
“…the resulting manifestation of what and how a person thinks about the system’s competencies (labelled “trust”); the second is based on the behavior elicited by the AI advice and is defined by the extent to which the AI advice influences a person’s decision-making (labelled “reliance”) [26, 35, 44].”
BackgroundResultAlgorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Prospect Theory: An Analysis of Decision Under Risk
Kahneman, D., & Tversky, A. (2013) · World Scientific Handbook in Financial Economics Series · doi:10.1142/9789814417358_0006 · Cited by 3,091
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921 · cited 2×
“Glikson and Woolley’s [23] comprehensive review mentions several cavities and the lack of clear research directions.”
BackgroundTrust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
Gray, H. M., Gray, K., & Wegner, D. M. (2007) · Science · doi:10.1126/science.1134475 · Cited by 2,342 · cited 1×
“Gray et al. [25] propose that AI systems can be perceived as having abilities, such as the ability to think, plan, and act autonomously.”
Background(2014) · Choice Reviews Online · doi:10.5860/choice.51-5559 · Cited by 2,342 · cited 1×
“Among other reasons, this may happen because people have (limited) capabilities for a specific (complex) task [7, 33, 46], act based on (unconsciously) biased perceptions, traits or beliefs [4, 38], lack information or access to understanding the algorithm and its recommendation [20, 47], or are…”
BackgroundNass, C., Steuer, J., & Tauber, E. R. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191703 · Cited by 2,148
Research works printed but not linked: 14
- Phoebe E Bailey, Tarren Leon, Natalie C Ebner, Ahmed A Moustafa, and Gabrielle Weidemann. 2022. A meta-analysis of the weight of advice in decision-making. Current Psychology (2022), 1–26.
- Alvaro Chacon, Edgar E Kausel, and Tomas Reyes. 2022. A longitudinal approach for understanding algorithm use. Journal of Behavioral Decision Making (2022).
- Franz Faul, Edgar Erdfelder, Axel Buchner, and Albert-Georg Lang. 2009. Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior research methods 41, 4 (2009), 1149–1160.
- Heather M Gray, Kurt Gray, and Daniel M Wegner. 2007. Dimensions of mind perception. science 315, 5812 (2007), 619–619.
- Stephan Lewandowsky, Michael Mundy, and Gerard Tan. 2000. The dynamics of trust: comparing humans to automation.Journal of Experimental Psychology: Applied 6, 2 (2000), 104.
- Maria Madsen and Shirley Gregor. 2000. Measuring human-computer trust. In 11th australasian conference on information systems, Vol. 53. Citeseer, 6–8.
- JB Manchon, Mercedes Bueno, and Jordan Navarro. 2021. Calibration of Trust in Automated Driving: A Matter of Initial Level of Trust and Automated Driving Style?Human Factors (2021), 00187208211052804.
- Raja Parasuraman and Victor Riley. 1997. Humans and automation: Use, misuse, disuse, abuse. Human factors 39, 2 (1997), 230–253.
- Tobias M Peters and Roel W Visser. 2023. The Importance of Distrust in AI. arXiv preprint arXiv:2307.13601 (2023). · arXiv
- Victor Andrew Riley. 1994. Human use of automation. University of Minnesota.
- Daniel B Shank and Alexander Gott. 2019. People’s self-reported encounters of Perceiving Mind in Artificial Intelligence. Data in brief 25 (2019), 104220.
- Keng Siau and Weiyu Wang. 2018. Building trust in artificial intelligence, machine learning, and robotics. Cutter business technology journal 31, 2 (2018), 47–53.
- Janet A Sniezek and Timothy Buckley. 1995. Cueing and cognitive conflict in judge-advisor decision making. Organizational behavior and human decision processes 62, 2 (1995), 159–174.
- X Jessie Yang, Christopher Schemanske, and Christine Searle. 2021. Toward quantifying trust dynamics: How people adjust their trust after moment-to-moment interaction with automation. arXiv preprint arXiv:2107.07374 (2021). · arXiv
Cited by
de Jong, S., Paananen, V., Tag, B., & van Berkel, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710946 · Cited by 32
Personalising AI Assistance Based on Overreliance Rate in AI-Assisted Decision Making
Swaroop, S., Buçinca, Z., Gajos, K. Z., & Doshi-Velez, F. (2025) · Proceedings of the 30th International Conference on Intelligent User Interfaces · doi:10.1145/3708359.3712128 · Cited by 20
Kahr, P. K., Rooks, G., Snijders, C. C. P., & Willemsen, M. C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713099 · Cited by 26
Self-citationTo Solicit or Not to Solicit? Impact of AI Assistance Delivery Mechanisms on Decision-Making
Nair, P., Gressel, G., Anand, M. N., & Achuthan, K. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2536617 · Cited by 17
“This is important to consider, especially in light of Kahr et al. who highlighted the role of trust recovery mechanisms in AI-assisted decision-making (Kahr et al., 2024).”
AI systems should be trustworthy, not trusted
Zanotti, G. (2025) · AI & Society · doi:10.1007/s00146-025-02728-6 · Cited by 13
Metadata
- Authors
- Kahr, P. K., Rooks, G., Snijders, C., & Willemsen, M. C.
- Affiliations
- Eindhoven University of Technology (NL)
- Venue
- Proceedings of the 29th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2024-03-18
- Pages
- 609-622
- OpenAlex
- W4393970495
- Cited by
- 21
- References
- 44
- Citations by year
- 2026: 9 · 2025: 12
- FWCI
- 14.08
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion, cc-by)
- TU/e Research Portal (publishedVersion, cc-by)
- OSF Preprints (OSF Preprints) (submittedVersion, cc-by)
- Funders
- Technische Universiteit Eindhoven
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- trust in AI, trust repair, human-AI interaction, algorithmic advice, trust loss
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.