User Trust Dynamics: An Investigation Driven by Differences in System Performance
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Trust is a key factor affecting the way people rely on automated systems. On the other hand, system performance has comprehensive implications on a user's trust variations. This paper examines systems of varied levels of accuracy, in order to reveal the relationship between system performance, a user's trust and reliance on the system. In particular, it is identified that system failures have a stronger effect on trust than system successes. We also describe how patterns of trust change according to a number of consecutive system failures or successes. Importantly, we show that increasing user familiarity with the system decreases the rate of trust change, which provides new insights on the development of user trust. Finally, our analysis established a correlation between a user's reliance on a system and their trust level. Combining all these findings can have important implications in general system design and implementation, by predicting how trust builds and when it stabilizes, as well as allowing for indirectly reading a user's trust in real time based on system reliance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Artificial Intelligence (AI) systems have been shown to outperform humans in various decisionmaking tasks across many domains.”
From Cognitive Forcing for Better Decision-Making: Reducing Overreliance on AI Systems Through Partial Explanations (2025) · cites this paper as [79]
In the citing paper
“For example, there is a particular focus on ensuring competence of AI (i.e., that it performs accurately and eciently) [20,41], which could potentially lead to increase in end-users' condence in AI [61] (though without necessarily always implying trust [39]).”
From Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust (2023) · cites this paper as [61]
In the citing paper
“Note that human-AI trust goes beyond the end-user's condence in the competence of the AI [33,59,60].”
From Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust (2023) · cites this paper as [60]
Citations
Cites
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
Moorman, C., Zaltman, G., & Deshpande, R. (1992) · Journal of Marketing Research · doi:10.1177/002224379202900303 · Cited by 1,901 · cited 1×
“…trust definitions show that trust is a hypothesised variable that has been demonstrated to be a key mitigating factor in system use or disuse (sometimes called reliance) [13], can be a means to characterize the interpersonal dependency [5] or the relationship within or between organizations [18].”
BackgroundTrust, control strategies and allocation of function in human-machine systems
LEE, J., & MORAY, N. (1992) · Ergonomics · doi:10.1080/00140139208967392 · Cited by 1,459
On-line trust: concepts, evolving themes, a model
Corritore, C. L., Kracher, B., & Wiedenbeck, S. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00041-7 · Cited by 1,200
Proceedings of the Human Factors and Ergonomics Society 54th Annual Meeting
Kluge, A., Badura, B., Urbas, L., & Burkolter, D. (2010) · Cited by 1,127
Trust, self-confidence, and operators' adaptation to automation
Lee, J. D., & Moray, N. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1007 · Cited by 932
MUIR, B. M. (1994) · Ergonomics · doi:10.1080/00140139408964957 · Cited by 873
Trust between humans and machines, and the design of decision aids
Muir, B. M. (1987) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(87)80013-5 · Cited by 762
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2008) · Journal of Cognitive Engineering and Decision Making · doi:10.1518/155534308x284417 · Cited by 695
Dimensions and Levels of Trust: Implications for Commitment to a Relationship
Ganesan, S., & Hess, R. (1997) · Marketing Letters · doi:10.1023/a:1007955514781 · Cited by 545
Adaptive automation, trust, and self-confidence in fault management of time-critical tasks.
Moray, N., Inagaki, T., & Itoh, M. (2000) · Journal of Experimental Psychology: Applied · doi:10.1037//1076-898x.6.1.44 · Cited by 345
Trust building with explanation interfaces
Pu, P., & Chen, L. (2006) · Proceedings of the 11th international conference on Intelligent user interfaces · doi:10.1145/1111449.1111475 · Cited by 300 · cited 1×
“…an intelligent system shares the same goal as the user, and the level of system transparency, i.e. the explanatory information provided by the system impacts the trust of the user, while the importance of explanations in trust building has also been confirmed in the work of Pu and Chen [24].”
Backgroundde Vries, P., Midden, C., & Bouwhuis, D. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00039-9 · Cited by 264
Verberne, F. M. F., Ham, J., & Midden, C. J. H. (2012) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720812443825 · Cited by 262
adaptive aiding for human/computer control
Rouse, W. B. (1988) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/001872088803000405 · Cited by 252
Empathy and online interpersonal trust: A fragile relationship
Feng, J., Lazar, J., & Preece, J. (2004) · Behaviour & Information Technology · doi:10.1080/01449290310001659240 · Cited by 216
McGuirl, J. M., & Sarter, N. B. (2006) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872006779166334 · Cited by 212
Toward establishing trust in adaptive agents
Glass, A., McGuinness, D. L., & Wolverton, M. (2008) · Proceedings of the 13th international conference on Intelligent user interfaces · doi:10.1145/1378773.1378804 · Cited by 211
Towards content trust of web resources
Gil, Y., & Artz, D. (2007) · Journal of Web Semantics · doi:10.1016/j.websem.2007.09.005 · Cited by 138
A crisis of trust: for science, scientists or for institutions?
Millstone, E. P., & van Zwanenberg, P. (2000) · Nature Medicine · doi:10.1038/82102 · Cited by 120
Trust Modeling and Management: From Social Trust to Digital Trust
Yan, Z., & Holtmanns, S. (2008) · Computer Security, Privacy and Politics · doi:10.4018/978-1-59904-804-8.ch013 · Cited by 96
Trust Modeling and Management: From Social Trust to Digital Trust
Yan, Z., & Holtmanns, S. (2013) · Examining the Concepts, Issues, and Implications of Internet Trolling · doi:10.4018/978-1-4666-2803-8.ch018 · Cited by 91
Merritt, S. M., Lee, D., Unnerstall, J. L., & Huber, K. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814561675 · Cited by 83 · cited 1×
“People adjust their trust in the system based on their interaction, during which system performance is posited as being among the most critical factors affecting a user’s trust [13,15,16,19].”
BackgroundSauer, J., Chavaillaz, A., & Wastell, D. (2016) · Ergonomics · doi:10.1080/00140139.2015.1094577 · Cited by 73
An empirical study on the trustworthiness of life-like interface agents
van Mulken, S., André, E., & Müller, J. M. (1999) · Cited by 50
Type of Automation Failure: The Effects on Trust and Reliance in Automation
Johnson, J. D., Sanchez, J., Fisk, A. D., & Rogers, W. A. (2004) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120404801807 · Cited by 41 · cited 1×
“Johnson [11] examined how different types of automation errors affects user trust and reliance as well as the perceived reliability of automated”
BackgroundIncorporating behavioral trust theory into system development for ubiquitous applications
Hoffmann, H., & Söllner, M. (2012) · Personal and Ubiquitous Computing · doi:10.1007/s00779-012-0631-1 · Cited by 34
Trust-based decision-making for smart and adaptive environments
Hammer, S., Wißner, M., & André, E. (2015) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-015-9160-8 · Cited by 32
Trust and Reliance Based on System Accuracy
Yu, K., Berkovsky, S., Conway, D., Taib, R., Zhou, J., & Chen, F. (2016) · Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization · doi:10.1145/2930238.2930290 · Cited by 32
Self-citationIndividual Differences, Trust, and Vehicle Autonomy: A pilot study
Schaefer, K. E., & Scribner, D. R. (2015) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931215591242 · Cited by 23
Type of automation failure: The effects on trust and reliance in automation
Johnson, J. D., Sánchez, J., Fisk, A. D., & Rogers, W. A. (2004) · PsycEXTRA Dataset · doi:10.1037/e577212012-007 · Cited by 18
How to use behavioral research insights on trust for HCI system design
Söllner, M., Hoffmann, A., Hoffmann, H., & Leimeister, J. M. (2012) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2212776.2223696 · Cited by 17
How to Use Behavioral Research Insights on Trust for HCI System DesignPreprint
Söllner, M., Hoffmann, A., Hoffmann, H., et al. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2484637 · Cited by 3
Research works printed but not linked: 1
- John Gall . 2002. The Systems Bible: The Beginner's Guide to Systems Large and Small . General Systemantics Press . John Gall. 2002. The Systems Bible: The Beginner's Guide to Systems Large and Small. General Systemantics Press.
Cited by
Evaluating the quality of machine learning explanations: a survey on methods and metrics
Zhou, J., Gandomi, A. H., Chen, F., & Holzinger, A. (2021) · Electronics · doi:10.3390/electronics10050593 · Cited by 606
“Since human-centered evaluations employ experiments with end-users or lay humans to assess the quality of explanations, subjective measures, such as trust and confidence, have been embraced as the focal point for the evaluation of explainable systems [33,64,65,69,92,93].”
BackgroundMethodSelf-citationUnderstanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527
How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies
Vereschak, O., Bailly, G., & Caramiaux, B. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476068 · Cited by 229
A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
De-Arteaga, M., Fogliato, R., & Chouldechova, A. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376638 · Cited by 221
Akalin, N., Kristoffersson, A., & Loutfi, A. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102744 · Cited by 159
Metadata
- Authors
- Yu, K., Berkovsky, S., Taib, R., Conway, D., Zhou, J., & Chen, F.
- Affiliations
- Commonwealth Scientific and Industrial Research Organisation (AU)
- Venue
- Proceedings of the 22nd International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2017-03-07
- Pages
- 307-317
- OpenAlex
- W2593883017
- Cited by
- 90
- References
- 35
- Citations by year
- 2026: 9 · 2025: 15 · 2024: 13 · 2023: 12 · 2022: 8 · 2021: 11 · 2020: 5 · 2019: 10 · 2018: 5 · 2017: 2
- FWCI
- 9.74
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- Funders
- AOARD (FA2386-14-1-0022)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Decision Sciences › Information Systems and Management › Technology Adoption and User Behaviour
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- user trust, trust dynamics, automated systems, system failure, system performance, user familiarity, trust prediction, human-computer interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.