The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Evaluating Effects of User Experience and System Transparency on Trust in Automation
Open accessAbstract
Existing research assessing human operators' trust in automation and robots has primarily examined trust as a steady-state variable, with little emphasis on the evolution of trust over time. With the goal of addressing this research gap, we present a study exploring the dynamic nature of trust. We defined trust of entirety as a measure that accounts for trust across a human's entire interactive experience with automation, and first identified alternatives to quantify it using real-time measurements of trust. Second, we provided a novel model that attempts to explain how trust of entirety evolves as a user interacts repeatedly with automation. Lastly, we investigated the effects of automation transparency on momentary changes of trust. Our results indicated that trust of entirety is better quantified by the average measure of "area under the trust curve" than the traditional post-experiment trust measure. In addition, we found that trust of entirety evolves and eventually stabilizes as an operator repeatedly interacts with a technology. Finally, we observed that a higher level of automation transparency may mitigate the "cry wolf" effect -- wherein human operators begin to reject an automated system due to repeated false alarms.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In addition, some studies also employed linear trust meters [137], or visual analog scales [139].”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [139]
In the citing paper
“Some authors work with correlations to get a better sense of perceived trustworthiness and system capabilities [5,12,24,39,66,67,81,94,130,139,142].Here the larger the correlation between perceived trustworthiness and system capabilities, the better the calibration.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [139]
In the citing paper
“With few exceptions, we have little understanding of how trust strengthens or decays due to moment-to-moment interactions with automation”
From Toward Quantifying Trust Dynamics: How People Adjust Their Trust After Moment-to-Moment Interaction With Automation (2023) · cites this paper as Yang et al., 2017
In the citing paper
“trust is a dynamic variable that can change over time”
From Toward Quantifying Trust Dynamics: How People Adjust Their Trust After Moment-to-Moment Interaction With Automation (2023) · cites this paper as Yang et al., 2017
In the citing paper
“Critically, the opaqueness of the AI's internal computational processes may decrease calibrated reliance on the system [63 , 64] .”
From Enhancing human-AI collaboration: The case of colonoscopy (2024) · cites this paper as [64]
In the citing paper
“Such measures include, but are not limited to measuring end-users' decision time [19,62], how many times end-users requested AI recommendation [52], and how many times they agreed with AI recommendations [7,23,58].”
From Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust (2023) · cites this paper as [58]
In the citing paper
“Most of the studies investigated how to maintain appropriate trust calibration by continuously presenting system information to prevent over-trust or under-trust.”
From Adaptive trust calibration for human-AI collaboration (2020) · cites this paper as [13]
In the citing paper
“Several studies [13, 23, 26] have demonstrated that reliance behavior can be explained by the relationship between a user’s trust in the agents and the user’s self-confidence.”
From Adaptive trust calibration for human-AI collaboration (2020) · cites this paper as [13]
Citations
Cites
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Engineering Psychology and Human Performance
Wickens, C. D., Hollands, J. G., Banbury, S. P., & Parasuraman, R. (2015) · Psychology Press eBooks · doi:10.4324/9781315665177 · Cited by 5,017
Memory: A Contribution to Experimental Psychology
Ebbinghaus, H. (2013) · Annals of Neurosciences · doi:10.5214/ans.0972.7531.200408 · Cited by 2,827
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., & Parasuraman, R. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811417254 · Cited by 2,051 · cited 2×
“Research has indicated that the calibration between operators’ trust and an automated/robotic technology’s actual ability is often imperfect [9, 10].”
BackgroundMemory: A contribution to experimental psychology.
Ebbinghaus, H. (1913) · doi:10.1037/10011-000 · Cited by 1,645
Trust, control strategies and allocation of function in human-machine systems
LEE, J., & MORAY, N. (1992) · Ergonomics · doi:10.1080/00140139208967392 · Cited by 1,459
A decision-making theory of visual detection.
Tanner, W. P., & Swets, J. A. (1954) · Psychological Review · doi:10.1037/h0058700 · Cited by 1,357
Trust, self-confidence, and operators' adaptation to automation
Lee, J. D., & Moray, N. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1007 · Cited by 932
Trust between humans and machines, and the design of decision aids
Muir, B. M. (1987) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(87)80013-5 · Cited by 808
Human–Robot Interaction in Rescue Robotics
Murphy, R. R. (2004) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2004.826267 · Cited by 681
Cooperative forest fire surveillance using a team of small unmanned air vehicles
Casbeer, D. W., Kingston, D. B., Beard, R. W., & McLain, T. W. (2006) · International Journal of Systems Science · doi:10.1080/00207720500438480 · Cited by 536
Overtrust of robots in emergency evacuation scenarios
Robinette, P., Li, W., Allen, R., Howard, A. M., & Wagner, A. R. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451740 · Cited by 468
Intelligent agent transparency in human–agent teaming for multi-UxV management
Mercado, J. E., Rupp, M. A., Chen, J. Y. C., Barnes, M. J., Barber, D., & Procci, K. (2016) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815621206 · Cited by 403
Border patrol and surveillance missions using multiple unmanned air vehicles
Girard, A., Howell, A. S., & Hedrick, J. K. (2004) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2004.1428713 · Cited by 287
Manzey, D., Reichenbach, J., & Onnasch, L. (2012) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343411433844 · Cited by 277
On the Independence of Compliance and Reliance: Are Automation False Alarms Worse Than Misses?
Dixon, S. R., Wickens, C. D., & McCarley, J. S. (2007) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872007x215656 · Cited by 265
Impact of robot failures and feedback on real-time trust
Desai, M., Kaniarasu, P., Medvedev, M., Steinfeld, A., & Yanco, H. (2013) · 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2013.6483596 · Cited by 234
Trust calibration within a human-robot team: Comparing automatically generated explanations
Wang, N., Pynadath, D. V., & Hill, S. G. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451741 · Cited by 220
Effects of changing reliability on trust of robot systems
Desai, M., Medvedev, M., Vázquez, M., McSheehy, S., Gadea-Omelchenko, S., Bruggeman, C., Steinfeld, A., & Yanco, H. (2012) · Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction · doi:10.1145/2157689.2157702 · Cited by 188
Sorkin, R. D., Kantowitz, B. H., & Kantowitz, S. C. (1988) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/001872088803000406 · Cited by 187
Madhavan, P., & Wiegmann, D. A. (2007) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872007x230154 · Cited by 176
Impact of robot failures and feedback on real-time trust
Desai, M., Kaniarasu, P., Medvedev, M., Steinfeld, A. M., & Yanco, H. A. (2013) · doi:10.5555/2447556.2447663 · Cited by 144
The DARPA Robotics Challenge [Competitions]
Pratt, G. A., & Manzo, J. E. (2013) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2013.2255424 · Cited by 105
Effects of changing reliability on trust of robot systems
Desai, Medvedev, Vázquez, McSheehy, Gadea-Omelchenko, Bruggeman, Steinfeld, & Yanco (2012) · Cited by 72
Supporting Attention Allocation in Multitask Environments
Wiczorek, R., & Manzey, D. (2014) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814528534 · Cited by 68
Coordination in human-robot teams using mental modeling and plan recognition
Talamadupula, K., Briggs, G., Chakraborti, T., Scheutz, M. J., & Subbarao Kambhampati, B. (2014) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2014.6942970 · Cited by 59
How users adjust trust in automation
Yang, X. J., Wickens, C. D., & Hölttä‐Otto, K. (2016) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213601044 · Cited by 40
Self-citationHuman-Robot Teaming in Urban Search and Rescue
Bartlett, C. E., & Cooke, N. J. (2015) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931215591051 · Cited by 33
Social interaction moderates human-robot trust-reliance relationship and improves stress coping
Lohani, M., Stokes, C. K., McCoy, M., Bailey, C. A., & Rivers, S. E. (2016) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451811 · Cited by 32
Robotics operator performance in a military multi-tasking environment
Chen, J. Y. C., & Barnes, M. J. (2008) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/1349822.1349859 · Cited by 30
Effect of indirect information on system trust and control allocation
de Vries, P. W., & Midden, C. C. (2006) · Behaviour and Information Technology · doi:10.1080/01449290600874956 · Cited by 26
FACTORS THAT AFFECT TRUST AND RELIANCE ON AN AUTOMATED AID
Sánchez, J. (2006) · UPT. Syiah Kuala University Library (Syiah Kuala University) · Cited by 24
Social Interaction Moderates Human-Robot Trust-Reliance Relationship and Improves Stress Coping
Lohani, M., Stokes, C. K., McCoy, M., Bailey, C. A., & Rivers, S. E. (2016) · doi:10.5555/2906831.2906932 · Cited by 18
43rd IEEE Conference on Decision and Control
(2011) · IEEE Control Systems · doi:10.1109/mcs.2011.942937 · Cited by 3
Flagged first, then by citations
Research works printed but not linked: 1
- Chen , J. Y. C. 2010. Robotics operator performance in a multi-tasking environment: Human-Robot Interactions in Future Military Operations . Ashgate Publishing , 294--314. Chen, J. Y. C. 2010. Robotics operator performance in a multi-tasking environment: Human-Robot Interactions in Future Military Operations. Ashgate Publishing, 294--314.
Cited by
Sorted by citations
Towards a Theory of Longitudinal Trust Calibration in Human–Robot Teams
de Visser, E. J., Peeters, M. M. M., Jung, M. F., Kohn, S., Shaw, T. H., Pak, R., & Neerincx, M. A. (2020) · International Journal of Social Robotics · doi:10.1007/s12369-019-00596-x · Cited by 447
Measurement of trust in automation: A narrative review and reference guide
Kohn, S. C., de Visser, E. J., Wiese, E., Lee, Y.-C., & Shaw, T. H. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.604977 · Cited by 315
Kraus, J., Scholz, D., Stiegemeier, D., & Baumann, M. (2020) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720819853686 · Cited by 276
Adaptive trust calibration for human-AI collaboration
Okamura, K., & Yamada, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0229132 · Cited by 239
How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies
Vereschak, O., Bailly, G., & Caramiaux, B. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476068 · Cited by 229
Metadata
- Authors
- Yang, X. J., Unhelkar, V. V., Li, K., & Shah, J. A.
- Affiliations
- University of Michigan (US)
- Massachusetts Institute of Technology (US)
- Venue
- Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction
- Publisher
- ACM
- Published
- 2017-03-06
- Pages
- 408-416
- OpenAlex
- W2594678573
- Cited by
- 208
- References
- 36
- Citations by year
- 2026: 13 · 2025: 26 · 2024: 41 · 2023: 29 · 2022: 22 · 2021: 25 · 2020: 23 · 2019: 18 · 2018: 10 · 2017: 1
- FWCI
- 23.05
- Type
- conference paper
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- Locations
- ACM/IEEE International Conference on Human-Robot Interaction (HRI) (publishedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-sa)
- Funders
- Singapore University of Technology and Design
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- trust in automation, automation transparency, user experience, trust dynamics, false alarms, human-automation interaction, trust measurement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
