Agent Transparency and Reliability in Human–Robot Interaction: The Influence on User Confidence and Perceived Reliability
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Agent transparency is an important contributor to human performance, situation awareness (SA), and trust in human-agent teaming. However, agent transparency's effects on human performance when the agent is unreliable have yet to be examined. This paper examined how the transparency and reliability of an autonomous robotic squad member (ASM) affected a human observer's task performance, workload, SA, trust in the robot, and perceptions of the robot. In a 2 (ASM transparency) × 2 (ASM reliability) within-subject design experiment, participants monitored a simulated soldier squad that included an ASM as it traversed a simulated training environment, while concurrently monitoring the environment for targets. There was no difference in participants' performance on the target detection task, workload, or SA due to either ASM transparency or reliability. ASM reliability influenced participant trust and perceptions of the robot. Results suggest that reliability may be a stronger influence on the human's perceptions of the robot than transparency. Robot errors had a profound and lasting effect on the participants' perception of the robot's future reliability and resulted in reduced confidence in their assessments of the robot's reliability. These findings could have important implications for the continued use of automated systems when the user is aware of system errors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 5×
“SA is defined as “ the perception of the elements in the environments within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future ” [9].”
BackgroundResultInfluentialHumans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · cited 5×
“Appropriate trust is essential to effective human–agent teaming; too much trust can result in operator complacency or misuse, while too little trust can result in disuse [21].”
BackgroundResultInfluentialA model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 2×
“Participants also reported higher trust in the ASM when it was reliable for each of the four automation functions [41] than when it was unreliable.”
BackgroundBartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009) · International Journal of Social Robotics · doi:10.1007/s12369-008-0001-3 · Cited by 3,193 · cited 1×
“7) Humanness: The godspeed questionnaire series (GQS) was administered after each mission [44].”
MethodDehumanization: An integrative review
Haslam, N. (2006) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr1003_4 · Cited by 2,919 · cited 1×
“4) Humanness: Humanness includes both characteristics that are uniquely human (e.g., morality, civility, etc.) and those that are part of human nature (e.g., cognitive flexibility, emotion-ality, etc.) [33].”
BackgroundTrust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817 · cited 1×
“This finding suggests that in-depth knowledge of the underlying reasons for an agent’s actions and reasoning may bolster trust in the agent and mitigate the effects of unreliable automation, at least so far as specific functions are concerned [51], [52]–[54].”
BackgroundConditions for intuitive expertise: A failure to disagree.
Kahneman, D., & Klein, G. (2009) · American Psychologist · doi:10.1037/a0016755 · Cited by 2,443 · cited 1×
“While subjective confidence is not a reliable indicator of the validity of one’s judgments [49], persons who lack confidence tend to be overly cautious, hesitant in decision-making, and slow to enact action [24].”
BackgroundCacioppo, J. T., & Berntson, G. G. (1994) · Psychological Bulletin · doi:10.1037//0033-2909.115.3.401 · Cited by 1,882
Foundations for an Empirically Determined Scale of Trust in Automated Systems
Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000) · International Journal of Cognitive Ergonomics · doi:10.1207/s15327566ijce0401_04 · Cited by 1,799
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · cited 1×
“This finding suggests that in-depth knowledge of the underlying reasons for an agent’s actions and reasoning may bolster trust in the agent and mitigate the effects of unreliable automation, at least so far as specific functions are concerned [51], [52]–[54].”
BackgroundPerformance Consequences of Automation-Induced 'Complacency'
Parasuraman, R., Molloy, R., & Singh, I. L. (1993) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0301_1 · Cited by 875 · cited 1×
“On the other hand, there are also examples where highly reliable automation was detrimental to human performance, resulting in overtrust and complacent behavior [7], [13], [19], [22].”
BackgroundMUIR, B. M. (1994) · Ergonomics · doi:10.1080/00140139408964957 · Cited by 873 · cited 1×
“This finding suggests that in-depth knowledge of the underlying reasons for an agent’s actions and reasoning may bolster trust in the agent and mitigate the effects of unreliable automation, at least so far as specific functions are concerned [51], [52]–[54].”
BackgroundSalem, M., Lakatos, G., Amirabdollahian, F., & Dautenhahn, K. (2015) · Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2696454.2696497 · Cited by 494 · cited 1×
“the agent committed [31], and if the display of information to support transparency information aided in mitigating the impact of the errors [32].”
BackgroundThe benefits of imperfect diagnostic automation: a synthesis of the literature
Wickens, C. D., & Dixon, S. R. (2007) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500370105 · Cited by 423
Intelligent agent transparency in human–agent teaming for multi-UxV management
Mercado, J. E., Rupp, M. A., Chen, J. Y. C., Barnes, M. J., Barber, D., & Procci, K. (2016) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815621206 · Cited by 403 · cited 6×
“Researchers have used the SAT model to inform human–robot interfaces in different agent paradigms, ranging from virtual decision aides to physically instantiated robotic teammates [6], [13], [14].”
BackgroundMethodResultInfluentialSelf-citationde Visser, E., & Parasuraman, R. (2011) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343411410160 · Cited by 207 · cited 1×
“Consistent with the findings of previous studies on automation reliability and trust [9], [21], [50], reliability, not information level, influenced participant trust.”
ResultTransparency in a Human-Machine Context: Approaches for Fostering Shared Awareness/Intent
Lyons, J. B., & Havig, P. R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-07458-0_18 · Cited by 96 · cited 1×
“Agent transparency has been identified as an essential factor in calibrating operator trust in an agent [2], [10], as well as being critical for the development of appropriate SA in human–robot teams [11], [12].”
BackgroundInsights into Human-Agent Teaming: Intelligent Agent Transparency and Uncertainty
Stowers, K., Kasdaglis, N., Rupp, M., Chen, J., Barber, D., & Barnes, M. (2017) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-319-41959-6_13 · Cited by 28
Self-citation
Cited by
Towards a Theory of Longitudinal Trust Calibration in Human–Robot Teams
de Visser, E. J., Peeters, M. M. M., Jung, M. F., Kohn, S., Shaw, T. H., Pak, R., & Neerincx, M. A. (2020) · International Journal of Social Robotics · doi:10.1007/s12369-019-00596-x · Cited by 447
Bobko, P., Hirshfield, L., Eloy, L., Spencer, C., Doherty, E., Driscoll, J., & Obolsky, H. (2023) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2022.2086644 · Cited by 33
Social Influence under uncertainty in interaction with peers, robots and computers
Zonca, J., Folsø, A., & Sciutti, A. (2023) · International Journal of Social Robotics · doi:10.1007/s12369-022-00959-x · Cited by 18
Metadata
- Authors
- Wright, J. L., Chen, J. Y. C., & Lakhmani, S. G.
- Venue
- IEEE Transactions on Human-Machine Systems
- Published
- 2020-06
- Volume
- 50
- Issue
- 3
- Pages
- 254-263
- ISSN
- 2168-2291 (ISSN-L)
- 2168-2305
- OpenAlex
- W2968357651
- Cited by
- 120
- References
- 19
- FWCI
- 12.40
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.