Modeling the Human-Robot Trust Phenomenon: A Conceptual Framework based on Risk
Abstract
This article presents a conceptual framework for human-robot trust which uses computational representations inspired by game theory to represent a definition of trust, derived from social psychology. This conceptual framework generates several testable hypotheses related to human-robot trust. This article examines these hypotheses and a series of experiments we have conducted which both provide support for and also conflict with our framework for trust. We also discuss the methodological challenges associated with investigating trust. The article concludes with a description of the important areas for future research on the topic of human-robot trust.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Wagner et al. proposed a risk-based trust model defining trust as a belief that the trustee will act to mitigate the trustor's risk when the trustor's outcomes are at risk.
“Wagner et al. [75] propose a trust model based on risk.They define trust as “a belief, held by the trustor, that the trustee will act in a manner that mitigates the trustor's risk in a situation in which the trustor has put its outcomes at risk” [75, p.26:4].”
From Taxonomy of Trust-Relevant Failures and Mitigation Strategies (2020) · cites this paper as [75]
Cites
Axelrod, R., & Hamilton, W. D. (1981) · Science · doi:10.1126/science.7466396 · Cited by 20,065 · see citing works
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · see citing works
Not So Different After All: A Cross-Discipline View Of Trust
Rousseau, D. M., Sitkin, S. B., Burt, R. S., & Camerer, C. (1998) · Academy of Management Review · doi:10.5465/amr.1998.926617 · Cited by 10,274 · see citing works
Construal-level theory of psychological distance.
Trope, Y., & Liberman, N. (2010) · Psychological Review · doi:10.1037/a0018963 · Cited by 6,618 · see citing works
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 4× · see citing works
“From a research standpoint, many technical definitions have been posed [3, 8, 13, 16, 24, 26, 28, 29, 53].”
BackgroundMethodInfluentialNielsen, R. P., & Axelrod, R. (1985) · Academy of Management Review · doi:10.2307/257983 · Cited by 6,517 · see citing works
(1994) · Cited by 5,513 · see citing works
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767 · see citing works
Rempel, J. K., Holmes, J. G., & Zanna, M. P. (1985) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.49.1.95 · Cited by 3,146 · cited 2× · see citing works
“Thus, our framework predicts and it has been shown that iterative favorable interaction with a robot increases trust [34].”
Background(1979) · Medical Entomology and Zoology · Cited by 3,065 · see citing works
Interpersonal Relations: A Theory of Interdependence
Hansen, D. A., Kelley, H. H., & Thibaut, J. W. (1982) · Journal of Marriage and the Family · doi:10.2307/351281 · Cited by 2,782 · see citing works
Nebraska Symposium on Motivation, 1961
Whitehorn, J. C. (1963) · American Journal of Psychiatry · doi:10.1176/ajp.119.9.904 · Cited by 2,361 · see citing works
A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., & Parasuraman, R. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811417254 · Cited by 2,051 · cited 3× · see citing works
“Hancock et al. [17] performed a meta-analysis over the existing human-robot trust literature identifying 11 relevant research articles and found that, for these articles, robot performance is most strongly associated with trust.”
BackgroundBarber, B. (1983) · DigitalGeorgetown (Georgetown University Library) · Cited by 1,801 · see citing works
Interpersonal relations — a theory of interdependence
Maldé, B. (1979) · Applied Ergonomics · doi:10.1016/0003-6870(79)90259-x · Cited by 1,780 · see citing works
Artificial Intelligence: A Modern Approach
Russell, S., & Norvig, P. (1995) · Cited by 1,753 · see citing works
The Stag Hunt and the Evolution of Social Structure
Skyrms, B. (2003) · Cambridge University Press eBooks · doi:10.1017/cbo9781139165228 · Cited by 1,527 · see citing works
Formalising Trust as a Computational Concept
Marsh, S. (1994) · Cited by 1,472 · see citing works
A neural basis for social cooperation
Rilling, J. K., Gutman, D. A., Zeh, T. R., Pagnoni, G., Berns, G. S., & Kilts, C. D. (2002) · Neuron · doi:10.1016/s0896-6273(02)00755-9 · Cited by 1,451 · see citing works
Getting to Know You: Reputation and Trust in a Two-Person Economic Exchange
King-Casas, B., Tomlin, D., Anen, C., Camerer, C. F., Quartz, S. R., & Montague, P. R. (2005) · Science · doi:10.1126/science.1108062 · Cited by 1,345 · see citing works
Automatic and intentional brain responses during evaluation of trustworthiness of faces
Winston, J. S., Strange, B. A., O'Doherty, J., & Dolan, R. J. (2002) · Nature Neuroscience · doi:10.1038/nn816 · Cited by 993 · see citing works
The neural correlates of theory of mind within interpersonal interactions
Rilling, J. K., Sanfey, A. G., Aronson, J. A., Nystrom, L. E., & Cohen, J. D. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.04.015 · Cited by 638 · see citing works
Psychological Foundations of Trust
Simpson, J. A. (2007) · Current Directions in Psychological Science · doi:10.1111/j.1467-8721.2007.00517.x · Cited by 541 · see citing works
Salem, M., Lakatos, G., Amirabdollahian, F., & Dautenhahn, K. (2015) · Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2696454.2696497 · Cited by 494 · see citing works
A Cross-cultural Study: Effect of Robot Appearance and Task
Li, D., Rau, P. P., & Li, Y. (2010) · International Journal of Social Robotics · doi:10.1007/s12369-010-0056-9 · Cited by 492 · see citing works
The Benefits of Interactions with Physically Present Robots over Video-Displayed Agents
Bainbridge, W. A., Hart, J., Kim, E. S., & Scassellati, B. M. (2010) · International Journal of Social Robotics · doi:10.1007/s12369-010-0082-7 · Cited by 478 · see citing works
Overtrust of robots in emergency evacuation scenarios
Robinette, P., Li, W., Allen, R., Howard, A. M., & Wagner, A. R. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451740 · Cited by 468 · see citing works
Self-citationTrust Theory: A Socio-Cognitive and Computational Model
Castelfranchi, C., & Falcone, R. (2010) · Cited by 392 · see citing works
The effects of personality and situational variables on behavioral trust.
Schlenker, B. R., Helm, B., & Tedeschi, J. T. (1973) · Journal of Personality and Social Psychology · doi:10.1037/h0034088 · Cited by 327 · see citing works
Seeing is believing: Trustworthiness as a dynamic belief
Chang, L. J., Doll, B. B., Wout‐Frank, M. V. ‘., Frank, M. J., & Sanfey, A. G. (2010) · Cognitive Psychology · doi:10.1016/j.cogpsych.2010.03.001 · Cited by 308 · see citing works
Using trust for detecting deceitful agents in artificial societies
Schillo, M., Funk, P., & Rovatsos, M. (2000) · Applied Artificial Intelligence · doi:10.1080/08839510050127579 · Cited by 273 · see citing works
Trust, trustworthiness, and the F scale.
Deutsch, M. (1960) · Journal of Abnormal & Social Psychology · doi:10.1037/h0046501 · Cited by 267 · see citing works
Trust as a form of social intelligence.
Yamagishi, T. (2001) · Cited by 259 · see citing works
Group decision making under stress.
Driskell, J. E., & Eduardo Salas, A. (1991) · Journal of Applied Psychology · doi:10.1037/0021-9010.76.3.473 · Cited by 256 · see citing works
You Want Me to Trust a ROBOT? The Development of a Human–Robot Interaction Trust Scale
Yagoda, R. E., & Gillan, D. (2012) · International Journal of Social Robotics · doi:10.1007/s12369-012-0144-0 · Cited by 252 · see citing works
Effect of Robot Performance on Human–Robot Trust in Time-Critical Situations
Robinette, P., Howard, A. M., & Wagner, A. R. (2017) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2017.2648849 · Cited by 205 · see citing works
Self-citationTrust in virtual teams: towards an integrative model of trust formation
Hung, Y. C., Dennis, A. R., & Robert, L. (2004) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2004.1265156 · Cited by 150 · see citing works
Impact of robot failures and feedback on real-time trust
Desai, M., Kaniarasu, P., Medvedev, M., Steinfeld, A. M., & Yanco, H. A. (2013) · doi:10.5555/2447556.2447663 · Cited by 144 · see citing works
Prakash, A., & Rogers, W. A. (2014) · International Journal of Social Robotics · doi:10.1007/s12369-014-0269-4 · Cited by 124 · see citing works
Timing Is Key for Robot Trust Repair
Robinette, P., Howard, A. M., & Wagner, A. R. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-25554-5_57 · Cited by 121 · see citing works
Self-citationLearning to trust in indefinitely repeated games
Engle‐Warnick, J., & Slonim, R. L. (2004) · Games and Economic Behavior · doi:10.1016/j.geb.2004.10.009 · Cited by 115 · see citing works
Do Robot Performance and Behavioral Style affect Human Trust?
van den Brule, R., Dotsch, R., Bijlstra, G., Wigboldus, D. H. J., & Haselager, P. (2014) · International Journal of Social Robotics · doi:10.1007/s12369-014-0231-5 · Cited by 115 · see citing works
Proceedings of the 4th ACM/IEEE international conference on Human robot interaction
Scheutz, M. J., Michaud, F., HINDS, P. J., & Scassellati, B. M. (2009) · doi:10.1145/1514095 · Cited by 92 · see citing works
The 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2016)
OKADA, M. (2016) · Journal of the Robotics Society of Japan · doi:10.7210/jrsj.34.697 · Cited by 89 · see citing works
The theoretical description of interdependence by means of transition lists.
Kelley, H. H. (1984) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.47.5.956 · Cited by 89 · see citing works
Booth, S., Tompkin, J., Pfister, H., Waldo, J., Gajos, K. Z., & Nagpal, R. (2017) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/2909824.3020211 · Cited by 84 · see citing works
When Giving Is Good: Ventromedial Prefrontal Cortex Activation for Others' Intentions
Cooper, J. C., Kreps, T. A., Wiebe, T., Pirkl, T., & Knutson, B. D. (2010) · Neuron · doi:10.1016/j.neuron.2010.06.030 · Cited by 78 · see citing works
Modeling human behavior during building fires
Kuligowski, E. D. (2008) · doi:10.6028/nist.tn.1619 · Cited by 57 · see citing works
The role of trust and relationships in human-robot social interaction
Wagner, A. R. (2009) · Cited by 50 · see citing works
Self-citationOvertrust of Pediatric Health-Care Robots: A Preliminary Survey of Parent Perspectives
Borenstein, J., Wagner, A. R., & Howard, A. (2018) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2017.2778743 · Cited by 44 · see citing works
Self-citationKnowledge and the development of interpersonal trust: a dynamic model
Luna‐Reyes, L. F., Cresswell, A. M., & Richardson, G. P. (2004) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2004.1265241 · Cited by 38 · see citing works
Recognizing situations that demand trust
Wagner, A. R., & Arkin, R. C. (2011) · doi:10.1109/roman.2011.6005228 · Cited by 36 · see citing works
Self-citationBuilding and maintaining trust between humans and guidance robots in an emergency
Robinette, P., Wagner, A. R., & Howard, A. (2013) · Cited by 25 · see citing works
Self-citationBoundedly Rational and Emotional Agents Cooperation, Trust and Rumor
Prietula, M. J., & Carley, K. M. (2001) · Springer eBooks · doi:10.1007/978-94-017-3614-5_9 · Cited by 21 · see citing works
Investigating Human-Robot Trust in Emergency Scenarios: Methodological Lessons Learned
Robinette, P., Wagner, A. R., & Howard, A. (2016) · Springer eBooks · doi:10.1007/978-1-4899-7668-0_8 · Cited by 21 · see citing works
Self-citationCreating and using matrix representations of social interaction
Wagner, A. R. (2009) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/1514095.1514119 · Cited by 17 · see citing works
Self-citationWagner, A. R., & Robinette, P. (2015) · Interaction Studies Social Behaviour and Communication in Biological and Artificial Systems · doi:10.1075/is.16.1.05wag · Cited by 14 · see citing works
Self-citation11th International Conference on Social Robotics, ICSR
Rossi, S., & Lakatos, G. (2016) · Cited by 12 · see citing works
The Stag Hunt and the Evolution of Social Structure
van Rooij, R. (2007) · Studia Logica · doi:10.1007/s11225-007-9024-2 · Cited by 8 · see citing works
Modeling Human-Robot Trust in Emergencies
Robinette, P., Wagner, A. R., & Howard, A. (2014) · Cited by 5 · see citing works
Self-citationRobots that stereotype: creating and using categories of people for human-robot interaction
Wagner, A. R. (2015) · doi:10.5555/3109842.3109847 · Cited by 4 · see citing works
Self-citationDeveloping Robots that Recognize When They Are Being Trusted
Wagner, A. R. (2013) · Cited by 4 · see citing works
Self-citationA Framework for Situation-based Social Interaction
Wagner, A. R., & Arkin, R. C. (2006) · IEEE RO-MAN · doi:10.1109/roman.2006.314432 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Taxonomy of Trust-Relevant Failures and Mitigation Strategies
Tolmeijer, S., Weiss, A., Hanheide, M., Lindner, F., Powers, T. M., Dixon, C., & Tielman, M. L. (2020) · Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3319502.3374793 · Cited by 103 · see citing works
Integrity-based Explanations for Fostering Appropriate Trust in AI Agents
Mehrotra, S., Jorge, C. C., Jonker, C. M., & Tielman, M. L. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3610578 · Cited by 51 · see citing works
Trust Dynamics and Verbal Assurances in Human Robot Physical Collaboration
Alhaji, B., Prilla, M., & Rausch, A. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.703504 · Cited by 29 · see citing works
A Computational Model of Coupled Human Trust and Self-confidence Dynamics
Williams, K. J., Yuh, M. S., & Jain, N. (2023) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3594715 · Cited by 18 · see citing works
- Authors
- Wagner, A. R., Robinette, P., & Howard, A.
- Affiliations
- Pennsylvania State University (US)
- Massachusetts Institute of Technology (US)
- Georgia Institute of Technology (US)
- Venue
- ACM Transactions on Interactive Intelligent Systems
- Published
- 2018-11-16
- Volume
- 8
- Issue
- 4
- Pages
- 1-24
- ISSN
- 2160-6455 (ISSN-L)
- 2160-6463
- OpenAlex
- W2901499449
- Cited by
- 51
- References
- 63
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 12 · 2023: 11 · 2022: 8 · 2021: 7 · 2020: 8 · 2019: 1
- FWCI
- 5.42
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Interactive Intelligent Systems (publishedVersion)
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Social Sciences › Sociology and Political Science › Access Control and Trust
- Keywords
- trust in robots, trust model, game theory, social psychology of trust, conceptual framework, human-robot interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
