Planning with Trust for Human-Robot Collaboration
Abstract
Trust is essential for human-robot collaboration and user adoption of autonomous systems, such as robot assistants. This paper introduces a computational model which integrates trust into robot decision-making. Specifically, we learn from data a partially observable Markov decision process (POMDP) with human trust as a latent variable. The trust-POMDP model provides a principled approach for the robot to (i) infer the trust of a human teammate through interaction, (ii) reason about the effect of its own actions on human behaviors, and (iii) choose actions that maximize team performance over the long term. We validated the model through human subject experiments on a table-clearing task in simulation (201 participants) and with a real robot (20 participants). The results show that the trust-POMDP improves human-robot team performance in this task. They further suggest that maximizing trust in itself may not improve team performance.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The need for trust in collaborative tasks might increase for situations of human-AI teaming, where the human team member is dependent on the performance of the system.
“However, need for trust in collaborative tasks might increase for situations of human-AI teaming, where the human team member is dependent on the performance of the system, such as [18,24,116] report.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [18]
Chen et al. developed a POMDP computational model that integrates users' trust into system decision making by inferring users' trust through interactions.
“Similary, Chen and colleagues [18] developed a computational model, the Partially observable Markov decision process (POMDP), which integrates the extend users trust the system into the system's decision making by, among others, inferring users' trust through interactions.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [18]
Chen et al. experiment showed the system adapted to users' level of trust, which increased overall human-robot team performance.
“Through an experiment Chen et al. could show that the system adopted to users' level of trust which increased the overall human-robot team performance.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [18]
Cites
<i>Stan</i> : A Probabilistic Programming Language
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017) · Journal of Statistical Software · doi:10.18637/jss.v076.i01 · Cited by 7,754 · cited 2× · see citing works
“We performed Bayesian inference on the two models above using Hamiltonian Monte Carlo sampling [4].”
MethodTrust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 2× · see citing works
“Trust in automation differs from trust between people in that automation lacks intentionality [20].”
BackgroundEvaluating Effects of User Experience and System Transparency on Trust in Automation
Yang, X. J., Unhelkar, V. V., Li, K., & Shah, J. A. (2017) · Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2909824.3020230 · Cited by 208 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Wischnewski, M., Krämer, N., & Müller, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581197 · Cited by 146 · see citing works
“Similary, Chen and colleagues [18] developed a computational model, the Partially observable Markov decision process (POMDP), which integrates the extend users trust the system into the system’s decision making by, among others, inferring users’ trust through interactions.”
BackgroundInfluentialKim, J. Y., Lester, C., & Yang, X. J. (2025) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208251326795 · Cited by 21 · see citing works
A Computational Model of Coupled Human Trust and Self-confidence Dynamics
Williams, K. J., Yuh, M. S., & Jain, N. (2023) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3594715 · Cited by 18 · see citing works
Reasoning about Cognitive Trust in Stochastic Multiagent Systems
Huang, X., Kwiatkowska, M., & Olejnik, M. (2019) · ACM Transactions on Computational Logic · doi:10.1145/3329123 · Cited by 18 · see citing works
McMahon, G., Akash, K., Reid, T., & Jain, N. (2020) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2021.04.113 · Cited by 10 · see citing works
Towards Explainable and Trustworthy Autonomous Physical Systems
Omeiza, D., Anjomshoae, S., Kollnig, K., Camburu, O.-M., Främling, K., & Kunze, L. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3441338 · Cited by 2 · see citing works
- Authors
- Chen, M., Nikolaidis, S., Soh, H., Hsu, D., & Srinivasa, S.
- Affiliations
- National University of Singapore (SG)
- Carnegie Mellon University (US)
- University of Washington (US)
- Venue
- Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction
- Publisher
- ACM
- Published
- 2018-02-26
- Pages
- 307-315
- OpenAlex
- W2783036728
- Cited by
- 221
- References
- 3
- Citations by year
- 2026: 8 · 2025: 25 · 2024: 32 · 2023: 35 · 2022: 24 · 2021: 46 · 2020: 27 · 2019: 20 · 2018: 4
- FWCI
- 43.48
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/IEEE International Conference on Human-Robot Interaction (HRI) (publishedVersion)
- Funders
- National Science Foundation CPS (#1544797)
- National Science Foundation NRI (#1637748)
- National Institute of Health R01 (#R01EB019335)
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Keywords
- human-robot collaboration, trust, robot decision-making, partially observable Markov decision processes, robotic assistants, human-robot interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
