CoChat

The Development of Overtrust: An Empirical Simulation and Psychological Analysis in the Context of Human–Robot Interaction

Open access
Ullrich, Daniel, Butz, Andreas, Diefenbach, Sarah · 2021 · Frontiers in Robotics and AI
Cited by 69

The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.

Abstract

With impressive developments in human–robot interaction it may seem that technology can do anything. Especially in the domain of social robots which suggest to be much more than programmed machines because of their anthropomorphic shape, people may overtrust the robot's actual capabilities and its reliability. This presents a serious problem, especially when personal well-being might be at stake. Hence, insights about the development and influencing factors of overtrust in robots may form an important basis for countermeasures and sensible design decisions. An empirical study [N= 110] explored the development of overtrust using the example of a pet feeding robot. A 2 × 2 experimental design and repeated measurements contrasted the effect of one's own experience, skill demonstration, and reputation through experience reports of others. The experiment was realized in a video environment where the participants had to imagine they were going on a four-week safari trip and leaving their beloved cat at home, making use of a pet feeding robot. Every day, the participants had to make a choice: go to a day safari without calling options (risk and reward) or make a boring car trip to another village to check if the feeding was successful and activate an emergency call if not (safe and no reward). In parallel to cases of overtrust in other domains (e.g., autopilot), the feeding robot performed flawlessly most of the time until in the fourth week; it performed faultily on three consecutive days, resulting in the cat's death if the participants had decided to go for the day safari on these days. As expected, with repeated positive experience about the robot's reliability on feeding the cat, trust levels rapidly increased and the number of control calls decreased. Compared to one's own experience, skill demonstration and reputation were largely neglected or only had a temporary effect. We integrate these findings in a conceptual model of (over)trust over time and connect these to related psychological concepts such as positivism, instant rewards, inappropriate generalization, wishful thinking, dissonance theory, and social concepts from human–human interaction. Limitations of the present study as well as implications for robot design and future research are discussed.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/frobt.2021.554578

Cites

19 of 19 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 29

  1. Barber, 1983, The logic and limits of trust
  2. Bartneck, 2009, Does the design of a robot influence its animacy and perceived intelligence?, Int. J. Soc. Robotics., 1, 195, 10.1007/s12369-009-0013-7 · DOI · Paper page
  3. Böhm, 1996, Instrumental or emotional evaluations: what determines preferences?, Acta Psychol (Amst)., 93, 135, 10.1016/0001-6918(96)00017-0 · DOI
  4. Carlson, 2014, Identifying factors that influence trust in automated cars and medical diagnosis systems
  5. Cathcart, 1997, The media equation: how people treat computers, television, and new media like real people and places, 305
  6. Chen, 2010, Supervisory control of unmanned vehicles
  7. Coeckelbergh, 2012, Can we trust robots?, Ethics Inf. Technol., 14, 53, 10.1007/s10676-011-9279-1 · DOI · Paper page
  8. Darling, 2012, Extending legal rights to social robots, 10.2139/ssrn.2044797 · DOI
  9. Diefenbach, 2019, Combining model-based analysis with phenomenological insight: a case study on hedonic product quality, Qual. Psychol., 6, 3, 10.1037/qup0000096 · DOI
  10. Festinger, 1957, A theory of cognitive dissonance, 10.1515/9781503620766 · DOI · Paper page
  11. Gazzaniga, 2006, Cognitive Neuroscience. The biology of the mind
  12. Hoffman, 2009, The dynamics of trust in cyberdomains, IEEE Intell. Syst., 24, 5, 10.1109/mis.2009.124 · DOI · Paper page
  13. Johns, 1996, A concept analysis of trust, J. Adv. Nurs., 24, 76, 10.1046/j.1365-2648.1996.16310.x · DOI · Paper page
  14. Luhmann, 2018, Trust and power
  15. Monroe, 2018, AI, explain yourself, Commun. ACM., 61, 11, 10.1145/3276742 · DOI
  16. Mosier, 1992, Electronic checklists: implications for decision making, 7
  17. Muir, 1987, Trust between humans and machines, and the design of decision aids, Int. J. Man-Machine Stud., 27, 527, 10.1016/s0020-7373(87)80013-5 · DOI · Paper page
  18. Parasuraman, 2008, Situation awareness, mental workload, and trust in automation: viable, empirically supported cognitive engineering constructs, J. Cogn. Eng. Decis. making., 2, 140, 10.1518/155534308x284417 · DOI · Paper page
  19. Percival, 2014, Confirmation versus falsificationism, encyclopedia Clin. Psychol., 10.1002/9781118625392.wbecp388 · doi:10.1002/9781118625392.wbecp388
  20. Rousseau, 1998, Not so different after all: a cross-discipline view of trust, Acad. Manag. Review Amr., 23, 393, 10.5465/amr.1998.926617 · DOI · Paper page
  21. Sanchez, 2006, Factors that affect trust and reliance on an automated aid
  22. Sheridan, 2002, Humans and automation: system design and research issues
  23. Slater, 2006, A virtual reprise of the Stanley Milgram obedience experiments, PloS one, 1, e39, 10.1371/journal.pone.0000039 · DOI · Paper page
  24. Sparaco, 1995, Airbus seeks to keep pilot, new technology in harmony, Aviation Week Space Tech., 142, 62
  25. Syrdal, 2007, Looking good? Appearance preferences and robot personality inferences at zero acquaintance, 86
  26. Wagner, 2018, Overtrust in the robotic age, Commun. ACM., 61, 22, 10.1145/3241365 · DOI · Paper page
  27. Wagner, 2009, The role of trust and relationships in human-robot social interaction
  28. Yagoda, 2012, You want me to trust a ROBOT? The development of a human–robot interaction trust scaleYou Want Me to Trust a ROBOT? The Development of a Human-Robot Interaction Trust Scale, Int J Soc Robotics., 4, 235, 10.1007/s12369-012-0144-0 · DOI · Paper page
  29. Zuboff, 1989, The age of the smart machine: the future of work and power

Cited by

See all 69

Metadata

Authors
Ullrich, D., Butz, A., & Diefenbach, S.
Venue
Frontiers in Robotics and AI
Published
2021-04-13
Volume
8
Pages
554578-554578
ISSN
2296-9144 (ISSN-L)
OpenAlex
W3129716156
Cited by
69
References
19
FWCI
8.03
Type
journal article
Language
en
Open access
gold

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.