CoChat

The Development of Overtrust: An Empirical Simulation and Psychological Analysis in the Context of Human–Robot Interaction

Open access
Ullrich, Daniel, Butz, Andreas, Diefenbach, Sarah · 2021 · Frontiers in Robotics and AI
Cited by 69

The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.

Abstract

With impressive developments in human–robot interaction it may seem that technology can do anything. Especially in the domain of social robots which suggest to be much more than programmed machines because of their anthropomorphic shape, people may overtrust the robot's actual capabilities and its reliability. This presents a serious problem, especially when personal well-being might be at stake. Hence, insights about the development and influencing factors of overtrust in robots may form an important basis for countermeasures and sensible design decisions. An empirical study [N= 110] explored the development of overtrust using the example of a pet feeding robot. A 2 × 2 experimental design and repeated measurements contrasted the effect of one's own experience, skill demonstration, and reputation through experience reports of others. The experiment was realized in a video environment where the participants had to imagine they were going on a four-week safari trip and leaving their beloved cat at home, making use of a pet feeding robot. Every day, the participants had to make a choice: go to a day safari without calling options (risk and reward) or make a boring car trip to another village to check if the feeding was successful and activate an emergency call if not (safe and no reward). In parallel to cases of overtrust in other domains (e.g., autopilot), the feeding robot performed flawlessly most of the time until in the fourth week; it performed faultily on three consecutive days, resulting in the cat's death if the participants had decided to go for the day safari on these days. As expected, with repeated positive experience about the robot's reliability on feeding the cat, trust levels rapidly increased and the number of control calls decreased. Compared to one's own experience, skill demonstration and reputation were largely neglected or only had a temporary effect. We integrate these findings in a conceptual model of (over)trust over time and connect these to related psychological concepts such as positivism, instant rewards, inappropriate generalization, wishful thinking, dissonance theory, and social concepts from human–human interaction. Limitations of the present study as well as implications for robot design and future research are discussed.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/frobt.2021.554578

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

  • Trust in Automation: Designing for Appropriate Reliance

    Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 5×

    “Accordingly, Lee and See (2004), p. 51) de fi ne trust as "the attitude that an agent will help achieve an individual ’ s goals in a situation characterized by uncertainty and vulnerability."”
    BackgroundInfluential
  • Humans and automation: Use, misuse, disuse, abuse

    Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · cited 1×

    “…the de fi nition of automation as "the execution by a machine agent (usually a computer) of a function that was previously carried out by a human" (Parasuraman and Riley, 1997, p. 231), robots expand the fi eld by perception and intelligence and other important factors (Feil-Seifer and Mataric,…”
    Background
  • A new scale for the measurement of interpersonal trust1

    Rotter, J. B. (1967) · Journal of Personality · doi:10.1111/j.1467-6494.1967.tb01454.x · Cited by 3,503

  • Trust in close relationships.

    Rempel, J. K., Holmes, J. G., & Zanna, M. P. (1985) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.49.1.95 · Cited by 3,146 · cited 1×

    “A review of trust de fi nitions in general (e.g., Rotter, 1967; Barber, 1983; Rempel et al., 1985; Luhmann, 2018) highlights the multidimensionality of the concept, each focusing on different aspects of people ’ s everyday usage of trust.”
    Background
  • The role of the basal ganglia in habit formation

    Yin, H. H., & Knowlton, B. J. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1919 · Cited by 2,402

  • Computers are social actors

    Nass, C., Steuer, J., & Tauber, E. R. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191703 · Cited by 2,148 · cited 1×

    “Related to the discussion of robots and computers as social actors, as it has already started in the 90 ’ s (Nass et al., 1994; Lee and Nass, 2003), is the factor of anthropomorphism.”
  • A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction

    Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., & Parasuraman, R. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811417254 · Cited by 2,051 · cited 3×

    “In a literature review on trust in the domain of robots (Hancock et al., 2011), one of the most dominant in fl uencing factors turned out to be reputation in the sense of knowledge about the robot ’ s reliance (Bagheri and Jamieson, 2004) or knowledge about the robot ’ past performance (Lee and See,…”
    BackgroundInfluential
  • Foundations for an Empirically Determined Scale of Trust in Automated Systems

    Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000) · International Journal of Cognitive Ergonomics · doi:10.1207/s15327566ijce0401_04 · Cited by 1,799 · cited 1×

    “To some degree, trust in humans, automation, and robots are based on similar fundamental characteristics such as reliability, predictability, and ability (Jian et al., 2000).”
    Background
  • Learning and Memory Functions of the Basal Ganglia

    Packard, M. G., & Knowlton, B. J. (2002) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.25.112701.142937 · Cited by 1,717 · cited 1×

    “…on rather vague and intuitive associations based on unconscious, non-declarative memory systems, such as in the case of perceptual learning (e.g., Packard and Knowlton, 2002; Gazzaniga et al., 2006; Yin and Knowlton, 2006) and the improved abilities of sensory systems to respond to stimuli…”
    Background
  • Justification effects on consumer choice of hedonic and utilitarian goods.

    Okada, E. M. (2005) · Journal of Marketing Research · doi:10.1509/jmkr.42.1.43.56889 · Cited by 1,069 · cited 1×

    “First, it is well known from consumer choice that people have a natural preference for hedonic, experiential options (here: the safari) over pragmatic options (here: the control call), especially if they can fi nd a reason to justify their choice (e.g., Böhm and P fi ster, 1996; Okada, 2005).”
    Background

Show all 18 references

Research works printed but not linked: 30

  1. Barber, 1983, The logic and limits of trust
  2. Bartneck, 2009, Does the design of a robot influence its animacy and perceived intelligence?, Int. J. Soc. Robotics., 1, 195, 10.1007/s12369-009-0013-7 · DOI · Paper page
  3. Böhm, 1996, Instrumental or emotional evaluations: what determines preferences?, Acta Psychol (Amst)., 93, 135, 10.1016/0001-6918(96)00017-0 · DOI
  4. Carlson, 2014, Identifying factors that influence trust in automated cars and medical diagnosis systems
  5. Cathcart, 1997, The media equation: how people treat computers, television, and new media like real people and places, 305
  6. Chen, 2010, Supervisory control of unmanned vehicles
  7. Coeckelbergh, 2012, Can we trust robots?, Ethics Inf. Technol., 14, 53, 10.1007/s10676-011-9279-1 · DOI · Paper page
  8. Darling, 2012, Extending legal rights to social robots, 10.2139/ssrn.2044797 · DOI
  9. Diefenbach, 2019, Combining model-based analysis with phenomenological insight: a case study on hedonic product quality, Qual. Psychol., 6, 3, 10.1037/qup0000096 · DOI
  10. Festinger, 1957, A theory of cognitive dissonance, 10.1515/9781503620766 · DOI · Paper page
  11. Gazzaniga, 2006, Cognitive Neuroscience. The biology of the mind
  12. Hoffman, 2009, The dynamics of trust in cyberdomains, IEEE Intell. Syst., 24, 5, 10.1109/mis.2009.124 · DOI · Paper page
  13. Johns, 1996, A concept analysis of trust, J. Adv. Nurs., 24, 76, 10.1046/j.1365-2648.1996.16310.x · DOI · Paper page
  14. Luhmann, 2018, Trust and power
  15. Monroe, 2018, AI, explain yourself, Commun. ACM., 61, 11, 10.1145/3276742 · DOI
  16. Moorman, 1993, Factors affecting trust in market research relationships, J. mark., 57, 81, 10.2307/1252059 · DOI · Paper page
  17. Mosier, 1992, Electronic checklists: implications for decision making, 7
  18. Muir, 1987, Trust between humans and machines, and the design of decision aids, Int. J. Man-Machine Stud., 27, 527, 10.1016/s0020-7373(87)80013-5 · DOI · Paper page
  19. Parasuraman, 2008, Situation awareness, mental workload, and trust in automation: viable, empirically supported cognitive engineering constructs, J. Cogn. Eng. Decis. making., 2, 140, 10.1518/155534308x284417 · DOI · Paper page
  20. Percival, 2014, Confirmation versus falsificationism, encyclopedia Clin. Psychol., 10.1002/9781118625392.wbecp388 · doi:10.1002/9781118625392.wbecp388
  21. Rousseau, 1998, Not so different after all: a cross-discipline view of trust, Acad. Manag. Review Amr., 23, 393, 10.5465/amr.1998.926617 · DOI · Paper page
  22. Sanchez, 2006, Factors that affect trust and reliance on an automated aid
  23. Sheridan, 2002, Humans and automation: system design and research issues
  24. Slater, 2006, A virtual reprise of the Stanley Milgram obedience experiments, PloS one, 1, e39, 10.1371/journal.pone.0000039 · DOI · Paper page
  25. Sparaco, 1995, Airbus seeks to keep pilot, new technology in harmony, Aviation Week Space Tech., 142, 62
  26. Syrdal, 2007, Looking good? Appearance preferences and robot personality inferences at zero acquaintance, 86
  27. Wagner, 2018, Overtrust in the robotic age, Commun. ACM., 61, 22, 10.1145/3241365 · DOI · Paper page
  28. Wagner, 2009, The role of trust and relationships in human-robot social interaction
  29. Yagoda, 2012, You want me to trust a ROBOT? The development of a human–robot interaction trust scaleYou Want Me to Trust a ROBOT? The Development of a Human-Robot Interaction Trust Scale, Int J Soc Robotics., 4, 235, 10.1007/s12369-012-0144-0 · DOI · Paper page
  30. Zuboff, 1989, The age of the smart machine: the future of work and power

Cited by

See all 69

Metadata

Authors
Ullrich, D., Butz, A., & Diefenbach, S.
Venue
Frontiers in Robotics and AI
Published
2021-04-13
Volume
8
Pages
554578-554578
ISSN
2296-9144 (ISSN-L)
OpenAlex
W3129716156
Cited by
69
References
18
FWCI
8.03
Type
journal article
Language
en
Open access
gold

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The Development of Overtrust: An Empirical Simulation and Psychological Analysis in the Context of Human–Robot Interaction. https://app.cochat.ai/papers/10.3389/frobt.2021.554578
The paper
Ullrich, D., Butz, A., & Diefenbach, S. (2021). The Development of Overtrust: An Empirical Simulation and Psychological Analysis in the Context of Human–Robot Interaction. Frontiers in Robotics and AI. https://doi.org/10.3389/frobt.2021.554578

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.