Trust and Social Engineering in Human Robot Interaction: Will a Robot Make You Disclose Sensitive Information, Conform to Its Recommendations or Gamble?
The DOI matches its Crossref record; 50 of the 53 references it cites were found in a registry or an index.
Abstract
Robots such as information security and overtrust in them are gaining increasing relevance. This research aims at giving an insight into how trust toward robots could be exploited for the purpose of social engineering. Drawing on Mitnick's model, a well-known social engineering framework, an interactive scenario with the humanoid robot iCub was designed to emulate a social engineering attack. At first, iCub attempted to collect the kind of personal information usually gathered by social engineers by asking a series of private questions. Then, the robot tried to develop trust and rapport with participants by offering reliable clues during a treasure hunt game. At the end of the treasure hunt, the robot tried to exploit the gained trust in order to make participants gamble the money they won. The results show that people tend to build rapport with and trust toward the robot, resulting in the disclosure of sensitive information, conformation to its suggestions and gambling.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Cyber criminals choose these attacks when there is no way to hack a system with no technical vulnerabilities [3].”
From Social Engineering Attacks: A Survey (2019) · cites this paper as [3]
Citations
Cites
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 2×
“, as “the attitude that an agent will help achieve an individual’s goals in a situation characterized by uncertainty and vulnerability” [1] while Hancock et al.”
BackgroundInclusion of Other in the Self Scale and the structure of interpersonal closeness.
Aron, A., Aron, E. N., & Smollan, D. (1992) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.63.4.596 · Cited by 5,157 · cited 1×
“In the post-experiment phase, few more questionnaires were given to the participants: (i) NASA-TLX workload assessment [45]; (ii) several subscales regarding trust, perceived information quality, altruism and engagement, adapted to HRI scenarios [46]; (iii) inclusion of other in self scale (IOS) [47].”
MethodHumans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · cited 1×
“This is a sign of overtrust, as stated in [1], that is a poor calibration between the person’s trust and the system’s capabilities; more precisely, overtrust is described as how a system could be inappropriately relied upon, even compromising safety and profitability [13].”
BackgroundBartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009) · International Journal of Social Robotics · doi:10.1007/s12369-008-0001-3 · Cited by 3,193 · cited 1×
“At this point, participants had to watch a descriptive video of iCub performing several activities and then answer the following questions regarding their perception of iCub: (i) questions to measure rapport with iCub, inspired from [41]; (ii) dimensions of mind perception regarding iCub [42]; (iii) trust in robots’ a…”
BackgroundGray, H. M., Gray, K., & Wegner, D. M. (2007) · Science · doi:10.1126/science.1134475 · Cited by 2,342
A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., & Parasuraman, R. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811417254 · Cited by 2,051 · cited 1×
“Environment factors and robot characteristics such as performance, can affect the development of trust [4].”
BackgroundThe mind in the machine: Anthropomorphism increases trust in an autonomous vehicle
Waytz, A., Heafner, J., & Epley, N. (2014) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2014.01.005 · Cited by 1,439
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · cited 1×
“Additionally, robot’s transparency [5], [6] has been shown to influence trust.”
BackgroundSocially intelligent robots: Dimensions of human-robot interaction
Dautenhahn, K. (2007) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2006.2004 · Cited by 1,210 · cited 1×
“To foster a natural social interaction between the participants and the iCub, the robot was endowed with the ability to exhibit a range of social skills, inspired by [27].”
BackgroundTrust between humans and machines, and the design of decision aids
Muir, B. M. (1987) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(87)80013-5 · Cited by 762
(2001) · Cited by 636
The iCub humanoid robot: An open-systems platform for research in cognitive development
Metta, G., Natale, L., Nori, F., Sandini, G., Vernon, D., Fadiga, L., von Hofsten, C., Rosander, K., Lopes, M., Santos-Victor, J., et al. (2010) · Neural Networks · doi:10.1016/j.neunet.2010.08.010 · Cited by 580
Self-citationThe Art of Deception, Controlling the Human Element of Security
Barlow, J. (2003) · CommonKnowledge Research Repository (Pacific University Oregon) · Cited by 524
Salem, M., Lakatos, G., Amirabdollahian, F., & Dautenhahn, K. (2015) · Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2696454.2696497 · Cited by 494 · cited 2×
“[11], show that participants comply with awkward orders from a robot even when they could result in information leakage or property damage, and also when the robot openly exhibits faulty behavior.”
BackgroundOvertrust of robots in emergency evacuation scenarios
Robinette, P., Li, W., Allen, R., Howard, A. M., & Wagner, A. R. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451740 · Cited by 468 · cited 2×
“[12] demonstrate that participants in a fake fire emergency scenario tend to follow a robot, rather than the emergency signs, even if it shows clear malfunctioning in its navigation.”
BackgroundThe Role of Healthcare Robots for Older People at Home: A Review
Robinson, H., MacDonald, B., & Broadbent, E. (2014) · International Journal of Social Robotics · doi:10.1007/s12369-014-0242-2 · Cited by 459 · cited 1×
“Moreover, it is necessary to study how trust changes over time in order to be able to predict and prevent the risk of overtrust and trust exploitation in important domains such as healthcare, homecare [24] or education [25].”
BackgroundThe hidden dimension : man's use of space in public and private
Hall, E. T. (1969) · Bodley Head eBooks · Cited by 362
Security Engineering: A Guide to Building Dependable Distributed Systems
Anderson, R. (2020) · doi:10.1002/9781119644682 · Cited by 345
Social engineering : the art of human hacking
Hadnagy, C. (2011) · Cited by 322
Ferrari, F., Paladino, M. P., & Jetten, J. (2016) · International Journal of Social Robotics · doi:10.1007/s12369-016-0338-y · Cited by 314
Syrdal, D. S., Dautenhahn, K., Koay, K. L., & Walters, M. L. (2009) · University of Hertfordshire Research Archive (University of Hertfordshire) · Cited by 310
How anthropomorphism affects empathy toward robots
Riek, L. D., Rabinowitch, T.-C., Chakrabarti, B., & Robinson, P. (2009) · Proceedings of the 4th ACM/IEEE international conference on Human robot interaction · doi:10.1145/1514095.1514158 · Cited by 307
The effect of presence on human-robot interaction
Bainbridge, W. A., Hart, J., Kim, E. S., & Scassellati, B. (2008) · RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication · doi:10.1109/roman.2008.4600749 · Cited by 300 · cited 2×
“The trust toward the robot might have been influenced by different factors, such as the creation of a good rapport [17], its physical appearance [53], its behavior [54], and reliability [8], [9].”
BackgroundResultToward Acceptable Domestic Robots: Applying Insights from Social Psychology
Young, J. E., Hawkins, R., Sharlin, E., & Igarashi, T. (2009) · International Journal of Social Robotics · doi:10.1007/s12369-008-0006-y · Cited by 266
Trust calibration within a human-robot team: Comparing automatically generated explanations
Wang, N., Pynadath, D. V., & Hill, S. G. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451741 · Cited by 220
Gaudiello, I., Zibetti, E., Lefort, S., Chetouani, M., & Ivaldi, S. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.03.057 · Cited by 203
Gaining Access with Social Engineering: An Empirical Study of the Threat
Workman, M. D. (2007) · Information Systems Security · doi:10.1080/10658980701788165 · Cited by 189
Hancock, P. A., Billings, D. R., & Schaefer, K. E. (2011) · Ergonomics in Design: The Quarterly of Human Factors Applications · doi:10.1177/1064804611415045 · Cited by 188
Effects of changing reliability on trust of robot systems
Desai, M., Medvedev, M., Vázquez, M., McSheehy, S., Gadea-Omelchenko, S., Bruggeman, C., Steinfeld, A., & Yanco, H. (2012) · Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction · doi:10.1145/2157689.2157702 · Cited by 188
Personal knowledge questions for fallback authentication: security questions in the era of Facebook
Rabkin, A. (2008) · Proceedings of the 4th symposium on Usable privacy and security · doi:10.1145/1408664.1408667 · Cited by 183
Bonaci, T., Herron, J. A., Yusuf, T., Yan, J., Kohno, T., & Chizeck, H. J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.04339 · Cited by 152
It's No Secret. Measuring the Security and Reliability of Authentication via “Secret” Questions
Schechter, S., Brush, A. J. B., & Egelman, S. (2009) · 2009 30th IEEE Symposium on Security and Privacy · doi:10.1109/sp.2009.11 · Cited by 141
A Model of Human-Robot Trust: Theoretical Model Development
Sanders, T., Oleson, K. E., Billings, D. R., Chen, J. Y. C., & Hancock, P. A. (2011) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181311551298 · Cited by 122
Harrison, M. A., & Hall, A. E. (2010) · Journal of Social, Evolutionary, and Cultural Psychology · doi:10.1037/h0099303 · Cited by 102
Sociable robots : the role of presence and task in human-robot interaction
Kidd, C. D. (2003) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 94
Appearance of a Robot Affects the Impact of its Behaviour on Perceived Trustworthiness and Empathy
Złotowski, J., Sumioka, H., Nishio, S., Glas, D. F., Bartneck, C., & Ishiguro, H. (2016) · Paladyn, Journal of Behavioral Robotics · doi:10.1515/pjbr-2016-0005 · Cited by 86
Piggybacking Robots: Human-Robot Overtrust in University Dormitory Security
Booth, S., Tompkin, J., Pfister, H., Waldo, J., Gajos, K., & Nagpal, R. (2017) · Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2909824.3020211 · Cited by 84
Ososky, S., Sanders, T., Jentsch, F., Hancock, P., & Chen, J. Y. C. (2014) · SPIE Proceedings · doi:10.1117/12.2050622 · Cited by 77
Effects of changing reliability on trust of robot systems
Desai, Medvedev, Vázquez, McSheehy, Gadea-Omelchenko, Bruggeman, Steinfeld, & Yanco (2012) · Cited by 72
Ashleigh, M. J., Higgs, M., & Dulewicz, V. (2012) · Human Resource Management Journal · doi:10.1111/1748-8583.12007 · Cited by 63
Will People Keep the Secret of a Humanoid Robot?: Psychological Intimacy in HRI
Kahn, P. H., Kanda, T., Ishiguro, H., Gill, B. T., Shen, S., Gary, H. E., & Ruckert, J. H. (2015) · Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2696454.2696486 · Cited by 55
Risk Attitude Scales: Concepts, Questionnaires, Utilizations
Rohrmann, B. (2008) · Cited by 47
Sandoval, E. B., Brandstetter, J., & Bartneck, C. (2016) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451742 · Cited by 37
Call Center Customer Verification by Query-Directed Passwords
O’Gorman, L., Bagga, A., & Bentley, J. L. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-27809-2_6 · Cited by 23
An Analysis of Risk Assessment Questions Based on Loss-Averse Preferences
Guillemette, M. A., Yao, R., & James, R. N. (2015) · Journal of Financial Counseling and Planning · doi:10.1891/1052-3073.26.1.17 · Cited by 23
Considering the bystander's perspective for indirect human-robot interaction
Tsui, K. M., Desai, M., & Yanco, H. A. (2010) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2010.5453230 · Cited by 20
Robots in education: New trends and challenges from the Japanese market
Basoeki, F., Libera, F. D., Menegatti, E., & Moro, M. (2013) · Padua Research Archive (University of Padova) · Cited by 18
Sandoval, E. B., Brandstetter, J., & Bartneck, C. (2016) · PUB – Publications at Bielefeld University (Bielefeld University) · doi:10.5555/2906831.2906853 · Cited by 17
Considering the bystander's perspective for indirect human-robot interaction
Tsui, K. M., Desai, M., & Yanco, H. A. (2010) · doi:10.5555/1734454.1734506 · Cited by 16
Will You Rely on a Robot to Find a Treasure?
Aroyo, A. M., Rea, F., & Sciutti, A. (2017) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3029798.3038394 · Cited by 13
Self-citation
Research works printed but not linked: 8
- alexander, 2016, Methods for Understanding and Reducing Social Engineering Attacks, SANS Inst, 1, 1
- milano-bicocca, 2015, Versione Italiana dei Big Five Markers di Goldberg Giovanni Battista Flebus
- 10.1518/hfes.46.1.50.30392 · doi:10.1518/hfes.46.1.50.30392 · Paper page
- franceschi-bicchierai, 2017, How this internet of things stuffed animal can be remotely turned into a spy device, Motherboards
- cerrudo, 2017, Hacking Robots Before Skynet, Cybersecurity Insight IOActive Report, 1
- gibbs, 2015, Hackers can hijack Wi-Fi Hello Barbie to spy on your children, The Guardian
- wang, 0, Building trust in a human-robot team with automatically generated explanations, Proc Interservice/Industry Train Simul Edu Conf, 1
- bracco, 2008, Versione Italiana del NASA-TLX
Cited by
Social Engineering Attacks: A Survey
Salahdine, F., & Kaabouch, N. (2019) · Future Internet · doi:10.3390/fi11040089 · Cited by 509
“dark [11].”
Trust in Robots: Challenges and Opportunities
Kok, B. C., & Soh, H. (2020) · Current Robotics Reports · doi:10.1007/s43154-020-00029-y · Cited by 221
A survey on the design and evolution of social robots — Past, present and future
Mahdi, H., Akgun, S. A., Saleh, S., & Dautenhahn, K. (2022) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2022.104193 · Cited by 162
Cyber security of robots: A comprehensive survey
Alessio Botta, B., Rotbei, S., Zinno, S., & Ventre, G. (2023) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2023.200237 · Cited by 133
Tell me more! assessing interactions with social robots from speech
Laban, G., George, J.-N., Morrison, V., & Cross, E. S. (2020) · Paladyn, Journal of Behavioral Robotics · doi:10.1515/pjbr-2021-0011 · Cited by 81
Metadata
- Authors
- Aroyo, A. M., Rea, F., Sandini, G., & Sciutti, A.
- Affiliations
- Italian Institute of Technology (IT)
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2018-10
- Volume
- 3
- Issue
- 4
- Pages
- 3701-3708
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W2883608792
- Cited by
- 85
- References
- 53
- Citations by year
- 2026: 10 · 2025: 10 · 2024: 10 · 2023: 11 · 2022: 12 · 2021: 13 · 2020: 11 · 2019: 6 · 2018: 2
- FWCI
- 10.83
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Robotics and Automation Letters (publishedVersion)
- CINECA IRIS Institutial Research Information System (University of Genoa) (submittedVersion)
- Funders
- European CODEFROR project (PIRSES-2013-612555)
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- human-robot interaction, trust in robots, social engineering, information security, iCub humanoid robot, sensitive information disclosure, conformity, gambling behavior, rapport building, treasure hunt
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.