Sacrifice One For the Good of Many?: People Apply Different Moral Norms to Human and Robot Agents
Abstract
Moral norms play an essential role in regulating human interaction. With the growing sophistication and proliferation of robots, it is important to understand how ordinary people apply moral norms to robot agents and make moral judgments about their behavior. We report the first comparison of people's moral judgments (of permissibility, wrongness, and blame) about human and robot agents. Two online experiments (total N = 316) found that robots, compared with human agents, were more strongly expected to take an action that sacrifices one person for the good of many (a "utilitarian" choice), and they were blamed more than their human counterparts when they did not make that choice. Though the utilitarian sacrifice was generally seen as permissible for human agents, they were blamed more for choosing this option than for doing nothing. These results provide a first step toward a new field of Moral HRI, which is well placed to help guide the design of social robots.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
For robot failures, humans blame robots more for failures of inaction than for failures of action.
“However, Malle et al. have shown that for robot failures, there is an asymmetry of blame-that humans blame robots more for failures of inaction than of action [53,54].”
From Taxonomy of Trust-Relevant Failures and Mitigation Strategies (2020) · cites this paper as [53]
Humans often anticipate more utilitarian behaviour from machines.
“Remarkably, humans often anticipate more utilitarian behaviour from them 103”
From Machine culture (2023) · cites this paper as [103]
The citation lists haven’t been fetched yet
References 51 · Cited by 246
- Authors
- Malle, B. F., Scheutz, M., Arnold, T., Voiklis, J., & Cusimano, C.
- Affiliations
- Brown University (US)
- Tufts University (US)
- Harvard University (US)
- Venue
- Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction
- Publisher
- ACM
- Published
- 2015-03-02
- Pages
- 117-124
- OpenAlex
- W2135226325
- Cited by
- 246
- References
- 51
- Citations by year
- 2026: 19 · 2025: 27 · 2024: 20 · 2023: 31 · 2022: 14 · 2021: 34 · 2020: 20 · 2019: 25 · 2018: 19 · 2017: 15 · 2016: 15 · 2015: 7
- FWCI
- 31.10
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/IEEE International Conference on Human-Robot Interaction (HRI) (publishedVersion)
- Funders
- Office of Naval Research (N00014-14-l-0144)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Psychology of Moral and Emotional Judgment
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Social Sciences › Arts and Humanities › Philosophy › Philosophical Ethics and Theory
- Keywords
- moral judgment, human-robot interaction, moral norms, utilitarianism, sacrifice, blame, moral permissibility, social robots, robot ethics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
