Moral competence in social robots
The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
We propose that any robots that collaborate with, look after, or help humans-in short, social robots-must have moral competence. But what does moral competence consist of? We offer a framework for moral competence that attempts to be comprehensive in capturing capacities that make humans morally competent and that therefore represent candidates for a morally competent robot. We posit that human moral competence consists of four broad components: (1) A system of norms and the language and concepts needed to communicate about these norms; (2) moral cognition and affect; (3) moral decision making and action; and (4) moral communication. We sketch what we know and don't know about these four elements of moral competence in humans and, for each component, ask how we could equip an artificial agent with these capacities.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The emotional dog and its rational tail: A social intuitionist approach to moral judgment.
Haidt, J. (2001) · Psychological Review · doi:10.1037/0033-295x.108.4.814 · Cited by 8,048
Wimmer, H. (1983) · Cognition · doi:10.1016/0010-0277(83)90004-5 · Cited by 6,305
The Neural Bases of Cognitive Conflict and Control in Moral Judgment
Greene, J. D., Nystrom, L. E., Engell, A. D., Darley, J. M., & Cohen, J. D. (2004) · Neuron · doi:10.1016/j.neuron.2004.09.027 · Cited by 2,620 · cited 1×
“A popular theoretical view of such situations is that initial affective responses can be overridden by deliberation [48].”
BackgroundEmotion, regulation, and moral development
Eisenberg, N. (2000) · Annual Review of Psychology · doi:10.1146/annurev.psych.51.1.665 · Cited by 2,422
Malle, B. F., Guglielmo, S., & Monroe, A. E. (2014) · Psychological Inquiry · doi:10.1080/1047840x.2014.877340 · Cited by 632
Self-citationThe moral emotions: A social–functionalist account of anger, disgust, and contempt.
Hutcherson, C. A., & Gross, J. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0022408 · Cited by 609
Psychopaths know right from wrong but don’t care
Cima, M., Tonnaer, F., & Hauser, M. D. (2010) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsp051 · Cited by 362
Prolegomena to any future artificial moral agent
Allen, C., Varner, G., & Zinser, J. (2000) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/09528130050111428 · Cited by 326
A Psychophysiological Investigation of Moral Judgment after Ventromedial Prefrontal Damage
Moretto, G., Làdavas, E., Mattioli, F., & di Pellegrino, G. (2010) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21367 · Cited by 182
Patterns of Moral Judgment Derive From Nonmoral Psychological Representations
Cushman, F., & Young, L. (2011) · Cognitive Science · doi:10.1111/j.1551-6709.2010.01167.x · Cited by 179 · cited 1×
“The core elements here are causal and counterfactual reasoning and social cognition, and that is why a number of researchers suggest that moral cognition is no unique “module” or “engine” but derives from ordinary cognition [39], [40], but reasoning within the context of norms.”
Background
Cited by
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550
Building Machines That Learn and Think Like People
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2016) · arXiv · doi:10.48550/arxiv.1604.00289 · Cited by 2,550
People are averse to machines making moral decisions.
Bigman, Y. E., & Gray, K. (2018) · Cognition · doi:10.1016/j.cognition.2018.08.003 · Cited by 738
“Consistent with this idea, many argue that—normatively speaking—machines need agency in order to make moral decisions (Floridi & Sanders, 2004; Hellström, 2013; Malle & Scheutz, 2014; Steinert, 2014; Wallach & Allen, 2009; Wallach, Franklin, & Allen, 2010).”
InfluentialSacrifice One For the Good of Many?: People Apply Different Moral Norms to Human and Robot Agents
Malle, B. F., Scheutz, M., Arnold, T., Voiklis, J., & Cusimano, C. (2015) · Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2696454.2696458 · Cited by 246
Self-citationMalle, B. F., Scheutz, M., Forlizzi, J., & Voiklis, J. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451743 · Cited by 107
“Some argue that we should refrain from building and deploying any machines that could harm humans [1]; others argue that stopping the deployment of increasingly autonomous robots is not realistic, and we therefore need to equip robots with moral competence to avoid unnecessary harm to humans [2], [3].”
BackgroundSelf-citation
Metadata
- Authors
- Malle, B. F., & Scheutz, M.
- Venue
- 2014 IEEE International Symposium on Ethics in Science, Technology and Engineering
- Publisher
- IEEE
- Published
- 2014-05
- Pages
- 1-6
- OpenAlex
- W2007749887
- Cited by
- 97
- References
- 12
- FWCI
- 11.06
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Moral competence in social robots. https://app.cochat.ai/papers/10.1109/ethics.2014.6893446
- The paper
- Malle, B. F., & Scheutz, M. (2014). Moral competence in social robots. 2014 IEEE International Symposium on Ethics in Science, Technology and Engineering. https://doi.org/10.1109/ethics.2014.6893446
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.