A Conceptual and Computational Model of Moral Decision Making in Human and Artificial Agents
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
Recently, there has been a resurgence of interest in general, comprehensive models of human cognition. Such models aim to explain higher-order cognitive faculties, such as deliberation and planning. Given a computational representation, the validity of these models can be tested in computer simulations such as software agents or embodied robots. The push to implement computational models of this kind has created the field of artificial general intelligence (AGI). Moral decision making is arguably one of the most challenging tasks for computational approaches to higher-order cognition. The need for increasingly autonomous artificial agents to factor moral considerations into their choices and actions has given rise to another new field of inquiry variously known as Machine Morality, Machine Ethics, Roboethics, or Friendly AI. In this study, we discuss how LIDA, an AGI model of human cognition, can be adapted to model both affective and rational features of moral decision making. Using the LIDA model, we will demonstrate how moral decisions can be made in many domains using the same mechanisms that enable general decision making. Comprehensive models of human cognition typically aim for compatibility with recent research in the cognitive and neural sciences. Global workspace theory, proposed by the neuropsychologist Bernard Baars (1988), is a highly regarded model of human cognition that is currently being computationally instantiated in several software implementations. LIDA (Franklin, Baars, Ramamurthy, & Ventura, 2005) is one such computational implementation. LIDA is both a set of computational tools and an underlying model of human cognition, which provides mechanisms that are capable of explaining how an agent's selection of its next action arises from bottom-up collection of sensory data and top-down processes for making sense of its current situation. We will describe how the LIDA model helps integrate emotions into the human decision-making process, and we will elucidate a process whereby an agent can work through an ethical problem to reach a solution that takes account of ethically relevant factors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Principles of Biomedical Ethics
(2012) · Thieves of Virtue · doi:10.7551/mitpress/9079.003.0009 · Cited by 13,453
Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry
Flavell, J. H. (1979) · American Psychologist · doi:10.1037/0003-066x.34.10.906 · Cited by 11,282 · cited 2×
“32 and regulating cognitive strategies (Flavell, 1979).”
BackgroundReinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382
The Embodied Mind: Cognitive Science and Human Experience
Varela, F. J., Rosch, E., & Thompson, E. (1991) · doi:10.7551/mitpress/6730.001.0001 · Cited by 5,765
Ericsson, K. A., & Kintsch, W. (1995) · Psychological Review · doi:10.1037/0033-295x.102.2.211 · Cited by 3,498
The Neural Bases of Cognitive Conflict and Control in Moral Judgment
Greene, J. D., Nystrom, L. E., Engell, A. D., Darley, J. M., & Cohen, J. D. (2004) · Neuron · doi:10.1016/j.neuron.2004.09.027 · Cited by 2,620
The Embodied Mind: Cognitive Science and Human Experience
Varela, F. J., Thompson, E., & Rosch, E. (2017) · doi:10.7551/mitpress/9780262529365.001.0001 · Cited by 2,449
First Impressions: Making Up Your Mind After a 100-Ms Exposure to a Face
Willis, J., & Todorov, A. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01750.x · Cited by 2,300
Is It an agent, or just a program?: A taxonomy for autonomous agents
Franklin, S., & Graesser, A. (1997) · Lecture Notes in Computer Science · doi:10.1007/bfb0013570 · Cited by 2,286 · cited 1×
“Every autonomous agent (Franklin and Graesser 1997), human, animal, or artificial, must frequently sample (sense) its environment and select an appropriate response (action).”
Self-citation
Cited by
People are averse to machines making moral decisions.
Bigman, Y. E., & Gray, K. (2018) · Cognition · doi:10.1016/j.cognition.2018.08.003 · Cited by 738
“Scholars have also highlighted the importance of experience for machine moral decisionmaking (Allen, Wallach, & Smit, 2006; Coeckelbergh, 2010; Himma, 2009; Malle, 2016; Malle & Scheutz, 2014; Wallach et al., 2010), including the ability to feel moral emotions (Malle & Scheutz, 2014) and having an “inner subjective ex…”
LIDA: A Systems-level Architecture for Cognition, Emotion, and Learning
Franklin, S., Madl, T., D'Mello, S., & Snaider, J. (2014) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2013.2277589 · Cited by 261
Self-citationSacrifice One For the Good of Many?: People Apply Different Moral Norms to Human and Robot Agents
Malle, B. F., Scheutz, M., Arnold, T., Voiklis, J., & Cusimano, C. (2015) · Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2696454.2696458 · Cited by 246
“But even if the architectures can keep pace (with attempts underway seen in [28]–[30]), a critical question is what capacities people want robots to have—what kinds of moral decisions and judgments robots ought to make and what norms they should obey.”
BackgroundArtificial intelligence for aging and longevity research: Recent advances and perspectives
Zhavoronkov, A., Mamoshina, P., Vanhaelen, Q., Scheibye-Knudsen, M., Moskalev, A., & Aliper, A. (2019) · Ageing Research Reviews · doi:10.1016/j.arr.2018.11.003 · Cited by 246
“…e ff orts to develop arti fi cial general intelligence (AGI), also referred to as the sentient AI, and even transfer of the human memory and capabilities into computers, there is no proof of concept demonstrating the feasibility of any of these approaches (Wallach et al., 2010; Deca and Koene, 2014).”
Artificial Moral Agents: A Survey of the Current Status
Cervantes, J.-A., López, S., Rodríguez, L.-F., Cervantes, S., Cervantes, F., & Ramos, F. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-019-00151-x · Cited by 151
“Soar (Laird 2008; Laird et al. 2012), ACT-R (Borst and Anderson 2015; Trafton et al. 2013), LIDA (Wal-lach et al. 2010), and Icarus (Choi and Langley 2018) are examples of computational models based on human cognition.”
BackgroundMethodInfluential
Metadata
- Authors
- Wallach, W., Franklin, S., & Allen, C.
- Venue
- Topics in Cognitive Science
- Publisher
- Wiley
- Published
- 2010-05-13
- Volume
- 2
- Issue
- 3
- Pages
- 454-485
- ISSN
- 1756-8757 (ISSN-L)
- OpenAlex
- W2077959903
- Cited by
- 127
- References
- 24
- FWCI
- 5.44
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Conceptual and Computational Model of Moral Decision Making in Human and Artificial Agents. https://app.cochat.ai/papers/10.1111/j.1756-8765.2010.01095.x
- The paper
- Wallach, W., Franklin, S., & Allen, C. (2010). A Conceptual and Computational Model of Moral Decision Making in Human and Artificial Agents. Topics in Cognitive Science. https://doi.org/10.1111/j.1756-8765.2010.01095.x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.