The logic of universalization guides moral judgment
Open accessAbstract
Significance: Humans have several different ways to decide whether an action is wrong: We might ask whether it causes harm or whether it breaks a rule. Moral psychology attempts to understand the mechanisms that underlie moral judgments. Inspired by theories of “universalization” in moral philosophy, we describe a mechanism that is complementary to existing approaches, demonstrate it in both adults and children, and formalize a precise account of its cognitive mechanisms. Specifically, we show that, when making judgments in novel circumstances, people adopt moral rules that would lead to better consequences if (hypothetically) universalized. Universalization may play a key role in allowing people to construct new moral rules when confronting social dilemmas such as voting and environmental stewardship.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prompting agents to consider the universalization of their action significantly improves survival time.
“we find that prompting agents to consider the universalization of their action [43], a process used by people when making moral judgments in social dilemmas, significantly improves survival time.”
From Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents (2024) · cites this paper as [43]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Collective action and the evolution of social norms.
Ostrom, E. (2000) · Journal of Economic Perspectives · doi:10.1257/jep.14.3.137 · Cited by 4,195 · see citing works
The Constructive, Destructive, and Reconstructive Power of Social Norms
Schultz, P. W., Nolan, J. M., Cialdini, R. B., Goldstein, N. J., & Griskevicius, V. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.01917.x · Cited by 3,643 · see citing works
Human-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817 · see citing works
Self-citationHow to grow a mind: Statistics, structure, and abstraction
Tenenbaum, J. B., Kemp, C., Griffiths, T. L., & Goodman, N. D. (2011) · Science · doi:10.1126/science.1192788 · Cited by 1,815 · see citing works
Self-citationCultural group selection, coevolutionary processes and large-scale cooperation
Henrich, J. (2004) · Journal of Economic Behavior & Organization · doi:10.1016/s0167-2681(03)00094-5 · Cited by 1,096 · cited 1× · see citing works
“Thus, biologically or culturally evolved moral norms tend toward payoff-maximizing rules that everyone would agree to (49).”
Inference of trustworthiness from intuitive moral judgments.
Everett, J. A. C., Pizarro, D. A., & Crockett, M. J. (2016) · Journal of Experimental Psychology: General · doi:10.1037/xge0000165 · Cited by 317 · see citing works
Flagged first, then by citations
Research works printed but not linked: 38
- [3]Ledyard J. O., “ Public goods: A survey of experimental research” in Handbook of Experimental Economics, Kagel J., Roth A., Eds. (Princeton University Press, Princeton, NJ, 1995) pp. 111– 194.
- [5]Greene J. D., Moral Tribes: Emotion, Reason, and the Gap between Us and Them (Penguin, 2014).
- [6]Cushman F., Action, outcome, and value: A dual-system framework for morality. Pers. Soc. Psychol. Rev. 17, 273– 292 (2013). 23861355
- [7]Kleiman-Weiner M., Gerstenberg T., Levine S., Tenenbaum J. B., “ Inference of intention and permissibility in moral decision making” in Proceedings of the 37th Annual Conference of the Cognitive Science Society, Dale R. et al., Eds. (Cognitive Science Society, 2015), pp. 1123– 1128.
- [8]Pellizzoni S., Siegal M., Surian L., The contact principle and utilitarian moral judgments in young children. Dev. Sci. 13, 265– 270 (2010). 20136922
- [9]Levine S., Leslie A., Preschoolers use the means-ends structure of intention to make moral judgments. 10.31234/osf.io/np9a5 (16 September 2020). · DOI
- [10]Haidt J., The emotional dog and its rational tail: A social intuitionist approach to moral judgment. Psychol. Rev. 108, 814– 834 (2001). 11699120
- [12]Nichols S., Mallon R., Moral dilemmas and moral rules. Cognition 100, 530– 542 (2006). 16157325
- [13]Mikhail J., Elements of Moral Cognition: Rawls’ Linguistic Analogy and the Cognitive Science of Moral and Legal Judgment (Cambridge University Press, 2011).
- [14]Turiel E., The Development of Social Knowledge: Morality and Convention (Cambridge University Press, 1983).
- [15]Kant I., Groundwork for the Metaphysics of Morals, Gregor M., Ed. (Cambridge University Press, 1785).
- [17]Hare R. M., Hare R. M., Hare R. M. H., Hare R. M., Moral Thinking: Its Levels, Method, and Point (Oxford Clarendon Press, Oxford University Press, New York, NY, 1981).
- [19]Baumard N., André J. B., Sperber D., A mutualistic approach to morality: The evolution of fairness by partner choice. Behav. Brain Sci. 36, 59– 78 (2013). 23445574
- [20]Misyak J. B., Melkonyan T., Zeitoun H., Chater N., Unwritten rules: Virtual bargaining underpins social interaction, culture, and society. Trends Cognit. Sci. 18, 512– 519 (2014). 25073460
- [21]Parfit D., On What Matters (Oxford University Press, 2011), Vol. 1.
- [22]Scanlon T., What We Owe to Each Other (Harvard University Press, 1998).
- [23]Rawls J., A Theory of Justice (Harvard University Press, 1971).
- [24]Habermas J., Moral Consciousness and Communicative Action (MIT Press, 1990).
- [25]Pogge T., “ The categorical imperative” in Kant’s Groundwork of the Metaphysics of Morals: Critical Essays, Guyer P., Ed. (Rowman & Littlefield, Lanham, MD, 1998), pp. 189– 213.
- [29]Sutton R. S., Barto A. G., Reinforcement Learning: An Introduction (MIT Press, 2018).
- [30]McFadden D., “ Conditional logit analysis of qualitative choice behavior” in Frontiers in Econometrics, Zarembka P., Ed. (University of California, Berkeley, 1973), pp. 105– 142.
- [31]Bicchieri C., The Grammar of Society: The Nature and Dynamics of Social Norms (Cambridge University Press, 2005).
- [32]Nagel T., Equality and Partiality (Oxford University Press, 1995).
- [33]Rawls J., Herman B., Lectures on the History of Moral Philosophy (Harvard University Press, 2009).
- [35]Jones M., Love B. C., Bayesian fundamentalism or enlightenment? On the explanatory status and theoretical contributions of bayesian models of cognition. Behav. Brain Sci. 34, 169– 231 (2011). 21864419
- [36]Van de Vondervoort J. W., Hamlin J. K., The early emergence of sociomoral evaluation: Infants prefer prosocial others. Current Opinion Psychology 20, 77– 81 (2018). 28858770
- [37]Killen M., Smetana J. G., “ Origins and development of morality” in Handbook of Child Psychology and Developmental Science, Lerner R. M., Ed. (Wiley, 2015), pp. 1– 49.
- [38]McAuliffe K., Blake P. R., Steinbeis N., Warneken F., The developmental foundations of human fairness. Nature Human Behaviour 1, 42 (2017).
- [39]Ting F., Dawkins M., Stavans M., Baillargeon R., Principles and concepts in early moral cognition, in The Social Brain: A Developmental Perspective, Decety J., Ed. (MIT Press, 2020).
- [40]Rand D. G., Social heuristics shape intuitive cooperation. Nat. Commun. 5, 3677 (2014). 24751464
- [44]Lieder F., Griffiths T. L., Strategy selection as rational metareasoning. Psychol. Rev. 124, 762– 794 (2017). 29106268
- [45]Harsanyi J. C., Selten R, A General Theory of Equilibrium Selection in Games (MIT Press, 1988), Vol. 1.
- [47]Smith J. M., Evolution and the Theory of Games (Cambridge University Press, 1982).
- [48]Boyd R., Richerson P. J., Group selection among alternative evolutionarily stable strategies. J. Theor. Biol. 145, 331– 342 (1990). 2232821
- [50]Gershman S., Goodman N., “ Amortized inference in probabilistic reasoning” in Proceedings of the Annual Meeting of the Cognitive Science Society, Iricinschi C., Ed. (2014), Vol. 36, Abstract 34j1h7k5.
- [52]Wagenmakers E. J., Lee M., Lodewyckx T., Iverson G. J., “ Bayesian versus frequentist inference” in Bayesian Evaluation of Informative Hypotheses, Hoijtink H., Klugkist I., Boelen P., Eds. (Springer, 2008), pp. 181– 207.
- [53]Wagenmakers E. J., A practical solution to the pervasive problems of p values. Psychon. Bull. Rev. 14, 779– 804 (2007). 18087943
- [54]Rouder J. N., Optional stopping: No problem for bayesians. Psychon. Bull. Rev. 21, 301– 308 (2014). 24659049
Cited by
Sorted by citations
Large pre-trained language models contain human-like biases of what is right and wrong to do
Schramowski, P., Turan, C., Andersen, N., Rothkopf, C. A., & Kersting, K. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00458-8 · Cited by 291 · see citing works
Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents
Piatti, G., Jin, Z., Kleiman-Weiner, M., Schölkopf, B., Sachan, M., & Mihalcea, R. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-3548 · Cited by 14 · see citing works
“The first is inspired by empirical work on understanding the norms that emerge in communities of fishermen that prevent overfishing [25, 43, 56].”
InfluentialSelf-citation
- Authors
- Levine, S., Kleiman-Weiner, M., Schulz, L., Tenenbaum, J., & Cushman, F.
- Affiliations
- Harvard University (US)
- Department of Psychology, Harvard University, Cambridge, MA 02138;
- Massachusetts Institute of Technology (US)
- Corresponding author
- Levine, Sydney
- Tenenbaum, Joshua
- Cushman, Fiery
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2020-10-02
- Volume
- 117
- Issue
- 42
- Pages
- 26158-26169
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC7584905
- OpenAlex
- W3089414779
- Cited by
- 88
- References
- 6
- Citations by year
- 2026: 29 · 2025: 9 · 2024: 17 · 2023: 18 · 2022: 11 · 2021: 3 · 2020: 1
- FWCI
- 4.40
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Funders
- John Templeton Foundation (Grant 61061)
- Center for Brain Minds and Machines (NSF STC award CCF-1231216)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Psychology of Moral and Emotional Judgment
- Social Sciences › Decision Sciences › Information Systems and Management › Ethics in Business and Education
- Social Sciences › Arts and Humanities › Philosophy › Philosophical Ethics and Theory
- Keywords
- moral judgment, universalization, moral reasoning, moral intuitions, children's moral judgment, rule-based reasoning, moral wrongness
- MeSH terms
- Adult, Child, Child, Preschool, Decision Making, Humans, Judgment, Middle Aged, Models, Psychological, Morals, Social Perception, Moral Development
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review