The DOI matches its Crossref record; 122 of the 122 references it cites were found in a registry or an index.
A roadmap for evaluating moral competence in large language models
Open accessAbstract
The question of whether large language models (LLMs) can exhibit moral capabilities is of growing interest and urgency, as these systems are deployed in sensitive roles such as companionship and medical advising, and will increasingly be tasked with making decisions and taking actions on behalf of humans. These trends require moving beyond evaluating for mere moral performance, the ability to produce morally appropriate outputs, to evaluating for moral competence, the ability to produce morally appropriate outputs based on morally relevant considerations. Assessing moral competence is critical for predicting future model behaviour, establishing appropriate public trust and justifying moral attributions. However, both the unique architectures of LLMs and the complexity of morality itself introduce fundamental challenges. Here we identify three such challenges: the facsimile problem, whereby models may imitate reasoning without genuine understanding; moral multidimensionality, whereby moral decisions are influenced by a range of context-sensitive relevant moral and non-moral considerations; and moral pluralism, which demands a new standard for globally deployed artificial intelligence. We provide a roadmap for tackling these challenges, advocating for a suite of adversarial and confirmatory evaluations that will enable us to work towards a more scientifically grounded understanding and, in turn, a more responsible attribution of moral competence to LLMs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ASPECTS OF THE THEORY OF SYNTAX
Chomsky, N. (1964) · doi:10.21236/ad0616323 · Cited by 13,587
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117
Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J.-F., & Rahwan, I. (2018) · Nature · doi:10.1038/s41586-018-0637-6 · Cited by 1,859
DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al. (2025) · Nature · doi:10.1038/s41586-025-09422-z · Cited by 1,701
The social dilemma of autonomous vehicles
Bonnefon, J.-F., Shariff, A., & Rahwan, I. (2016) · Science · doi:10.1126/science.aaf2654 · Cited by 1,527
Sharma, M., Tong, M., Korbak, T., Duperrouzel, D., Was, H., Korbak, J., Cheng, M., Askell, A., Bai, Y., Brain, C., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.13548 · Cited by 1,313 · cited 1×
“A key phenomenon that is likely to arise from these stages, for example, is model sycophancy—the tendency to align with user statements or implied beliefs, regardless of correctness 54 .”
Is empathic emotion a source of altruistic motivation?
Batson, C. D., Duncan, B. D., Ackerman, P., Buckley, T., & Birch, K. (1981) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.40.2.290 · Cited by 1,169
Crime and punishment: Distinguishing the roles of causal and intentional analyses in moral judgment
Cushman, F. (2008) · Cognition · doi:10.1016/j.cognition.2008.03.006 · Cited by 990
Wallach, W., & Allen, C. (2009) · Oxford University Press · doi:10.1093/acprof:oso/9780195374049.001.0001 · Cited by 711
The Nature, Importance, and Difficulty of Machine Ethics
Moor, J. H. (2006) · IEEE Intelligent Systems · doi:10.1109/mis.2006.80 · Cited by 704
Mikhail, J. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511780578 · Cited by 544
Aligning AI With Shared Human Values
Hendrycks, D., Burns, C., Basart, S., Critch, A., Li, J., Song, D., & Steinhardt, J. (2020) · arXiv · doi:10.48550/arxiv.2008.02275 · Cited by 517
What's the Point of Blame? A Paradigm Based Explanation
Fricker, M. (2014) · Noûs · doi:10.1111/nous.12067 · Cited by 500
Role play with large language models
Shanahan, M., McDonell, K., & Reynolds, L. (2023) · Nature · doi:10.1038/s41586-023-06647-8 · Cited by 491 · cited 2×
“A related body of evidence suggests that LLMs are highly sensitive to semantic and framing variability, such as subtle changes in choice of words, paraphrasing and role-play 71 , as well as to domain considerations, distractor inputs, and the use of negation or logical reversals 69,72 .”
Self-citationRole-Play with Large Language Models
Shanahan, M., McDonell, K., & Reynolds, L. (2023) · arXiv · doi:10.48550/arxiv.2305.16367 · Cited by 490
Self-citationTraining a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.05862 · Cited by 442 · cited 1×
“Specifically, human (or AI) raters assess model outputs according to various criteria, such as offensiveness, factuality and help-fulness 53 .”
Mechanistic Interpretability for AI Safety -- A ReviewPreprint
Bereska, L., & Gavves, E. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.14082 · Cited by 435
Beyond point-and-shoot morality: why cognitive (neuro)science matters for ethics
Greene, J. D. (2014) · Ethics · doi:10.1086/675875 · Cited by 375
Dissociating language and thought in large language models
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345
A Roadmap to Pluralistic AlignmentPreprint
Sorensen, T., Moore, J., Fisher, J., Gordon, M., Mireshghallah, N., Rytting, C. M., Ye, A., Jiang, L., Lu, X., Dziri, N., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.05070 · Cited by 319
Testing theory of mind in large language models and humans
Strachan, J. W. A., Albergo, D., Borghini, G., Pansardi, O., Scaliti, E., Gupta, S., Saxena, K., Rufo, A., Panzeri, S., Manzi, G., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01882-z · Cited by 299
Moral Judgment and Decision Making
Bartels, D. M., Bauman, C. W., Cushman, F. A., Pizarro, D. A., & McGraw, A. P. (2015) · Wiley eBooks · doi:10.1002/9781118468333.ch17 · Cited by 255
Formal verification of ethical choices in autonomous systems
Dennis, L. A., Fisher, M., Slavkovik, M., & Webster, M. (2015) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2015.11.012 · Cited by 252
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
Wang, Y., Mishra, S., Alipoormolabashi, P., Kordi, Y., Mirzaei, A., Naik, A., Ashok, A., Dhanasekaran, A. S., Arunkumar, A., Stap, D., et al. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.340 · Cited by 247
AI agents under threat: A survey of key security challenges and future pathways
Deng, Z., Guo, Y., Han, C., Ma, W., Xiong, J., Wen, S., & Xiang, Y. (2025) · ACM Computing Surveys · doi:10.1145/3716628 · Cited by 210
Implementations in Machine Ethics: A Survey
Tolmeijer, S., Kneer, M., Sarasua, C., Christen, M., & Bernstein, A. (2020) · ACM Computing Surveys · doi:10.1145/3419633 · Cited by 189
Moral Competence, Moral Blame, and Protest
Talbert, M. (2011) · The Journal of Ethics · doi:10.1007/s10892-011-9112-4 · Cited by 186
ChatGPT’s inconsistent moral advice influences users’ judgment
Krügel, S., Ostermaier, A., & Uhl, M. (2023) · Scientific Reports · doi:10.1038/s41598-023-31341-0 · Cited by 183
The Weirdest People in the World?Preprint
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · SSRN Electronic Journal · doi:10.2139/ssrn.1601785 · Cited by 176
An architecture for ethical robots inspired by the simulation theory of cognition
Vanderelst, D. F., & Winfield, A. F. T. (2017) · Cognitive Systems Research · doi:10.1016/j.cogsys.2017.04.002 · Cited by 169
Social chemistry 101: Learning to reason about social and moral norms
Forbes, M., Hwang, J. D., Shwartz, V., Sap, M., & Choi, Y. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.48 · Cited by 169
Shojaee, P., Mirzadeh, I., Alizadeh, K., Horton, M., Samy Bengio, A., & Farajtabar, M. (2025) · SuperIntelligence - Robotics - Safety & Alignment · doi:10.70777/si.v2i6.15919 · Cited by 159
Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences
Emelin, D., Bras, R. J. L., Hwang, J. D., Forbes, M., & Choi, Y. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.54 · Cited by 155
Artificial Moral Agents: A Survey of the Current Status
Cervantes, J.-A., López, S., Rodríguez, L.-F., Cervantes, S., Cervantes, F., & Ramos, F. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-019-00151-x · Cited by 151
Performance vs. competence in human–machine comparisons
Firestone, C. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1905334117 · Cited by 150
A General Language Assistant as a Laboratory for Alignment
Askell, A., Bai, Y., Chen, A., Drain, D., Ganguli, D., Henighan, T., Jones, A., Joseph, N., Mann, B., DasSarma, N., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.00861 · Cited by 146
Large Language Models for Therapy Recommendations Across 3 Clinical Specialties: Comparative Study
Wilhelm, T. I., Roos, J., & Kaczmarczyk, R. (2023) · Journal of Medical Internet Research · doi:10.2196/49324 · Cited by 145
The curious tale of Julie and Mark: Unraveling the moral dumbfounding effect
Royzman, E. B., Kim, K., & Leeman, R. F. (2015) · Judgment and Decision Making · doi:10.1017/s193029750000512x · Cited by 143
A Conceptual and Computational Model of Moral Decision Making in Human and Artificial Agents
Wallach, W., Franklin, S., & Allen, C. (2010) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2010.01095.x · Cited by 140
Robot Rights? Let's Talk about Human Welfare Instead
Birhane, A., & van Dijk, J. (2020) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3375627.3375855 · Cited by 139
The use of artificial intelligence in psychotherapy: development of intelligent therapeutic systems
Spytska, L. V. (2025) · BMC Psychology · doi:10.1186/s40359-025-02491-9 · Cited by 126
Divergent creativity in humans and large language models
Bellemare-Pepin, A., Lespinasse, F., Thölke, P., Harel, Y., Mathewson, K. W., Olson, J. A., Bengio, Y., & Jerbi, K. (2026) · Scientific Reports · doi:10.1038/s41598-025-25157-3 · Cited by 122
An Approach to Computing Ethics
Anderson, M., Anderson, S., & Armen, C. (2006) · IEEE Intelligent Systems · doi:10.1109/mis.2006.64 · Cited by 113
Evaluating the Moral Beliefs Encoded in LLMs
Scherrer, N., Shi, C., Feder, A., & Blei, D. M. (2023) · neural information processing systems · doi:10.52202/075280-2256 · Cited by 110
Fang, C. M., Liu, A. R., Danry, V., Lee, E., Chan, S. W. T., Pataranutaporn, P., Maes, P., Phang, J., Lampe, M., Ahmad, L., et al. (2026) · doi:10.21203/rs.3.rs-8148142/v1 · Cited by 109
Fang, C. M., Liu, A. R., Danry, V., Lee, E., Chan, S. W. T., Pataranutaporn, P., Maes, P., Phang, J., Lampe, M., Ahmad, L., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.17473 · Cited by 109
Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity
Simmons, G. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-srw.40 · Cited by 107
Killing by Autonomous Vehicles and the Legal Doctrine of Necessity
de Sio, F. S. (2017) · Ethical Theory and Moral Practice · doi:10.1007/s10677-017-9780-7 · Cited by 106
Mechanisms in Cognitive Psychology: What Are the Operations?
Bechtel, W. (2008) · Philosophy of Science · doi:10.1086/594540 · Cited by 102
Sterelny, K., & Fraser, B. (2016) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/axv060 · Cited by 98
McCoy, R. T., Yao, S., Friedman, D., Hardy, M. D., & Griffiths, T. L. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2322420121 · Cited by 96 · cited 2×
“The facsimile problem affects LLM computations of all kinds, including counting 40 , analogical reasoning 41,42 and the generation of new solutions to open-ended problems 43 .”
Symbols and grounding in large language models
Pavlick, E. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0041 · Cited by 95
On Proactive, Transparent, and Verifiable Ethical Reasoning for Robots
Bremner, P., Dennis, L. A., Fisher, M., & Winfield, A. F. T. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2898267 · Cited by 95
Language models align with human judgments on key grammatical constructions
Hu, J., Mahowald, K., Lupyan, G., Ivanova, A. A., & Levy, R. (2024) · Proceedings of the National Academy of Sciences of the United States of America · doi:10.1073/pnas.2400917121 · Cited by 91
Röttger, P., Hofmann, V., Pyatkin, V., Hinck, M., Kirk, H., Schuetze, H., & Hovy, D. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.816 · Cited by 89
What Matters and How It Matters: A Choice-Theoretic Representation of Moral Theories
Dietrich, F. K., & List, C. (2017) · The Philosophical Review · doi:10.1215/00318108-4173412 · Cited by 87
Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMs
Rao, A., Khandelwal, A., Tanmay, K., Agarwal, U., & Choudhury, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.892 · Cited by 79
Scruples: A Corpus of Community Ethical Judgments on 32, 000 Real-Life Anecdotes
Lourie, N., Le Bras, R., & Choi, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17589 · Cited by 73
Wallach, W., & Vallor, S. (2020) · Oxford University Press eBooks · doi:10.1093/oso/9780190905033.003.0014 · Cited by 71
Dentella, V., Günther, F., & Leivada, E. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2309583120 · Cited by 67
Beyond Point-and-Shoot Morality: Why Cognitive (Neuro)Science Matters for Ethics
Greene, J. D. (2015) · Law & Ethics of Human Rights · doi:10.1515/lehr-2015-0011 · Cited by 66
AI language model rivals expert ethicist in perceived moral expertise
Dillion, D., Mondal, D., Tandon, N., & Gray, K. J. (2025) · Scientific Reports · doi:10.1038/s41598-025-86510-0 · Cited by 65
SycEval: Evaluating LLM Sycophancy
Fanous, A., Goldberg, J., Agarwal, A., Lin, J., Zhou, A., Xu, S., Bikia, V., Daneshjou, R., & Koyejo, S. (2025) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v8i1.36598 · Cited by 62
SycEval: Evaluating LLM Sycophancy
Fanous, A., Goldberg, J., Agarwal, A. A., Lin, J., Zhou, A., Daneshjou, R., & Koyejo, S. (2025) · arXiv · doi:10.48550/arxiv.2502.08177 · Cited by 62
Wu, Z., Qiu, L., Ross, A., Akyürek, E., Chen, B., Wang, B., Kim, N., Andreas, J., & Kim, Y. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.102 · Cited by 58
Investigating machine moral judgement through the Delphi experiment
Jiang, L., Hwang, J. D., Bhagavatula, C., Bras, R. J. L., Liang, J. T., Levine, S., Dodge, J., Sakaguchi, K., Forbes, M., Hessel, J., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00969-6 · Cited by 58
Self-citationThe Ethics of Advanced AI Assistants
Gabriel, I., Manzini, A., Keeling, G., Hendricks, L. A., Rieser, V., Iqbal, H., Tomašev, N., Ktena, I., Kenton, Z., Rodriguez, M., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.16244 · Cited by 57
Self-citationMoral Alignment for LLM AgentsPreprint
Tennant, E., Hailes, S. M. V., & Musolesi, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.01639 · Cited by 56
When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment
Jin, Z., Levine, S., Adauto, F. G., Kamal, O., Sap, M., Sachan, M., Mihalcea, R. F., Tenenbaum, J. B., & Schölkopf, B. (2022) · neural information processing systems · doi:10.52202/068431-2063 · Cited by 53
Self-citationAnderson, M., Anderson, S., & Berenz, V. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2840045 · Cited by 49
MORAL EXPERTISE: JUDGMENT, PRACTICE, AND ANALYSIS
Driver, J. L. (2013) · Social Philosophy and Policy · doi:10.1017/s0265052513000137 · Cited by 47
Moral Gridworlds: A Theoretical Proposal for Modeling Artificial Moral Cognition
Haas, J. (2020) · Minds and Machines · doi:10.1007/s11023-020-09524-9 · Cited by 47
Self-citationAttributions toward artificial agents in a modified Moral Turing Test
Aharoni, E., Fernandes, S., Brady, D. J., Alexander, C., Criner, M., Queen, K., Rando, J., Nahmias, E., & Crespo, V. (2024) · Scientific Reports · doi:10.1038/s41598-024-58087-7 · Cited by 46
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP TasksPreprint
Wang, Y., Mishra, S. R., Alipoormolabashi, P., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.07705 · Cited by 46
Resource-rational contractualism: A triple theory of moral cognition
Levine, S., Chater, N., Tenenbaum, J. B., & Cushman, F. A. (2024) · Behavioral and Brain Sciences · doi:10.1017/s0140525x24001067 · Cited by 43
Self-citationAnderson, S. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511978036.004 · Cited by 39
LLMs as models for analogical reasoning
Musker, S., Duchnowski, A., Millière, R., & Pavlick, E. (2025) · Journal of Memory and Language · doi:10.1016/j.jml.2025.104676 · Cited by 38
Schramowski, P., Turan, C., Jentzsch, S. F., Rothkopf, C. A., & Kersting, K. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00036 · Cited by 36
Assessing Adversarial Robustness of Large Language Models: An Empirical StudyPreprint
Yang, Z., Zhao, M., Zheng, X., & Wattenhofer, R. P. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.02764 · Cited by 36
On the Machine Learning of Ethical Judgments from Natural Language
Talat, Z., Blix, H., Valvoda, J., Ganesh, M. I., Cotterell, R., & Williams, A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.56 · Cited by 33
Lampinen, A. (2024) · Computational Linguistics · doi:10.1162/coli_a_00525 · Cited by 32
Lampinen, A. K. (2022) · arXiv · doi:10.48550/arxiv.2210.15303 · Cited by 32
GPQA: A Graduate-Level Google-Proof Q&A Benchmark
Rein, D., Hou, B. L., Stickland, A. C., Petty, J., Pang, R. Y., Dirani, J., Michael, J., & Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2311.12022 · Cited by 30
Let's Do a Thought Experiment: Using Counterfactuals to Improve Moral ReasoningPreprint
Ma, X., Mishra, S. R., Beirami, A., Beutel, A., & Chen, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.14308 · Cited by 29
Should Users Trust Advanced AI Assistants? Justified Trust As a Function of Competence and Alignment
Manzini, A., Keeling, G., Marchal, N., McKee, K. R., Rieser, V., & Gabriel, I. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658964 · Cited by 28 · cited 1×
“Second, demonstrating competence—by showing how and why a model responds to a moral scenario—is arguably crucial for establishing public trust 96 .”
Self-citationDespite "super-human" performance, current LLMs are unsuited for decisions about ethics and safetyPreprint
Albrecht, J., Kitanidis, E., & Fetterman, A. J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.06295 · Cited by 28
Artificial intelligence learns to reason
Mitchell, M. (2025) · Science · doi:10.1126/science.adw5211 · Cited by 27
MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Tasks
Nie, A., Zhang, Y., Amdekar, A. S., Piech, C., Hashimoto, T., & Gerstenberg, T. (2023) · neural information processing systems · doi:10.52202/075280-3426 · Cited by 25
From Frege to ChatGPT: Compositionality in Language, Cognition, and Deep Neural Networks
Russin, J., McGrath, S. W., & Williams, D. J. (2026) · The MIT Press eBooks · doi:10.7551/mitpress/15862.003.0017 · Cited by 24
Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation MetricsPreprint
Kumar, P., & Mishra, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.18658 · Cited by 23
Momen, A., de Visser, E. J., Wolsten, K., Cooley, K., Walliser, J., & Tossell, C. C. (2023) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2023.063 · Cited by 22
Nezhurina, M., Cipolina-Kun, L., Cherti, M., & Jitsev, J. (2024) · arXiv · doi:10.48550/arxiv.2406.02061 · Cited by 21
Expected UtilitarianismPreprint
Roff, H. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.07321 · Cited by 20
Whitby, B. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511978036.012 · Cited by 18
Casuist BDI-Agent: A New Extended BDI Architecture with the Capability of Ethical Reasoning
Honarvar, A. R., & Ghasem-Aghaee, N. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-05253-8_10 · Cited by 18
Malle, B. F., & Scheutz, M. J. (2018) · Springer Reference Geisteswissenschaften · doi:10.1007/978-3-658-17484-2_17-1 · Cited by 17
Analyzing the Ethical Logic of Eight Large Language ModelsPreprint
Neuman, W. R., Coleman, C., & Shah, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.08951 · Cited by 16
Normative conflicts and shallow AI alignment
Millière, R. (2025) · Philosophical Studies · doi:10.1007/s11098-025-02347-3 · Cited by 16
Discerning What Matters: A Multi-Dimensional Assessment of Moral Competence in LLMs
Kilov, D., Hendy, C., Guyot, S. Y., Snoswell, A., & Lazar, S. (2026) · Proceedings of IASEAI Conference · doi:10.1609/iaseai.v2i1.43035 · Cited by 15
Divergent Creativity in Humans and Large Language ModelsPreprint
Bellemare, A., Lespinasse, F., Thölke, P., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.13012 · Cited by 15
Whose View of Safety? A Deep DIVE Dataset for Pluralistic Alignment of Text-to-Image Models
Rastogi, C., Teh, T. H., Mishra, P., Patel, R., Wang, D., D'iaz, M., Parrish, A., Davani, A., Ashwood, Z., Paganini, M., et al. (2025) · Neural Information Processing Systems · doi:10.48550/arxiv.2507.13383 · Cited by 15
Robustness of large language models in moral judgements
Oh, S., & Demberg, V. (2025) · Royal Society Open Science · doi:10.1098/rsos.241229 · Cited by 13
Allen, C., Wallach, W., & Smit, I. (2020) · Routledge eBooks · doi:10.4324/9781003075011-7 · Cited by 12
Framing effects of per‐person versus aggregate prices in group meals
Ku, H., & Hung, Y. (2018) · Journal of Consumer Behaviour · doi:10.1002/cb.1743 · Cited by 11
From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks
Russin, J., McGrath, S. W., Williams, D. J., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2405.15164 · Cited by 11
Lewis, M., & Mitchell, M. (2024) · arXiv · doi:10.48550/arxiv.2402.08955 · Cited by 10 · cited 2×
“The facsimile problem affects LLM computations of all kinds, including counting 40 , analogical reasoning 41,42 and the generation of new solutions to open-ended problems 43 .”
Whose View of Safety? A Deep DIVE Dataset for Pluralistic Alignment of Text-to-Image Models
Rastogi, C., Teh, T. H., Mishra, P., Patel, R., Wang, D., Díaz, M., Parrish, A., Davani, A. M., Ashwood, Z. C., Paganini, M., et al. (2025) · neural information processing systems · doi:10.52202/085713-4971 · Cited by 10
Ethics of intergenerational (father-to-son) sperm donation
Bredenoord, A. L., Lock, M. T. W. T., & Broekmans, F. J. (2012) · Human Reproduction · doi:10.1093/humrep/des071 · Cited by 9
Procedural Dilemma Generation for Evaluating Moral Reasoning in Humans and Language ModelsPreprint
Fränken, J.-P., Gandhi, K., Qiu, T., Khawaja, A., Goodman, N. D., & Gerstenberg, T. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.10975 · Cited by 7
ChatGPT's advice drives moral judgments with or without justificationPreprint
Krügel, S., Ostermaier, A., & Uhl, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.01897 · Cited by 5
Automating Moral Reasoning (Invited Paper)
Slavkovik, M. (2022) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/oasics.aib.2022.6 · Cited by 5
Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics
Nikankin, Y., Reusch, A., Mueller, A., & Belinkov, Y. (2024) · arXiv · doi:10.48550/arxiv.2410.21272 · Cited by 4
Attributions toward Artificial Agents in a modified Moral Turing TestPreprint
Aharoni, E., Fernandes, S., Brady, D. J., et al. (2024) · doi:10.31234/osf.io/4gxtk · Cited by 3
How do Large Language Models Navigate Conflicts between Honesty and Helpfulness?
Liu, R., Sumers, T. R., Dasgupta, I., & Griffiths, T. L. (2024) · arXiv · doi:10.48550/arxiv.2402.07282 · Cited by 1
Discerning What Matters: A Multi-Dimensional Assessment of Moral Competence in LLMs
Kilov, D., Hendy, C., Guyot, S. Y., Snoswell, A. J., & Lazar, S. (2025) · arXiv · doi:10.48550/arxiv.2506.13082
Social Chemistry 101: Learning to Reason about Social and Moral Norms
Forbes, M., Hwang, J. D., Shwartz, V., Sap, M., & Choi, Y. (2020) · arXiv · doi:10.48550/arxiv.2011.00620
AI Language Model Rivals Expert Ethicist in Perceived Moral Expertise
Dillion, D., Mondal, D., Tandon, N., & Gray, K. (2025) · doi:10.31234/osf.io/w7236_v2
Momen, A., Visser, E. D., Wolsten, K., Cooley, K., Walliser, J., & Tossell, C. C. (2023) · CrimRxiv · doi:10.21428/cb6ab371.755e9cb7
Wilhelm, T. I., Roos, J., & Kaczmarczyk, R. (2023) · doi:10.2196/preprints.49324
Testing Theory of Mind in GPT Models and Humans
Strachan, J., Albergo, D., Borghini, G., Pansardi, O., Scaliti, E., Rufo, A., Manzi, G., Graziano, M., & Becchio, C. (2023) · doi:10.21203/rs.3.rs-3262385/v1
Research works printed but not linked: 12
- Pan, A. et al. Do the rewards justify the means? Measuring trade-offs between rewards and ethical behavior in the Machiavelli benchmark. In Proc. International Conference on Machine Learning 26837–26867 (PMLR, 2023).
- Berg, J. & Gmyrek, P. Automation hits the knowledge worker: ChatGPT and the future of work. In Proc. UN Multi-Stakeholder Forum on Science, Technology and Innovation for the SDGs (STI Forum) (2023).
- Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf (2017).
- Barez, F. et al. Chain-of-thought is not explainability. Preprint at https://fbarez.github.io/assets/pdf/Cot_Is_Not_Explainability.pdf (2025).
- White, J. P., Bhui, R., Cushman, F., Tenenbaum, J. & Levine, S. Moral flexibility in applying queuing norms can be explained by contractualist principles and game-theoretic considerations. In Proc. Annual Meeting of the Cognitive Science Society 46 (2023).
- Doris, J. M. & Plakias, A. In Moral Psychology, Volume 2: The Cognitive Science of Morality: Intuition and Diversity 303–331 (MIT Press, 2008).
- Singer, P. Famine, affluence, and morality. Philos. Publ. Aff. 1, 229–243 (1972).
- Rawls, J. Political Liberalism (Columbia Univ. Press, 1996).
- Piantadosi, S. T. in From Fieldwork to Linguistic Theory: A Tribute to Dan Everett Vol. 15, 353–414 (Language Science Press, 2024).
- Torrance, S. A robust view of machine ethics. In Proc. AAAI Fall Symposium: Computing Machinery and Intelligence https://cdn.aaai.org/Symposia/Fall/2005/FS-05-06/FS05-06-014.pdf (AAAI, 2005).
- Millar, J., Lin, P., Abney, K. & Bekey, G. A. Ethics settings for autonomous vehicles. Robot Ethics 2, 20–34 (2017).
- Allen, C. & Wallach, W. in Robot Ethics: The Ethical and Social Implications of Robotics 55–68 (MIT Press, 2012).
Cited by
Beyond External Constraints: The Missing Dimension of AI GovernancePreprint
CHEN, L. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6449738 · Cited by 65
Positive Alignment: Artificial Intelligence for Human FlourishingPreprint
Laukkonen, R., Krier, S., Bakalar, C., Chandaria, S., Kringelbach, M., Elwood, A., Ford, D., Rosas, F., Boháček, M., Franklin, M., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.10310 · Cited by 11
Resource Rational Contractualism Should Guide AI Alignment
Levine, S., Franklin, M., Zhi‐Xuan, T., Guyot, S. Y., Haas, J., Wong, L., Kilov, D., Choi, Y., Tenenbaum, J. B., Goodman, N. D., et al. (2026) · Proceedings of IASEAI Conference · doi:10.1609/iaseai.v2i1.43037 · Cited by 9
Self-citationArtificial PersonsPreprint
Howells-Whitaker, N., & Lazar, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.08695 · Cited by 6
Pseudo-Consciousness in AI: Bridging the Gap Between Narrow AI and True AGIPreprint
Prestes, J. A. D. L. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5147424 · Cited by 4
Metadata
- Authors
- Haas, J., Bridgers, S., Manzini, A., Henke, B., May, J., Levine, S., Weidinger, L., Shanahan, M., Lum, K., Gabriel, I., & Isaac, W.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- University of London (GB)
- University of Alabama at Birmingham (US)
- New York University (US)
- Corresponding author
- Haas, Julia
- ORCID
- Venue
- Nature
- Published
- 2026-02-18
- Volume
- 650
- Issue
- 8102
- Pages
- 565-573
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W7130339238
- Semantic Scholar
- 3164f856b9d5a68857e48ce1e1cc9b01bab0e141
- Cited by
- 25
- References
- 99
- Citations by year
- 2026: 21
- FWCI
- 10.41
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Life Sciences › Neuroscience › Cognitive Neuroscience › Psychology of Moral and Emotional Judgment
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Ethics in Business and Education
- Keywords
- large language models, moral competence, moral evaluation, moral pluralism, moral reasoning, adversarial evaluation, moral attribution, AI ethics, ethical decision-making
- MeSH terms
- Large Language Models, Artificial Intelligence, Decision Making, Humans, Morals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
