The DOI matches its Crossref record; 30 of the 37 references it cites were found in a registry or an index.
The Pluralistic Moral Gap: Understanding Moral Judgment and Value Differences between Humans and Large Language Models
Open accessAbstract
People increasingly rely on Large Language Models (LLMs) for moral advice, but little is known about how closely LLMs align with human moral judgments.To address this, we introduce the Moral Dilemma Dataset, a benchmark of 1,618 real-world moral dilemmas paired with a distribution of human moral judgments consisting of a binary evaluation and a free-text rationale.We treat moral advice as a pluralistic distributional alignment task, comparing the distributions of LLM and human judgments across dilemmas.We find that models reproduce only high-consensus human judgments while alignment deteriorates sharply when human disagreement increases.Further, using a 60-value taxonomy built from 3,783 value expressions extracted from rationales, we show that LLMs rely on a narrower set of moral values than humans.These findings reveal a pluralistic moral gap-a mismatch in both the distribution and diversity of values expressed.To close this gap, we introduce Dynamic Moral Profiling (DMP), a Dirichletbased sampling method that conditions model outputs on human-derived value profiles.DMP improves alignment by 64.3% and enhances value diversity, offering a step toward more pluralistic and human-aligned moral advice.Judgment: You're in the wrong Rationale: If you can afford it without putting your kids at risk … Is it okay to not pay for my brother's graduate school?My brother got into grad school and asked me to help pay.I can afford it but I have two kids […] right Moral judgment alignment.vs Human Evaluation LLM Evaluation wrong right wrong Judgment: You're in the right Rationale: You have a greater obligation towards your kids … Judgment: You're in the wrong Rationale: Family comes first.This happens once in a lifetime … Judgment: You're in the wrong Rationale: You should help him to be a role model for your kids … … … Real-world moral dilemmas Moral value alignment based on rationales.vs most prevalent values:
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,681
Graham, J., Haidt, J., Koleva, S. P., Motyl, M. S., Iyer, R., Wojcik, S. P., & Ditto, P. H. (2013) · Advances in experimental social psychology · doi:10.1016/b978-0-12-407236-7.00002-4 · Cited by 2,009
A Roadmap to Pluralistic AlignmentPreprint
Sorensen, T., Moore, J., Fisher, J., Gordon, M., Mireshghallah, N., Rytting, C. M., Ye, A., Jiang, L., Lu, X., Dziri, N., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.05070 · Cited by 319
Hopp, F. R., Fisher, J. T., Cornell, D., Huskey, R., & Weber, R. (2020) · Behavior Research Methods · doi:10.3758/s13428-020-01433-0 · Cited by 236
Discovering Language Model Behaviors with Model-Written Evaluations
Perez, E., Ringer, S., Lukosiute, K., Nguyen, K., Chen, E., Heiner, S., Pettit, C., Olsson, C., Kundu, S., Kadavath, S., et al. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.847 · Cited by 194
Quantifying social organization and political polarization in online platforms
Waller, I., & Anderson, A. (2021) · Nature · doi:10.1038/s41586-021-04167-x · Cited by 167
Collective Constitutional AI: Aligning a Language Model with Public Input
Huang, S., Siddarth, D., Lovitt, L., Liao, T. I., Durmus, E., Tamkin, A., & Ganguli, D. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658979 · Cited by 149
Clio: Privacy-Preserving Insights into Real-World AI UsePreprint
Tamkin, A., McCain, M., Handa, K., Durmus, E., Lovitt, L., Rathi, A., Huang, S., Mountfield, A., Hong, J., Ritchie, S., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2412.13678 · Cited by 119
Evaluating the Moral Beliefs Encoded in LLMs
Scherrer, N., Shi, C., Feder, A., & Blei, D. M. (2023) · neural information processing systems · doi:10.52202/075280-2256 · Cited by 110
Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties
Sorensen, T., Jiang, L., Hwang, J. D., Levine, S., Pyatkin, V., West, P., Dziri, N., Lu, X., Rao, K., Bhagavatula, C., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i18.29970 · Cited by 105
MoralBench: Moral Evaluation of LLMs
Ji, J., Chen, Y., Jin, M., Xu, W., Hua, W., & Zhang, Y. (2025) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3748239.3748246 · Cited by 99
Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration
Feng, S., Sorensen, T., Liu, Y., Fisher, J., Park, C. Y., Choi, Y., & Tsvetkov, Y. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.240 · Cited by 96
Röttger, P., Hofmann, V., Pyatkin, V., Hinck, M., Kirk, H., Schuetze, H., & Hovy, D. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.816 · Cited by 89
Self-citationDo LLMs Exhibit Human-like Response Biases? A Case Study in Survey Design
Tjuatja, L., Chen, V., Wu, T., Talwalkwar, A., & Neubig, G. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00685 · Cited by 81
Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model BeliefsPreprint
Hase, P., Diab, M., Çelikyılmaz, A., Li, X., Kozareva, Z., Stoyanov, V., Bansal, M., & Iyer, S. V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.13654 · Cited by 80
Are Large Language Models Consistent over Value-laden Questions?
Moore, J., Deshpande, T., & Yang, D. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.891 · Cited by 64
Values in the Wild: Discovering and Analyzing Values in Real-World Language Model InteractionsPreprint
Huang, S., Durmus, E., McCain, R. M., Handa, K., Tamkin, A., Hong, J., Stern, M., Somani, A., Zhang, X., & Ganguli, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.15236 · Cited by 57
Questioning the Survey Responses of Large Language Models
Dominguez-Olmedo, R., Hardt, M., & Mendler-Dünner, C. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-1458 · Cited by 48
Towards Measuring the Representation of Subjective Global Opinions in Language Models
Durmus, E., Nguyen, K., Liao, T. I., Schiefer, N., Askell, A., Bakhtin, A., Chen, C., Hatfield-Dodds, Z., Hernandez, D., Joseph, N., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.16388 · Cited by 48
Moral Foundations of Large Language Models
Abdulhai, M., Serapio-García, G., Crepy, C., Valter, D., Canny, J., & Jaques, N. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.982 · Cited by 45
Localizing Paragraph Memorization in Language ModelsPreprint
Stoehr, N., Gordon, M., Zhang, C., & Lewis, O. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.19851 · Cited by 42
Normative Evaluation of Large Language Models with Everyday Moral Dilemmas
Sachdeva, P., & van Nuenen, T. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732044 · Cited by 26
Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models
Verga, P., Hofstatter, S., Althammer, S., Su, Y., Piktus, A., Arkhangorodsky, A., Xu, M., White, N., & Lewis, P. (2024) · arXiv · doi:10.48550/arxiv.2404.18796 · Cited by 16
Evaluating Gender Bias of LLMs in Making Morality Judgements
Bajaj, D., Lei, Y., Tong, J., & Huang, R. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.928 · Cited by 13
ELEPHANT: Measuring and understanding social sycophancy in LLMs
Cheng, M., Yu, S., Lee, C., Khadpe, P., Ibrahim, L., & Jurafsky, D. (2025) · arXiv · doi:10.48550/arxiv.2505.13995 · Cited by 12
The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates
Russo, G., Horta Ribeiro, M., Davidson, T. R., Veselovsky, V., & West, R. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757667 · Cited by 12
Self-citationWhich Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations
Handa, K., Tamkin, A., McCain, M., Huang, S., Durmus, E., Heck, S., Mueller, J., Hong, J., Ritchie, S., Belonax, T., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.04761 · Cited by 11
Explainable Patterns for Distinction and Prediction of Moral Judgement on RedditPreprint
Efstathiadis, I. S., Paulino-Passos, G., & Toni, F. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.11155 · Cited by 10
WildChat: 1M ChatGPT Interaction Logs in the Wild
Zhao, W., Ren, X., Hessel, J., Cardie, C., Choi, Y., & Deng, Y. (2024) · arXiv · doi:10.48550/arxiv.2405.01470 · Cited by 10
Self-Recognition in Language Models
Davidson, T. R., Surkov, V., Veselovsky, V., Russo, G. M., West, R., & Gülçehre, Ç. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.703 · Cited by 7
Self-citationKim, J., Yang, N., & Jung, K. (2025) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2025.findings-ijcnlp.51 · Cited by 6
Language Model Council: Democratically Benchmarking Foundation Models on Highly Subjective Tasks
Zhao, J., Plaza-del-Arco, F. M., Genchel, B., & Curry, A. C. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.617 · Cited by 6
Automating Moral Reasoning (Invited Paper)
Slavkovik, M. (2022) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/oasics.aib.2022.6 · Cited by 5
Disentangling Active and Passive Cosponsorship in the U.S. CongressPreprint
Russo, G., Gote, C., Brandenberger, L., Schlosser, S., & Schweitzer, F. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.09674 · Cited by 4
Self-citationCausal Estimation of Memorisation Profiles
Lesci, P., Meister, C., Hofmann, T. F., Vlachos, A., & Pimentel, T. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.834 · Cited by 3
SocialGaze: Improving the Integration of Human Social Norms in Large Language Models
Vijjini, A. R., Menon, R. R., Fu, J., Srivastava, S., & Chaturvedi, S. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.962 · Cited by 3
AI Language Model Rivals Expert Ethicist in Perceived Moral Expertise
Dillion, D., Mondal, D., Tandon, N., & Gray, K. (2025) · doi:10.31234/osf.io/w7236_v2
Research works printed but not linked: 3
- Xuelin Liu, Pengyuan Liu, and Dong Yu. 2025. What's the most important value? INVP: INvestigating the value priorities of LLMs through decision-making in social scenarios. In Proceedings of the 31st Inter- national Conference on Computational Linguistics, pages 4725-4752. Association for Computational Linguistics.
- Nicole Meister, Carlos Guestrin, and Tatsunori Hashimoto. 2025. Benchmarking distributional align- ment of large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Lin- guistics: Human Language Technologies (Volume 1: Long Papers), pages 24-49. Association for Compu- tational Linguistics. · DOI
- Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect? In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org.
Cited by
Krsteski, S., Russo, G., Chang, S. Y., West, R., & Gligorić, K. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.498 · Cited by 20
Self-citationInertia in Moral and Value Judgments of Large Language Models
Lee, B., Lee, Y., & Cho, H. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1246 · Cited by 16
Discerning What Matters: A Multi-Dimensional Assessment of Moral Competence in LLMs
Kilov, D., Hendy, C., Guyot, S. Y., Snoswell, A., & Lazar, S. (2026) · Proceedings of IASEAI Conference · doi:10.1609/iaseai.v2i1.43035 · Cited by 15
Generative Value Conflicts Reveal LLM PrioritiesPreprint
Liu, A., Ghate, K., Diab, M., Fried, D., Kasirzadeh, A., & Max Kleiman-weiner, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.25369 · Cited by 13
The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates
Russo, G., Horta Ribeiro, M., Davidson, T. R., Veselovsky, V., & West, R. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757667 · Cited by 12
Self-citation
Metadata
- Authors
- Russo, G., Nozza, D., Röttger, P., & Hovy, D.
- Affiliations
- École Polytechnique Fédérale de Lausanne (CH)
- Bocconi University (IT)
- University of Oxford (GB)
- Venue
- Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2026
- Pages
- 6481-6497
- OpenAlex
- W7140081780
- Cited by
- 19
- References
- 25
- Citations by year
- 2026: 14
- FWCI
- 18.51
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the European Chapter of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Archivio istituzionale della Ricerca - Bocconi (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- European Commission
- Università Bocconi
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Life Sciences › Neuroscience › Cognitive Neuroscience › Psychology of Moral and Emotional Judgment
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Ethics in Business and Education
- Keywords
- large language models, moral judgment, ethical dilemmas, human-LLM alignment, moral advice, moral values, value diversity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
