The DOI matches its Crossref record; 30 of the 37 references it cites were found in a registry or an index.
The Pluralistic Moral Gap: Understanding Moral Judgment and Value Differences between Humans and Large Language Models
Open accessAbstract
People increasingly rely on Large Language Models (LLMs) for moral advice, but little is known about how closely LLMs align with human moral judgments.To address this, we introduce the Moral Dilemma Dataset, a benchmark of 1,618 real-world moral dilemmas paired with a distribution of human moral judgments consisting of a binary evaluation and a free-text rationale.We treat moral advice as a pluralistic distributional alignment task, comparing the distributions of LLM and human judgments across dilemmas.We find that models reproduce only high-consensus human judgments while alignment deteriorates sharply when human disagreement increases.Further, using a 60-value taxonomy built from 3,783 value expressions extracted from rationales, we show that LLMs rely on a narrower set of moral values than humans.These findings reveal a pluralistic moral gap-a mismatch in both the distribution and diversity of values expressed.To close this gap, we introduce Dynamic Moral Profiling (DMP), a Dirichletbased sampling method that conditions model outputs on human-derived value profiles.DMP improves alignment by 64.3% and enhances value diversity, offering a step toward more pluralistic and human-aligned moral advice.Judgment: You're in the wrong Rationale: If you can afford it without putting your kids at risk … Is it okay to not pay for my brother's graduate school?My brother got into grad school and asked me to help pay.I can afford it but I have two kids […] right Moral judgment alignment.vs Human Evaluation LLM Evaluation wrong right wrong Judgment: You're in the right Rationale: You have a greater obligation towards your kids … Judgment: You're in the wrong Rationale: Family comes first.This happens once in a lifetime … Judgment: You're in the wrong Rationale: You should help him to be a role model for your kids … … … Real-world moral dilemmas Moral value alignment based on rationales.vs most prevalent values:
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,681
Graham, J., Haidt, J., Koleva, S. P., Motyl, M. S., Iyer, R., Wojcik, S. P., & Ditto, P. H. (2013) · Advances in experimental social psychology · doi:10.1016/b978-0-12-407236-7.00002-4 · Cited by 2,009
A Roadmap to Pluralistic AlignmentPreprint
Sorensen, T., Moore, J., Fisher, J., Gordon, M., Mireshghallah, N., Rytting, C. M., Ye, A., Jiang, L., Lu, X., Dziri, N., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.05070 · Cited by 319
Hopp, F. R., Fisher, J. T., Cornell, D., Huskey, R., & Weber, R. (2020) · Behavior Research Methods · doi:10.3758/s13428-020-01433-0 · Cited by 236
Discovering Language Model Behaviors with Model-Written Evaluations
Perez, E., Ringer, S., Lukosiute, K., Nguyen, K., Chen, E., Heiner, S., Pettit, C., Olsson, C., Kundu, S., Kadavath, S., et al. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.847 · Cited by 194
Quantifying social organization and political polarization in online platforms
Waller, I., & Anderson, A. (2021) · Nature · doi:10.1038/s41586-021-04167-x · Cited by 167
Collective Constitutional AI: Aligning a Language Model with Public Input
Huang, S., Siddarth, D., Lovitt, L., Liao, T. I., Durmus, E., Tamkin, A., & Ganguli, D. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658979 · Cited by 149
Clio: Privacy-Preserving Insights into Real-World AI UsePreprint
Tamkin, A., McCain, M., Handa, K., Durmus, E., Lovitt, L., Rathi, A., Huang, S., Mountfield, A., Hong, J., Ritchie, S., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2412.13678 · Cited by 119
Evaluating the Moral Beliefs Encoded in LLMs
Scherrer, N., Shi, C., Feder, A., & Blei, D. M. (2023) · neural information processing systems · doi:10.52202/075280-2256 · Cited by 110
Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties
Sorensen, T., Jiang, L., Hwang, J. D., Levine, S., Pyatkin, V., West, P., Dziri, N., Lu, X., Rao, K., Bhagavatula, C., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i18.29970 · Cited by 105
Research works printed but not linked: 3
- Xuelin Liu, Pengyuan Liu, and Dong Yu. 2025. What's the most important value? INVP: INvestigating the value priorities of LLMs through decision-making in social scenarios. In Proceedings of the 31st Inter- national Conference on Computational Linguistics, pages 4725-4752. Association for Computational Linguistics.
- Nicole Meister, Carlos Guestrin, and Tatsunori Hashimoto. 2025. Benchmarking distributional align- ment of large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Lin- guistics: Human Language Technologies (Volume 1: Long Papers), pages 24-49. Association for Compu- tational Linguistics. · DOI
- Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect? In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org.
Cited by
Krsteski, S., Russo, G., Chang, S. Y., West, R., & Gligorić, K. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.498 · Cited by 20
Self-citationInertia in Moral and Value Judgments of Large Language Models
Lee, B., Lee, Y., & Cho, H. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1246 · Cited by 16
Discerning What Matters: A Multi-Dimensional Assessment of Moral Competence in LLMs
Kilov, D., Hendy, C., Guyot, S. Y., Snoswell, A., & Lazar, S. (2026) · Proceedings of IASEAI Conference · doi:10.1609/iaseai.v2i1.43035 · Cited by 15
Generative Value Conflicts Reveal LLM PrioritiesPreprint
Liu, A., Ghate, K., Diab, M., Fried, D., Kasirzadeh, A., & Max Kleiman-weiner, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.25369 · Cited by 13
The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates
Russo, G., Horta Ribeiro, M., Davidson, T. R., Veselovsky, V., & West, R. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757667 · Cited by 12
Self-citation
Metadata
- Authors
- Russo, G., Nozza, D., Röttger, P., & Hovy, D.
- Affiliations
- École Polytechnique Fédérale de Lausanne (CH)
- Bocconi University (IT)
- University of Oxford (GB)
- Venue
- Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2026
- Pages
- 6481-6497
- OpenAlex
- W7140081780
- Cited by
- 19
- References
- 25
- Citations by year
- 2026: 14
- FWCI
- 18.51
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the European Chapter of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Archivio istituzionale della Ricerca - Bocconi (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- European Commission
- Università Bocconi
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Life Sciences › Neuroscience › Cognitive Neuroscience › Psychology of Moral and Emotional Judgment
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Ethics in Business and Education
- Keywords
- large language models, moral judgment, ethical dilemmas, human-LLM alignment, moral advice, moral values, value diversity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
