The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index; none of the 27 sources its claims cite is checked for support yet.
Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations
Open accessAbstract
Large language models (LLMs) have taken centre stage in debates on Artificial Intelligence. Yet there remains a gap in how to assess LLMs' conformity to important human values. In this paper, we investigate whether state-of-the-art LLMs, GPT-4 and Claude 2.1 (Gemini Pro and LLAMA 2 did not generate valid results) are moral hypocrites. We employ two research instruments based on the Moral Foundations Theory: (i) the Moral Foundations Questionnaire (MFQ), which investigates which values are considered morally relevant in abstract moral judgements; and (ii) the Moral Foundations Vignettes (MFVs), which evaluate moral cognition in concrete scenarios related to each moral foundation. We characterise conflicts in values between these different abstractions of moral evaluation as hypocrisy. We found that both models displayed reasonable consistency within each instrument compared to humans, but they displayed contradictory and hypocritical behaviour when we compared the abstract values present in the MFQ to the evaluation of concrete moral violations of the MFV.
Overview
CoChat Summary
TL;DR
The paper tests whether GPT-4 and Claude 2.1 display moral hypocrisy by comparing abstract moral values on the Moral Foundations Questionnaire (MFQ) with concrete moral judgments on the Moral Foundations Vignettes (MFVs). Both models showed within-instrument consistency comparable to humans, but they failed to show coherence across abstraction levels, which the authors characterize as hypocritical behavior.
Key findings
- Both models displayed reasonable consistency within each instrument: GPT-4 had α_MFV = 0.72 and α_MFQ = 0.81; Claude 2.1 had α_MFV = 0.91 and α_MFQ = 0.46; human comparison values were α_MFV = 0.87 and α_MFQ = 0.74.
- Across instruments, the models showed little to no coherent relation between abstract MFQ values and concrete MFV judgments. Most regression models had no significant predictors; when significant, predictors often did not correspond to the matching foundation.
- Regression coefficients of determination were very low for both LLMs (R² < 0.08, except Claude Loyalty and Liberty with R² ≈ 0.10), compared with human respondents in Clifford et al. (2015), where R² ∈ [0.17, 0.38].
- The authors conclude that GPT-4 and Claude 2.1 behaved as moral hypocrites in the experiment: consistent within instruments but incoherent between abstract values and concrete moral evaluations.
Methods
- The study used two Moral Foundations Theory instruments: the MFQ30 for abstract moral values and 68 Moral Foundations Vignettes for concrete moral violations, with human comparison data from Clifford et al. (2015) and Marques et al. (2020).
- Four state-of-the-art LLMs were initially selected: GPT-4 (gpt-4-0613), Claude 2.1, Gemini Pro, and LLAMA-2-Chat-70b. Only GPT-4 and Claude 2.1 generated valid outputs; Gemini Pro and LLAMA 2 were dropped due to invalid or token-limit-exceeding outputs.
- The experiment ran two order conditions, VQ (MFV first) and QV (MFQ first), with 100 responses per condition per model. Valid responses were Claude 2.1: QV 99, VQ 59, total 158; GPT-4: QV 100, VQ 89, total 189.
- Analysis used Cronbach’s α for within-instrument consistency, two-way ANOVAs, and ordinary linear regressions predicting MFV ratings from MFQ measures, following Clifford et al. (2015).
Limitations
- The paper states that only two commercial LLMs were analyzed, and the open-source LLAMA 2.1 model could not produce valid results under the study’s criteria.
- The authors note that future work should explore scaling trends, other open models, prompting styles, and temperature settings, since there is no agreed-upon procedure for questionnaire-style prompting and small prompt changes may significantly alter results.
- The study is based on a single task—answering questionnaires with one prompt structure—so the authors call for future research on different tasks and evaluation methods.
- Evident from the text: Gemini Pro was discarded after all but three outputs were invalid, and Claude 2.1 showed a large valid-response imbalance between QV and VQ conditions (99 vs. 59), which may affect interpretation.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 1 Not valid · 18 Not checked · 52 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As reported in our Appendix 16 , we observed a significant effect of order condition (i.e., whether the MFQ or the MFV came first) for GPT-4.The MFQ α values were higher for that model when that instrument was presented first.”
§ Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations > 5 Results
plainstates an associationSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The Agent term was not found to be significant (p = 0.063).”
§ Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations > Are Language Models' Moral Values Consistent within Instruments ?
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“That is: even in the order which most favours GPT-4 reliability, there was still virtually no relationship between abstract moral values and concrete moral judgments.”
§ Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations > 5 Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, our results utterly lacked any coherence in the values between abstraction levels.”
§ Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations > 7 Conclusion
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Most of our regression models had no independent variable as a significant predictor.”
§ Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations > Are Language Models' Values Coherent Across Different Abstractions, or Are They Hypocritical?
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
28 more claims
Create a free account to see all 71 claims and every source verdict.
- It may state two separate claims as one. 22
- Probably not a substantial claim. 23
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- ConsistentF(1, 30) = 27.93reported p < .001 · recomputed p < .001
“F (1,30) = 27.93,p < .001”
- ConsistentF(2, 30) = 25.8reported p < .001 · recomputed p < .001
“F (2,30) = 25.8, p < .001”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations > Moral Alignment
“we found no significant correlation between foundations expressed in these instruments, even though the instruments were answered in sequence as part of a single context.”
§ Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations > Are Language Models' Values Coherent Across Different Abstractions, or Are They Hypocritical?
“In contrast, GPT-4 displayed almost no significance patterns at a 0.05 level, with MFQ Authority correlated with MFV Liberty and a relation between Loyalty in both instruments.”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations > Data Processing
“We only discarded seven GPT-4 responses using this criterion from the QVcondition because of a zero or one score for the good item.”
§ Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations > 8 Limitations and Future Work
“| GPT-4 | 100 | 89 189 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations > 7 Conclusion
“Of those, only GPT-4 and Claude 2.1 generated valid outputs for our stimuli.”
§ Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations > Data Processing
“Through this procedure, we discarded all but three of Gemini Pro's outputs, which led us to discard the model as a whole.”
Citations
Cites
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Liberals and conservatives rely on different sets of moral foundations.
Graham, J., Haidt, J., & Nosek, B. A. (2009) · Journal of Personality and Social Psychology · doi:10.1037/a0015141 · Cited by 4,560
Graham, J., Nosek, B. A., Haidt, J., Iyer, R., Koleva, S., & Ditto, P. H. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0021847 · Cited by 2,666
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
The Sources of Four Commonly Reported Cutoff Criteria: What Did They Really Say?
Lance, C. E., Butts, M. M., & Michels, L. C. (2006) · Organizational Research Methods · doi:10.1177/1094428105284919 · Cited by 2,053
Sharma, M., Tong, M., Korbak, T., Duperrouzel, D., Was, H., Korbak, J., Cheng, M., Askell, A., Bai, Y., Brain, C., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.13548 · Cited by 1,313
Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581388 · Cited by 1,274
Emergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264
Gemini: A Family of Highly Capable Multimodal Models
Gemini Team, Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.11805 · Cited by 990
Understanding Libertarian Morality: The Psychological Dispositions of Self-Identified Libertarians
Iyer, R., Koleva, S. P., Graham, J., Ditto, P. H., & Haidt, J. (2012) · PLoS ONE · doi:10.1371/journal.pone.0042366 · Cited by 790
Out of One, Many: Using Language Models to Simulate Human Samples
Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., & Wingate, D. (2023) · Political Analysis · doi:10.1017/pan.2023.2 · Cited by 783
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Xu, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.01219 · Cited by 544
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Chen, Y., et al. (2025) · Computational Linguistics · doi:10.1162/coli.a.16 · Cited by 534
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.05862 · Cited by 442
Fine-Tuning Language Models from Human Preferences
Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., & Irving, G. (2019) · arXiv · doi:10.48550/arxiv.1909.08593 · Cited by 413
Morality beyond the WEIRD: How the nomological network of morality varies across cultures.
Atari, M., Haidt, J., Graham, J., Koleva, S. P., Stevens, S. T., & Dehghani, M. (2023) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000470 · Cited by 408
Can AI language models replace human participants?
Dillion, D., Tandon, N., Gu, Y., & Gray, K. (2023) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.04.008 · Cited by 405
Clifford, S., Iyengar, V., Cabeza, R., & Sinnott-Armstrong, W. (2015) · Behavior Research Methods · doi:10.3758/s13428-014-0551-2 · Cited by 351
Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., & Kaplan, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.08073 · Cited by 326
Evaluating Large Language Models in Generating Synthetic HCI Research Data: a Case Study
Hämäläinen, P., Tavast, M., & Kunnari, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580688 · Cited by 248
Hopp, F. R., Fisher, J. T., Cornell, D., Huskey, R., & Weber, R. (2020) · Behavior Research Methods · doi:10.3758/s13428-020-01433-0 · Cited by 236
Ideology Justifies Morality: Political Beliefs Predict Moral Foundations
Hatemi, P., Crabtree, C. D., & Smith, K. B. (2019) · American Journal of Political Science · doi:10.1111/ajps.12448 · Cited by 226
Wang, X., Zhu, W., Saxon, M., Mark Steyvers, B., & Wang, W. Y. (2023) · neural information processing systems · doi:10.52202/075280-0687 · Cited by 155
Wang, X., Zhu, W., & Wang, W. Y. (2023) · arXiv.org · doi:10.48550/arxiv.2301.11916 · Cited by 135
Testing Measurement Invariance of the Moral Foundations Questionnaire Across 27 Countries
Iurino, K., & Saucier, G. (2020) · Assessment · doi:10.1177/1073191118817916 · Cited by 129
GPT is an effective tool for multilingual psychological text analysisPreprint
Rathje, S., Mirea, D.-M., Sucholutsky, I., Marjieh, R., Robertson, C., & Bavel, J. J. V. (2023) · doi:10.31234/osf.io/sekf5 · Cited by 129
Yilmaz, O., Harma, M., Bahçekapili, H. G., & Cesur, S. (2016) · Personality and Individual Differences · doi:10.1016/j.paid.2016.04.090 · Cited by 117
Whose Opinions Do Language Models Reflect?
Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., & Hashimoto, T. (2023) · arXiv · doi:10.48550/arxiv.2303.17548 · Cited by 114
Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity
Simmons, G. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-srw.40 · Cited by 113
Isserow, J., & Klein, C. (2017) · Journal of Ethics and Social Philosophy · doi:10.26556/jesp.v12i2.224 · Cited by 103
Foundations of morality in Iran
Atari, M., Graham, J., & Dehghani, M. (2020) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2020.07.014 · Cited by 101
Diminished diversity-of-thought in a standard large language model
Park, P. S., Schoenegger, P., & Zhu, C. (2024) · Behavior Research Methods · doi:10.3758/s13428-023-02307-x · Cited by 83
Re-evaluating GPT-4’s bar exam performance
Martínez, E. (2024) · Artificial Intelligence and Law · doi:10.1007/s10506-024-09396-9 · Cited by 78
Hypocritical flip-flop, or courageous evolution? When leaders change their moral minds.
Kreps, T. A., Laurin, K., & Merritt, A. C. (2017) · Journal of Personality and Social Psychology · doi:10.1037/pspi0000103 · Cited by 78
Exploring the psychology of LLMs’ moral and legal reasoning
Almeida, G. F. C. F., Nunes, J. L., Engelmann, N., Wiegmann, A., & Araújo, M. D. (2024) · Artificial Intelligence · doi:10.1016/j.artint.2024.104145 · Cited by 70
Self-citationPredicting demographics, moral foundations, and human values from digital behaviours
Kalimeri, K., Beiró, M. G., Delfino, M., Raleigh, R., & Cattuto, C. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.11.024 · Cited by 68
Truth machines: synthesizing veracity in AI language models
Munn, L., Magee, L., & Arora, V. (2024) · AI & SOCIETY · doi:10.1007/s00146-023-01756-4 · Cited by 65
To protect science, we must use LLMs as zero-shot translators
Mittelstadt, B., Wachter, S., & Russell, C. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01744-0 · Cited by 58
Exploring the Use of Large Language Models for Improving the Awareness of Mindfulness
Kumar, H., Wang, Y., Shi, J., Musabirov, I., Farb, N. A. S., & Williams, J. J. (2023) · Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544549.3585614 · Cited by 50
Moral Foundations of Large Language Models
Abdulhai, M., Serapio-García, G., Crepy, C., Valter, D., Canny, J., & Jaques, N. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.982 · Cited by 45
Multiple moral foundations predict responses to sacrificial dilemmas
Crone, D. L., & Laham, S. M. (2015) · Personality and Individual Differences · doi:10.1016/j.paid.2015.04.041 · Cited by 45
LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
Guha, N., Nyarko, J., Ho, D. E., Ré, C., Chilton, A., Narayana, A., Chohlas-Wood, A., Peters, A., Waldon, B., Rockmore, D. N., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.11462 · Cited by 30
Hutson, M. (2023) · Science · doi:10.1126/science.adj6791 · Cited by 24
INACIA: Integrating Large Language Models in Brazilian Audit Courts: Opportunities and Challenges
Pereira, J., Assumpcao, A., Trecenti, J. A. Z., Airosa, L., Lente, C., Cléto, J., Dobins, G., Nogueira, R. F., Mitchell, L., & Lotufo, R. (2024) · Digital Government Research and Practice · doi:10.1145/3652951 · Cited by 21
Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification
Guha, N., Chen, M. F., Bhatia, K., Mirhoseini, A., Sala, F., & Ré, C. (2023) · neural information processing systems · doi:10.52202/075280-2762 · Cited by 10
Diminished Diversity-of-Thought in a Standard Large Language Model
Park, P. S., Schoenegger, P., & Zhu, C. (2023) · London School of Economics and Political Science Research Online (London School of Economics and Political Science) · doi:10.48550/arxiv.2302.07267 · Cited by 9
Marques, L. M., Clifford, S., Iyengar, V., Bonato, G. V., Cabral, P. M., dos Santos, R. B., Cabeza, R. E. L., Sinnott-Armstrong, W. P., & Boggio, P. S. (2020) · Judgment and Decision Making · doi:10.1017/s1930297500006963 · Cited by 5
Evaluating Large Language Models in Psychological Research: A Guide for ReviewersPreprint
Abdurahman, S., Ziabari, A. S., Moore, A. K., Bartels, D. M., & Dehghani, M. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/ag7hy · Cited by 3
Engel, C., Grossmann, M. R. P., & Ockenfels, A. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4655962 · Cited by 2
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
Kittay, E. F. (1982) · Metaphilosophy · doi:10.1111/j.1467-9973.1982.tb00685.x
Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification
Guha, N., Chen, M. F., Bhatia, K., Mirhoseini, A., Sala, F., & Ré, C. (2023) · arXiv · doi:10.48550/arxiv.2307.11031
INACIA: Integrating Large Language Models in Brazilian Audit Courts: Opportunities and Challenges
Pereira, J., Assumpcao, A., Trecenti, J., Airosa, L., Lente, C., Cléto, J., Dobins, G., Nogueira, R., Mitchell, L., & Lotufo, R. (2024) · arXiv · doi:10.48550/arxiv.2401.05273
Research works printed but not linked: 14
- Crisp, R.; and Cowton, C. 1994. Hypocrisy and Moral Se- riousness. American Philosophical Quarterly, 31(4): 343- 349.
- Frimer, J. 2019. Moral Foundations Dictionary 2.0. Gabriel, I. 2020. Artificial Intelligence, Values, and Align- ment. Minds and Machines, 30(3): 411-437.
- Graham, J.; Haidt, J.; Koleva, S.; Motyl, M.; Iyer, R.; Woj- cik, S. P.; and Ditto, P. H. 2013. Chapter Two -Moral Foun- dations Theory: The Pragmatic Validity of Moral Pluralism. In Devine, P.; and Plant, A., eds., Advances in Experimental Social Psychology, volume 47, 55-130. Academic Press.
- Griffith, S.; Subramanian, K.; Scholz, J.; Isbell, C. L.; and Thomaz, A. 2013. Policy Shaping: Integrating Human Feedback with Reinforcement Learning. In Proceedings of the 26th International Conference on Neural Information Processing Systems -Volume 2, NIPS'13, 2625-2633. Red Hook, NY, USA: Curran Associates Inc.
- Klein, R. A.; Vianello, M.; Hasselman, F.; Adams, B. G.; Adams, R. B.; Alper, S.; Aveyard, M.; Axt, J. R.; Babalola, M. T.; Bahník, Š.; Batra, R.; Berkics, M.; Bernstein, M. J.; Berry, D. R.; Bialobrzeska, O.; Binan, E. D.; Bocian, K.; Brandt, M. J.; Busching, R.; Rédei, A. C.; Cai, H.; Cambier, F.; Cantarero, K.; Carmichael, C. L.; Ceric, F.; Chandler, J.; Chang, J.-H.; Chatard, A.; Chen, E. E.; Cheong, W.; Cicero, D. C.; Coen, S.; Coleman, J. A.; Collisson, B.; Conway, M. A.; Corker, K. S.; Curran, P. G.; Cushman, F.; Dagona, Z. K.; Dalgar, I.; Dalla Rosa, A.; Davis, W. E.; De Bruijn, M.; De Schutter, L.; Devos, T.; De Vries, M.; Dogulu, C.; Dozo, N.; Dukes, K. N.; Dunham, Y.; Durrheim, K.; Eber- sole, C. R.; Edlund, J. E.; Eller, A.; English, A. S.; Finck, C.; Frankowska, N.; Freyre, M.-Á.; Friedman, M.; Galliani, E. M.; Gandi, J. C.; Ghoshal, T.; Giessner, S. R.; Gill, T.; Gnambs, T.; Gómez, Á.; González, R.; Graham, J.; Grahe, J. E.; Grahek, I.; Green, E. G. T.; Hai, K.; Haigh, M.; Haines, E. L.; Hall, M. P.; Heffernan, M. E.; Hicks, J. A.; Houdek, P.; Huntsinger, J. R.; Huynh, H. P.; IJzerman, H.; Inbar, Y.; Innes-Ker, Å. H.; Jiménez-Leal, W.; John, M.-S.; Joy-Gaba, J. A.; Kamiloglu, R. G.; Kappes, H. B.; Karabati, S.; Kar- ick, H.; Keller, V. N.; Kende, A.; Kervyn, N.; Knežević, G.; Kovacs, C.; Krueger, L. E.; Kurapov, G.; Kurtz, J.; Lakens, D.; Lazarević, L. B.; Levitan, C. A.; Lewis, N. A.; Lins, S.; Lipsey, N. P.; Losee, J. E.; Maassen, E.; Maitner, A. T.; Ma- lingumu, W.; Mallett, R. K.; Marotta, S. A.; Medjedović, J.; Mena-Pacheco, F.; Milfont, T. L.; Morris, W. L.; Murphy, S. C.; Myachykov, A.; Neave, N.; Neijenhuijs, K.; Nelson, A. J.; Neto, F.; Lee Nichols, A.; Ocampo, A.; O'Donnell, S. L.; Oikawa, H.; Oikawa, M.; Ong, E.; Orosz, G.; Os- owiecka, M.; Packard, G.; Pérez-Sánchez, R.; Petrović, B.; Pilati, R.; Pinter, B.; Podesta, L.; Pogge, G.; Pollmann, M. M. H.; Rutchick, A. M.; Saavedra, P.; Saeri, A. K.; Salomon, E.; Schmidt, K.; Schönbrodt, F. D.; Sekerdej, M. B.; Sir- lopú, D.; Skorinko, J. L. M.; Smith, M. A.; Smith-Castro, V.; Smolders, K. C. H. J.; Sobkow, A.; Sowden, W.; Spachtholz, P.; Srivastava, M.; Steiner, T. G.; Stouten, J.; Street, C. N. H.; Sundfelt, O. K.; Szeto, S.; Szumowska, E.; Tang, A. C. W.; Tanzer, N.; Tear, M. J.; Theriault, J.; Thomae, M.; Torres, D.; Traczyk, J.; Tybur, J. M.; Ujhelyi, A.; Van Aert, R. C. M.; Van Assen, M. A. L. M.; Van Der Hulst, M.; Van Lange, P. A. M.; Van 'T Veer, A. E.; Vásquez-Echeverría, A.; Ann Vaughn, L.; Vázquez, A.; Vega, L. D.; Verniers, C.; Verschoor, M.; Voermans, I. P. J.; Vranka, M. A.; Welch, C.; Wichman, A. L.; Williams, L. A.; Wood, M.; Woodz- icka, J. A.; Wronska, M. K.; Young, L.; Zelenski, J. M.; Zhi- jia, Z.; and Nosek, B. A. 2018. Many Labs 2: Investigating Variation in Replicability Across Samples and Settings. Ad- vances in Methods and Practices in Psychological Science, 1(4): 443-490.
- Nunnally, J. C.; and Bernstein, I. H. 1994. Psychometric Theory. McGraw-Hill Companies,Incorporated. ISBN 978- 0-07-047849-7.
- Park, J. S.; Popowski, L.; Cai, C.; Morris, M. R.; Liang, P.; and Bernstein, M. S. 2022. Social Simulacra: Creating Pop- ulated Prototypes for Social Computing Systems. In Pro- ceedings of the 35th Annual ACM Symposium on User Inter- face Software and Technology, UIST '22, 1-18. New York, NY, USA: Association for Computing Machinery. ISBN 978-1-4503-9320-1.
- Savelka, J. 2023. Unlocking Practical Applications in Legal Domain: Evaluation of GPT for Zero-Shot Semantic An- notation of Legal Texts. In Nineteenth International Con- ference on Artificial Intelligence and Law (ICAIL 2023), 5. Braga, Portugal: ACM, New York, NY, USA.
- Schaeffer, R.; Miranda, B.; and Koyejo, S. 2023. Are Emergent Abilities of Large Language Models a Mirage? Advances in Neural Information Processing Systems, 36: 55565-55581.
- Schein, C.; and Gray, K. 2018. The Theory of Dyadic Moral- ity: Reinventing Moral Judgment by Redefining Harm. Per- sonality and Social Psychology Review, 22(1): 32-70.
- Singhal, K.; Azizi, S.; Tu, T.; Mahdavi, S. S.; Wei, J.; Chung, H. W.; Scales, N.; Tanwani, A.; Cole-Lewis, H.; Pfohl, S.; Payne, P.; Seneviratne, M.; Gamble, P.; Kelly, C.; Babiker, A.; Schärli, N.; Chowdhery, A.; Mansfield, P.; Demner- Fushman, D.; Agüera y Arcas, B.; Webster, D.; Corrado, G. S.; Matias, Y.; Chou, K.; Gottweis, J.; Tomasev, N.; Liu, Y.; Rajkomar, A.; Barral, J.; Semturs, C.; Karthikesalingam, A.; and Natarajan, V. 2023. Large Language Models Encode Clinical Knowledge. Nature, 620(7972): 172-180.
- Smith, K. B.; Alford, J. R.; Hibbing, J. R.; Martin, N. G.; and Hatemi, P. K. 2017. Intuitive Ethics and Political Ori- entations: Testing Moral Foundations as a Theory of Politi- cal Ideology. American Journal of Political Science, 61(2): 424-437.
- Su, H.; Kasai, J.; Wu, C. H.; Shi, W.; Wang, T.; Xin, J.; Zhang, R.; Ostendorf, M.; Zettlemoyer, L.; Smith, N. A.; and Yu, T. 2022. Selective Annotation Makes Language Models Better Few-Shot Learners. In The Eleventh International Conference on Learning Representations.
- Thirunavukarasu, A. J.; Ting, D. S. J.; Elangovan, K.; Gutierrez, L.; Tan, T. F.; and Ting, D. S. W. 2023. Large Lan- guage Models in Medicine. Nature Medicine, 29(8): 1930- 1940.
Cited by
Normative Evaluation of Large Language Models with Everyday Moral Dilemmas
Sachdeva, P., & van Nuenen, T. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732044 · Cited by 26
A survey on moral foundation theory and pre-trained language models: current advances and challenges
Zangari, L., Greco, C. M., Picca, D., & Tagarelli, A. (2025) · AI & Society · doi:10.1007/s00146-025-02225-w · Cited by 22
Large Language Models meet moral values: A comprehensive assessment of moral abilities
Bulla, L., Giorgis, S. D., Mongiovì, M., & Aldo Gangemi, A. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100609 · Cited by 20
Evaluating Gender Bias of LLMs in Making Morality Judgements
Bajaj, D., Lei, Y., Tong, J., & Huang, R. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.928 · Cited by 13
Auditing the Ethical Logic of Generative AI ModelsPreprint
Neuman, W. R., Coleman, C., Dasdan, A., Ali, S., & Shah, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.17544 · Cited by 9
Metadata
- Authors
- Nunes, J. L., Almeida, G. F. C. F., Araujo, M. D., & Barbosa, S. D. J.
- Affiliations
- Pontifícia Universidade Católica do Rio de Janeiro (BR)
- Insper (BR)
- Venue
- Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society
- Published
- 2024-10-16
- Volume
- 7
- Pages
- 1074-1087
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W4404518322
- Cited by
- 17
- References
- 16
- Citations by year
- 2026: 8 · 2025: 8 · 2024: 1
- FWCI
- 2.60
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Open Science Framework
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Life Sciences › Neuroscience › Cognitive Neuroscience › Psychology of Moral and Emotional Judgment
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Ethics in Business and Education
- Keywords
- large language models, moral hypocrisy, moral foundations, Moral Foundations Questionnaire, GPT-4, Claude 2, value consistency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-13, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-13 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
