CoChat
Sign Up
PreprintCited by 5

The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.

Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity

Simmons, Gabriel·2022·arXiv
DOI 10.48550/arxiv.2209.12106arXiv 2209.12106
Open this paper in CoChat — free account

Abstract

Large Language Models (LLMs) have demonstrated impressive capabilities in generating fluent text, as well as tendencies to reproduce undesirable social biases. This study investigates whether LLMs reproduce the moral biases associated with political groups in the United States, an instance of a broader capability herein termed moral mimicry. This hypothesis is explored in the GPT-3/3.5 and OPT families of Transformer-based LLMs. Using tools from Moral Foundations Theory, it is shown that these LLMs are indeed moral mimics. When prompted with a liberal or conservative political identity, the models generate text reflecting corresponding moral biases. This study also explores the relationship between moral mimicry and model size, and similarity between human and LLM moral word use.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2209.12106

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • GPT-3 has moral biases in terms of the Moral Foundation theory.

    “In addition, LLMs like GPT-3 were found to have moral biases [176] in terms of the Moral Foundation theory [58]”

    From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [176]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

  • Scaling Laws for Neural Language Models

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×

    “Recent work suggests that Large Language Model (LLM) performance will continue to scale with model and training data sizes [1].”
    Background

Flagged first, then by citations

Research works printed but not linked: 46

  1. Gati Aher, Rosa I. Arriaga, and Adam Tauman Kalai. 2023. Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Stud- ies. · DOI · Paper page
  2. Milad Alshomary, Wei-Fan Chen, Timon Gurcke, and Henning Wachsmuth. 2021. Belief-based Generation of Argumentative Claims. In Proceedings of the 16th Conference of the European Chapter of the Associ- ation for Computational Linguistics: Main Volume, pages 224-233, Online. Association for Computa- tional Linguistics. · DOI
  3. Milad Alshomary and Henning Wachsmuth. 2021. To- ward audience-aware argument generation. Patterns, 2(6):100253. · DOI
  4. Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, Joshua R. Gubler, Christopher Rytting, and David Wingate. 2023. Out of One, Many: Using Language Mod- els to Simulate Human Samples. Political Analysis, pages 1-15. · DOI · Paper page
  5. Arnav Arora, Lucie-aimee Kaffee, and Isabelle Augen- stein. 2023. Probing pre-trained language models for cross-cultural differences in values. In Proceedings of the First Workshop on Cross-Cultural Considera- tions in NLP (C3NLP), pages 114-130, Dubrovnik, Croatia. Association for Computational Linguistics.
  6. Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On Meaning, Form, and Understand- ing in the Age of Data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185-5198, Online. Association for Computational Linguistics. · DOI · Paper page
  7. Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dal- las Card, Rodrigo Castellon, Niladri Chatterji, An- nie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Lev- ent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christo- pher Potts, Aditi Raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Flo- rian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michi- hiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. 2022. On the Opportunities and Risks of Foundation Mod- els. · DOI · Paper page
  8. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877-1901.
  9. Don E. Davis, Kenneth Rice, Daryl R. Van Tongeren, Joshua N. Hook, Cirleen DeBlaere, Everett L. Wor- thington Jr., and Elise Choe. 2016. The moral foun- dations hypothesis does not replicate well in Black samples. Journal of Personality and Social Psychol- ogy, 110(4):e23-e30. · DOI
  10. Burak Dogruyol, Sinan Alper, and Onurcan Yilmaz. 2019. The five-factor model of the moral founda- tions theory is stable across WEIRD and non-WEIRD cultures. Personality and Individual Differences, 151:109547. · DOI · Paper page
  11. Maxim Egorov, Karianne Kalshoven, Armin Pircher Ver- dorfer, and Claudia Peus. 2020. It's a Match: Moral- ization and the Effects of Moral Foundations Congru- ence on Ethical and Unethical Leadership Perception. Journal of Business Ethics, 167(4):707-723. · DOI
  12. Denis Emelin, Ronan Le Bras, Jena D. Hwang, Maxwell Forbes, and Yejin Choi. 2021. Moral Stories: Situ- ated Reasoning about Norms, Intents, Actions, and their Consequences. In Proceedings of the 2021 Con- ference on Empirical Methods in Natural Language Processing, pages 698-718, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. · DOI
  13. Hubert Etienne. 2021. The dark side of the 'Moral Machine' and the fallacy of computational ethical decision-making for autonomous vehicles. Law, In- novation and Technology, 13(1):85-107. · DOI
  14. Matthew Feinberg and Robb Willer. 2015. From Gulf to Bridge: When Do Moral Arguments Facilitate Po- litical Influence? Personality and Social Psychology Bulletin, 41(12):1665-1681. · DOI
  15. Maxwell Forbes, Jena D. Hwang, Vered Shwartz, Maarten Sap, and Yejin Choi. 2020. Social chem- istry 101: Learning to reason about social and moral norms. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 653-670, Online. Association for Computational Linguistics. · DOI · Paper page
  16. Kathleen C. Fraser, Svetlana Kiritchenko, and Esma Balkir. 2022. Does Moral Code have a Moral Code? Probing Delphi's Moral Philosophy. In Proceedings of the 2nd Workshop on Trustworthy Natural Lan- guage Processing (TrustNLP 2022), pages 26-42, Seattle, U.S.A. Association for Computational Lin- guistics. · DOI
  17. Jeremy Frimer. 2019. Moral Foundations Dictionary 2.0. · DOI
  18. Jeremy A. Frimer. 2020. Do liberals and conservatives use different moral languages? Two replications and six extensions of Graham, Haidt, and Nosek's (2009) moral text analysis. Journal of Research in Personal- ity, 84:103906. · DOI
  19. Jesse Graham, Jonathan Haidt, and Brian A. Nosek. 2009. Liberals and conservatives rely on different sets of moral foundations. Journal of Personality and Social Psychology, 96(5):1029-1046. · DOI · Paper page
  20. Jesse Graham, Brian A. Nosek, Jonathan Haidt, Ravi Iyer, Spassena Koleva, and Peter H. Ditto. 2011. Map- ping the Moral Domain. Journal of personality and social psychology, 101(2):366-385. · DOI · Paper page
  21. Craig A. Harper and Darren Rhodes. 2021. Reanalysing the factor structure of the moral foundations ques- tionnaire. The British Journal of Social Psychology, 60(4):1303-1329. · DOI
  22. Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021. Aligning AI with shared human values. In 9th International Conference on Learning Represen- tations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
  23. Geert Hofstede. 2001. Culture's Recent Consequences: Using Dimension Scores in Theory and Research. International Journal of Cross Cultural Management, 1(1):11-17. · DOI
  24. Frederic R. Hopp, Jacob T. Fisher, Devin Cornell, Richard Huskey, and René Weber. 2021. The ex- tended Moral Foundations Dictionary (eMFD): Devel- opment and applications of a crowd-sourced approach to extracting moral intuitions from text. Behavior Re- search Methods, 53(1):232-246. · DOI
  25. Srinivasan Iyer, Xi Victoria Lin, Ramakanth Pasunuru, Todor Mihaylov, Daniel Simig, Ping Yu, Kurt Shuster, Tianlu Wang, Qing Liu, Punit Singh Koura, Xian Li, Brian O'Horo, Gabriel Pereyra, Jeff Wang, Christo- pher Dewan, Asli Celikyilmaz, Luke Zettlemoyer, and Ves Stoyanov. 2023. OPT-IML: Scaling Lan- guage Model Instruction Meta Learning through the Lens of Generalization. · DOI · Paper page
  26. Hang Jiang, Doug Beeferman, Brandon Roy, and Deb Roy. 2022. CommunityLM: Probing partisan world- views from language models. In Proceedings of the 29th International Conference on Computational Lin- guistics, pages 6818-6826, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
  27. Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ro- nan Le Bras, Jenny Liang, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jon Borchardt, Saadia Gabriel, Yulia Tsvetkov, Oren Etzioni, Maarten Sap, Regina Rini, and Yejin Choi. 2021. Can Machines Learn Morality? The Delphi Experiment. · DOI · Paper page
  28. Zhijing Jin, Sydney Levine, Fernando Gonzalez Adauto, Ojasv Kamal, Maarten Sap, Mrinmaya Sachan, Rada Mihalcea, Josh Tenenbaum, and Bernhard Schölkopf. 2022. When to make exceptions: Exploring language models as accounts of human moral judgment. In NeurIPS.
  29. Kristen Johnson and Dan Goldwasser. 2018. Classifi- cation of Moral Foundations in Microblog Political Discourse. In Proceedings of the 56th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 720-730, Melbourne, Australia. Association for Computational Linguistics. · DOI
  30. Rabia I. Kodapanakkal, Mark J. Brandt, Christoph Kogler, and Ilja van Beest. 2022. Moral Frames Are Persuasive and Moralize Attitudes; Nonmoral Frames Are Persuasive and De-Moralize Attitudes. Psycho- logical Science, 33(3):433-449. · DOI
  31. Andrew Luttrell, Aviva Philipp-Muller, and Richard E. Petty. 2019. Challenging Moral Attitudes With Moral Messages. Psychological Science, 30(8):1136-1150. · DOI
  32. Ece Çigdem Mutlu, Toktam Oghaz, Ege Tütüncüler, and Ivan Garibay. 2020. Do Bots Have Moral Judgement? The Difference Between Bots and Humans in Moral Rhetoric. In 2020 IEEE/ACM International Confer- ence on Advances in Social Networks Analysis and Mining (ASONAM), pages 222-226. · DOI
  33. Matheus C. Pavan, Vitor G. Dos Santos, Alex G. J. Lan, Joao Martins, Wesley R. Santos, Caio Deutsch, Pablo B. Costa, Fernando C. Hsieh, and Ivandre Paraboni. 2020. Morality Classification in Natural Language Text. IEEE Transactions on Affective Com- puting, pages 1-1. · DOI · Paper page
  34. Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Ben Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain, Dustin Li, Eli Tran-Johnson, Guro Khun- dadze, Jackson Kernion, James Landis, Jamie Kerr, Jared Mueller, Jeeyoon Hyun, Joshua Landau, Ka- mal Ndousse, Landon Goldberg, Liane Lovitt, Mar- tin Lucas, Michael Sellitto, Miranda Zhang, Neerav Kingsland, Nelson Elhage, Nicholas Joseph, Noemí Mercado, Nova DasSarma, Oliver Rausch, Robin Larson, Sam McCandlish, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Brown, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Jack Clark, Samuel R. Bowman, Amanda Askell, Roger Grosse, Danny Hernandez, Deep Ganguli, Evan Hubinger, Nicholas Schiefer, and Jared Kaplan. 2022. Discover- ing Language Model Behaviors with Model-Written Evaluations. · DOI · Paper page
  35. Ming Qian, Jaye Laguardia, and Davis Qian. 2021. Morality Beyond the Lines: Detecting Moral Senti- ment Using AI-Generated Synthetic Context. In Arti- ficial Intelligence in HCI, Lecture Notes in Computer Science, pages 84-94, Cham. Springer International Publishing. · DOI
  36. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  37. Shamik Roy, Nishanth Sridhar Nakshatri, and Dan Gold- wasser. 2022. Towards Few-Shot Identification of Morality Frames using In-Context Learning. In Proceedings of the Fifth Workshop on Natural Lan- guage Processing and Computational Social Science (NLP+CSS), pages 183-196, Abu Dhabi, UAE. Asso- ciation for Computational Linguistics.
  38. Christopher Suhler and Pat Churchland. 2011. Can Innate, Modular "Foundations" Explain Morality? Challenges for Haidt's Moral Foundations Theory. Journal of cognitive neuroscience, 23:2103-16; dis- cussion 2117. · DOI
  39. Zeerak Talat, Hagen Blix, Josef Valvoda, Maya Indira Ganesh, Ryan Cotterell, and Adina Williams. 2022. On the Machine Learning of Ethical Judgments from Natural Language. In Proceedings of the 2022 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies, pages 769-779, Seattle, United States. Association for Computational Linguistics. · DOI
  40. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. LLaMA: Open and Efficient Foundation Language Models. · DOI · Paper page
  41. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems, volume 30.
  42. Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, Courtney Biles, Sasha Brown, Zac Kenton, Will Hawkins, Tom Stepleton, Abeba Birhane, Lisa Anne Hendricks, Laura Rimell, William Isaac, Julia Haas, Sean Legassick, Geoffrey Irving, and Iason Gabriel. 2022. Taxonomy of Risks posed by Language Mod- els. In 2022 ACM Conference on Fairness, Account- ability, and Transparency, FAccT '22, pages 214-229, New York, NY, USA. Association for Computing Ma- chinery. · DOI · Paper page
  43. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtow- icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Hug- gingFace's Transformers: State-of-the-art Natural Language Processing. · DOI · Paper page
  44. Bilge Yalçındag, Türker Özkan, Sevim Cesur, Onurcan Yilmaz, Beyza Tepe, Zeynep Ecem Piyale, Ali Furkan Biten, and Diane Sunar. 2019. An Investigation of Moral Foundations Theory in Turkey Using Different Measures. Current Psychology, 38(2):440-457. · DOI
  45. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mi- haylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. OPT: Open Pre- trained Transformer Language Models. · DOI · Paper page
  46. Caleb Ziems, Jane Yu, Yi-Chia Wang, Alon Halevy, and Diyi Yang. 2022. The moral integrity corpus: A benchmark for ethical dialogue systems. In Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3755-3773, Dublin, Ireland. Association for Computational Linguistics. · DOI

Cited by

Sorted by citations

Metadata

Authors
Simmons, G.
Venue
arXiv
Publisher
arXiv
Published
2022
DOI
10.48550/arxiv.2209.12106
arXiv
2209.12106
References
1
Type
Preprint
Licence
creativecommons.org/licenses/by-nc-nd/4.0/legalcode

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review