The DOI matches its DataCite record; the references it cites are not checked yet.
Do Multilingual Language Models Capture Differing Moral Norms?
Open accessAbstract
Massively multilingual sentence representations are trained on large corpora of uncurated data, with a very imbalanced proportion of languages included in the training. This may cause the models to grasp cultural values including moral judgments from the high-resource languages and impose them on the low-resource languages. The lack of data in certain languages can also lead to developing random and thus potentially harmful beliefs. Both these issues can negatively influence zero-shot cross-lingual model transfer and potentially lead to harmful outcomes. Therefore, we aim to (1) detect and quantify these issues by comparing different models in different languages, (2) develop methods for improving undesirable properties of the models. Our initial experiments using the multilingual model XLM-R show that indeed multilingual LMs capture moral norms, even with potentially higher human-agreement than monolingual ones. However, it is not yet clear to what extent these moral norms differ between languages.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Efforts to measure and instill human social norms in LLMs have so far yielded mixed results.
“efforts to measure and instill human social norms in LLMs have so far yielded mixed results (78, 79)”
From Emergent social conventions and collective bias in LLM populations (2025) · cites this paper as [79]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
The citation lists haven’t been fetched yet
Cited by 6
Research works printed but not linked: 15
- [1][1] Wissam Antoun, Fady Baly, and Hazem Hajj. Arabert: Transformer-based model for arabic language understanding. In LREC 2020 Workshop Language Resources and Evaluation Conference , may 2020. -
- [2][2] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, (NeurIPS) , 2020. -
- [3][3] Branden Chan, Stefan Schweter, and Timo M¨ oller. German's next language model. In Proceedings of the 28th International Conference on Computational Linguistics , pages 6788-6796, Barcelona, Spain (Online), December 2020. International Committee on Computational Linguistics. -
- [4][4] Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm´ an, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 8440-8451, Online, July 2020. Association for Computational Linguistics. -
- [5][5] Joe Davison, Joshua Feldman, and Alexander M. Rush. Commonsense knowledge mining from pretrained models. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, (EMNLPIJCNLP) . Association for Computational Linguistics, 2019. -
- [6][6] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. -
- [7][7] Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. Aligning AI with shared human values. In Proceedings of the International Conference on Learning Representations (ICLR) . OpenReview.net, 2021. -
- [8][8] Katharina H¨ ammerl, Jindˇ rich Libovick´ y, and Alexander Fraser. Combining static and contextualised multilingual embeddings. In Findings of ACL , 2022. -
- [9][9] Sophie Jentzsch, Patrick Schramowski, Constantin A. Rothkopf, and Kristian Kersting. Semantics derived automatically from language corpora contain human-like moral choices. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (AIES) , pages 37-44, 2019. -
- [10][10] Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Maxwell Forbes, Jon Borchardt, Jenny Liang, Oren Etzioni, Maarten Sap, and Yejin Choi. Delphi: Towards machine ethics and norms. CoRR , abs/2110.07574, 2021. - · arXiv
- [11][11] Jindˇ rich Libovick´ y, Rudolf Rosa, and Alexander Fraser. On the language neutrality of pre-trained multilingual representations. In Findings of EMNLP , November 2020. -
- [12][12] Fabio Petroni, Tim Rockt¨ aschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . Association for Computational Linguistics, 2019. -
- [13][13] Nils Reimers and Iryna Gurevych. Making monolingual sentence embeddings multilingual using knowledge distillation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 4512-4525, Online, November 2020. Association for Computational Linguistics. -
- [16][16] Milan Straka, Jakub N´ aplava, Jana Strakov´ a, and David Samuel. Robeczech: Czech roberta, a monolingual contextualized language representation model. In Kamil Ekˇ stein, Frantiˇ sek P´ artl, and Miloslav Konop´ ık, editors, Text, Speech, and Dialogue , pages 197-209, Cham, 2021. Springer International Publishing. -
- [17][17] Peter West, Chandra Bhagavatula, Jack Hessel, Jena D. Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. Symbolic knowledge distillation: from general language models to commonsense models. CoRR , abs/2110.07178, 2021. · arXiv
Metadata
- Authors
- Hämmerl, K., Deiseroth, B., Schramowski, P., Libovický, J., Fraser, A., & Kersting, K.
- Affiliations
- Ludwig-Maximilians-Universität München (DE)
- Technische Universität Darmstadt (DE)
- Charles University (CZ)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2203.09904
- OpenAlex
- W4221146509
- Cited by
- 6
- References
- 0
- Citations by year
- 2025: 1 · 2023: 3 · 2022: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Linguistics and Language › Language and Culture
- Keywords
- multilingual language models, moral norms, cross-lingual transfer, low-resource languages, language imbalance, XLM-R, cultural bias, zero-shot transfer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
