Morality classification in natural language text
Abstract
The language employed by an individual when discussing topics of a moral nature (of the kind typically found in, e.g., social media) is revealing not only of the text affective contents itself, but also of the individual who wrote the text in the first place. Based on these observations, this work intends to illustrate how two kinds of morality-related information may be inferred from text by presenting a number of shallow and deep learning models of moral stance and moral foundations classification. In doing so, we introduce a novel corpus of texts labelled with moral foundation scores, and a novel approach to fine-grained, human-centric moral foundations classification that is, to the best of our knowledge, among the first NLP studies of this kind.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Past works achieved average F1 scores of 0.50 and below despite training and evaluating their classifier for each moral value.
“For example, past works (e.g. (83, 88)) achieved average F1 scores of 0.50 and below despite training and evaluating their classifier for each moral value.”
From Perils and opportunities in using large language models in psychological research (2024) · cites this paper as [88]
Cites
Liberals and conservatives rely on different sets of moral foundations.
Graham, J., Haidt, J., & Nosek, B. A. (2009) · Journal of Personality and Social Psychology · doi:10.1037/a0015141 · Cited by 4,560 · see citing works
Graham, J., Nosek, B. A., Haidt, J., Iyer, R., Koleva, S., & Ditto, P. H. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0021847 · Cited by 2,666 · see citing works
When Morality Opposes Justice: Conservatives Have Moral Intuitions that Liberals may not Recognize
Haidt, J., & Graham, J. (2007) · Social Justice Research · doi:10.1007/s11211-007-0034-z · Cited by 2,385 · see citing works
Intuitive ethics: how innately prepared intuitions generate culturally variable virtues
Haidt, J., & Joseph, C. (2004) · Daedalus · doi:10.1162/0011526042365555 · Cited by 1,982 · see citing works
SemEval-2016 Task 6: Detecting Stance in Tweets
Mohammad, S. M., Kiritchenko, S., Sobhani, P., Zhu, X., & Cherry, C. A. (2016) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s16-1003 · Cited by 850 · see citing works
Stance and Sentiment in Tweets
Mohammad, S. M., Sobhani, P., & Kiritchenko, S. (2017) · ACM Transactions on Internet Technology · doi:10.1145/3003433 · Cited by 449 · see citing works
Volkova, S., Shaffer, K., Jang, J. Y., & Hodas, N. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p17-2102 · Cited by 381 · see citing works
Moralization in social networks and the emergence of violence during protests
Mooijman, M., Hoover, J., Lin, Y., Ji, H., & Dehghani, M. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0353-0 · Cited by 229 · see citing works
Garten, J., Hoover, J., JOHNSON, K., Boghrati, R., Iskiwitch, C., & Dehghani, M. (2017) · Behavior Research Methods · doi:10.3758/s13428-017-0875-9 · Cited by 181 · see citing works
Twitter Language Use Reflects Psychological Differences between Democrats and Republicans
Sylwester, K., & Purver, M. (2015) · PLoS ONE · doi:10.1371/journal.pone.0137422 · Cited by 179 · see citing works
MITRE at SemEval-2016 Task 6: Transfer Learning for Stance Detection
Zarrella, G., & Marsh, A. (2016) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s16-1074 · Cited by 178 · see citing works
Moral foundations twitter corpus: a collection of 35k tweets annotated for moral sentiment
Hoover, J., Portillo-Wightman, G., Yeh, L., Havaldar, S., Davani, A. M., Lin, Y., Kennedy, B., Atari, M., Kamel, Z., Mendlen, M., et al. (2020) · Social Psychological and Personality Science · doi:10.1177/1948550619876629 · Cited by 177 · see citing works
Wan, W., Zhang, X., Liu, X., Chen, W., & Wang, T. (2016) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s16-1062 · Cited by 125 · see citing works
Classification of Moral Foundations in Microblog Political Discourse
Johnson, K., & Goldwasser, D. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1067 · Cited by 72 · see citing works
Hoover, J., JOHNSON, K., Boghrati, R., Graham, J., & Dehghani, M. (2018) · Collabra Psychology · doi:10.1525/collabra.129 · Cited by 67 · see citing works
Enhancing the Measurement of Social Effects by Capturing Morality
Rezapour, R., Shah, S. H., & Diesner, J. (2019) · Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) · doi:10.18653/v1/w19-1305 · Cited by 52 · see citing works
Acquiring Background Knowledge to Improve Moral Value Prediction
Lin, Y., Hoover, J., Portillo-Wightman, G., Park, C., Dehghani, M., & Ji, H. (2018) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2018.8508244 · Cited by 46 · see citing works
Friends and Enemies of Clinton and Trump: Using Context for Detecting Stance in Political Tweets
Lai, M., Farías, D. I. H., Patti, V., et al. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-62434-1_13 · Cited by 25 · see citing works
Moral Stance Recognition and Polarity Classification from Twitter and Elicited Text
dos Santos, W. R., & Paraboni, I. (2019) · doi:10.26615/978-954-452-056-4_123 · Cited by 22 · see citing works
Self-citationIncorporating Demographic Embeddings Into Language Understanding
Garten, J., Kennedy, B., Hoover, J., Sagae, K., & Dehghani, M. (2019) · Cognitive Science · doi:10.1111/cogs.12701 · Cited by 14 · see citing works
Modeling Behavioral Aspects of Social Media Discourse for Moral Classification
Johnson, K., & Goldwasser, D. (2019) · doi:10.18653/v1/w19-2112 · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Perils and opportunities in using large language models in psychological research
Abdurahman, S., Atari, M., Karimi-Malekabadi, F., Xue, M. J., Trager, J., Park, P. S., Golazizian, P., Omrani, A., & Dehghani, M. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae245 · Cited by 111 · see citing works
- Authors
- Pavan, M. C., dos Santos, V. G., Lan, A. G. J., Martins, J. T., dos Santos, W. R., Deutsch, C., da Costa, P. B., Hsieh, F. C., & Paraboni, I.
- Affiliations
- Universidade de São Paulo (BR)
- School of Arts, Sciences and Humanities, University of São Paulo, São Paulo, Brazil
- ORCID
- Venue
- IEEE Transactions on Affective Computing
- Published
- 2023-01-01
- Volume
- 14
- Issue
- 1
- Pages
- 857-863
- ISSN
- 1949-3045 (ISSN-L)
- 2371-9850
- OpenAlex
- W3097654613
- Cited by
- 30
- References
- 21
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 9 · 2023: 9 · 2022: 7 · 2021: 1
- FWCI
- 1.67
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Affective Computing (publishedVersion)
- Funders
- University of São Paulo (668/2018, 88887.475828/2020-00, 88887.475847/2020-00)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- moral foundations, social media discourse, emotional content, deep learning models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
