The DOI matches its Crossref record; 164 of the 173 references it cites were found in a registry or an index.
Offensive, aggressive, and hate speech analysis: From data-centric to human-centered approach
Open accessAbstract
Analysis of subjective texts like offensive content or hate speech is a great challenge, especially regarding annotation process. Most of current annotation procedures are aimed at achieving a high level of agreement in order to generate a high quality reference source. However, the annotation guidelines for subjective content may restrict the annotators’ freedom of decision making . Motivated by a moderate annotation agreement in offensive content datasets, we hypothesize that personalized approaches to offensive content identification should be in place. Thus, we propose two novel perspectives of perception: group-based and individual. Using demographics of annotators as well as embeddings of their previous decisions (annotated texts), we are able to train multimodal models (including transformer-based) adjusted to personal or community profiles. Based on the agreement of individuals and groups, we experimentally showed that annotator group agreeability strongly correlates with offensive content recognition quality. The proposed personalized approaches enabled us to create models adaptable to personal user beliefs rather than to agreed offensiveness understanding. Overall, our individualized approaches to offensive content classification outperform classic data-centric methods that generalize offensiveness perception and it refers to all six tested models. Additionally, we developed requirements for annotation procedures, personalization and content processing to make the solutions human-centered.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“At the same time, it is difficult to identify universal ground truth in tasks such as predicting emotions or offensiveness of text, especially in the personalized context [20, 21, 22].”
From ChatGPT: Jack of all trades, master of none (2023) · cites this paper as [20]
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
An analysis of variance test for normality (complete samples)
SHAPIRO, S. S., & WILK, M. B. (1965) · Biometrika · doi:10.1093/biomet/52.3-4.591 · Cited by 23,480
An Analysis of Variance Test for Normality (Complete Samples)
Shapiro, S. S., & Wilk, M. B. (1965) · Biometrika · doi:10.2307/2333709 · Cited by 23,469
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,759
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733
Research works printed but not linked: 5
- Breckheimer, 2001, A haven for hate: The foreign and domestic implications of protecting Internet hate speech under the first amendment, Southern California Law Review, 75, 1493
- Buczynski, A., & Wawer, A. (2008). Shallow parsing in sentiment analysis of product reviews. In Proceedings of the partial parsing workshop at LREC (vol. 2008) (pp. 14–18).
- Marcinczuk, M., Kocon, J., & Gawor, M. (2018). Recognition of named entities for Polish - comparison of deep learning and conditional random fields approaches. In Proceedings of the PolEval 2018 workshop (pp. 77–92).
- Ptaszyński, M., Masui, F., Janz, A., Kocoń, J., Piasecki, M., & Zaśko-Zielińska, M., et al. (2017). Three attempts in PolEval 2017 sentiment analysis task. In LTC’17 the 8th language & technology conference (pp. 117–121).
- Troszyński, 2017, Czy komputer rozpozna hejtera? Wykorzystanie uczenia maszynowego (ML) w jakościowej analizie danych. [Can a Computer Recognize Hate Speech? Machine Learning (ML) in Qualitative Data Analysis], Przegla̧d Socjologii Jakościowej, XIII, 62
Cited by
Sorted by citations
ChatGPT: Jack of all trades, master of none
Kocoń, J., Cichecki, I., Kaszyca, O., Kochanek, M., Szydło, D., Baran, J., Bielaniewicz, J., Gruza, M., Janz, A., Kanclerz, K., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101861 · Cited by 758
Self-citationTaxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764
“Mitigation is further complicated by the context dependency of what constitutes profoundly offensive speech [80, 92, 105].”
BackgroundEthical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477
Gongane, V. U., Munot, M. V., & Anuse, A. D. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00951-3 · Cited by 153
A Literature Review of Textual Hate Speech Detection Methods and Datasets
Alkomah, F., & Ma, X. (2022) · Information · doi:10.3390/info13060273 · Cited by 167
Metadata
- Authors
- Kocoń, J., Figas, A., Gruza, M., Puchalska, D., Kajdanowicz, T., & Kazienko, P.
- Affiliations
- Wrocław University of Science and Technology (PL)
- Corresponding author
- Kocoń, Jan
- Venue
- Information Processing & Management
- Publisher
- Elsevier BV
- Published
- 2021-09
- Volume
- 58
- Issue
- 5
- Pages
- 102643
- ISSN
- 0020-0271 (ISSN-L)
- 0306-4573
- OpenAlex
- W3171716568
- Cited by
- 103
- References
- 173
- Citations by year
- 2026: 9 · 2025: 17 · 2024: 21 · 2023: 32 · 2022: 17 · 2021: 7
- FWCI
- 6.81
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Information Processing & Management (publishedVersion, cc-by)
- Funders
- Narodowe Centrum Nauki
- Wroclaw University of Science and Technology
- Polish Ministry of Science and Higher Education
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
- Keywords
- offensive content detection, hate speech, annotation agreement, annotator demographics, personalized classification, individualized models, multimodal models, transformer models, human-centered AI, annotation guidelines
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
