Hate speech operationalization: a preliminary examination of hate speech indicators and their structure
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Hate speech should be tackled and prosecuted based on how it is operationalized. However, the existing theoretical definitions of hate speech are not sufficiently fleshed out or easily operable. To overcome this inadequacy, and with the help of interdisciplinary experts, we propose an empirical definition of hate speech by providing a list of 10 hate speech indicators and the rationale behind them (the indicators refer to specific, observable, and measurable characteristics that offer a practical definition of hate speech). A preliminary exploratory examination of the structure of hate speech, with the focus on comments related to migrants (one of the most reported grounds of hate speech), revealed that two indicators in particular, denial of human rights and promoting violent behavior, occupy a central role in the network of indicators. Furthermore, we discuss the practical implications of the proposed hate speech indicators—especially (semi-)automatic detection using the latest natural language processing (NLP) and machine learning (ML) methods. Having a set of quantifiable indicators could benefit researchers, human right activists, educators, analysts, and regulators by providing them with a pragmatic approach to hate speech assessment and detection.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
A network theory of mental disorders
Borsboom, D. (2017) · World Psychiatry · doi:10.1002/wps.20375 · Cited by 3,482 · cited 2×
“The four network theory principles for conceptualizing mental disorders have been summarized by Borsboom [54] and can be applied to hate speech contexts and the indicators we”
BackgroundAutomated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
A Survey on Automatic Detection of Hate Speech in Text
Fortuna, P., & Nunes, S. (2019) · ACM Computing Surveys · doi:10.1145/3232676 · Cited by 1,241 · cited 7×
“First, Fortuna and Nunes [10] identified four dimensions (that can be considered analogous to indicators) that allowed them to compare definitions of hate speech: (1) hate speech has specific targets, (2) hate speech is to incite violence or hate; (3) hate speech is to attack or diminish; and (4) hate speech can be ex…”
BackgroundInfluentialRobinaugh, D. J., Hoekstra, R. H. A., Toner, E. R., & Borsboom, D. (2020) · Psychological Medicine · doi:10.1017/s0033291719003404 · Cited by 861
Hate speech detection: Challenges and solutions
MacAvaney, S., Yao, H.-R., Yang, E., Russell, K., Goharian, N., & Frieder, O. (2019) · PLOS ONE · doi:10.1371/journal.pone.0221152 · Cited by 566
“Fake News” Is Not Simply False Information: A Concept Explication and Taxonomy of Online Content
Molina, M. D., Sundar, S. S., Le, T., & Lee, D. (2021) · American Behavioral Scientist · doi:10.1177/0002764219878224 · Cited by 485 · cited 1×
“Previous studies have looked at indicators of cyberbullying [38], operational indicators of fake news [39],”
BackgroundLack of theory building and testing impedes progress in the factor and network literature
Fried, E. I. (2020) · Psychological Inquiry · doi:10.1080/1047840x.2020.1853461 · Cited by 317
Bilewicz, M., & Soral, W. (2020) · Political Psychology · doi:10.1111/pops.12670 · Cited by 283
What is hate speech? Part 1: The Myth of Hate
Brown, A. (2017) · Law and Philosophy · doi:10.1007/s10982-017-9297-1 · Cited by 246
Cited by
Kansok-Dusche, J., Ballaschk, C., Krause, N., Zeißig, A., Seemann-Herz, L., Wachs, S., & Bilz, L. (2023) · Trauma, Violence, & Abuse · doi:10.1177/15248380221108070 · Cited by 112
Towards a Definition of Hate Speech—With a Focus on Online Contexts
Hietanen, M., & Eddebo, J. (2023) · Journal of Communication Inquiry · doi:10.1177/01968599221124309 · Cited by 56
Hate speech in adolescents: A binational study on prevalence and demographic differences
Castellanos, M., Wettstein, A., Wachs, S., Kansok-Dusche, J., Ballaschk, C., Krause, N., & Bilz, L. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1076249 · Cited by 39
Srba, I., Razuvayevskaya, O., Leite, J. A., Moro, R., Schlicht, I. B., Tonelli, S., García, F. M., Lottmann, S. B., Teyssou, D., Porcellini, V., et al. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3770077 · Cited by 19
Self-citationSheombar, A. (2025) · Lecture Notes in Networks and Systems · doi:10.1007/978-3-031-93109-3_22 · Cited by 1
Metadata
- Authors
- Papcunová, J., Martončik, M., Fedáková, D., Kentoš, M., Bozogáňová, M., Srba, I., Moro, R., Pikuliak, M., Šimko, M., & Adamkovič, M.
- Venue
- Complex & Intelligent Systems
- Published
- 2021-10-11
- Volume
- 9
- Issue
- 3
- Pages
- 2827-2842
- ISSN
- 2198-6053 (ISSN-L)
- 2199-4536
- OpenAlex
- W3207934453
- Cited by
- 51
- References
- 10
- FWCI
- 2.82
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Hate speech operationalization: a preliminary examination of hate speech indicators and their structure. https://app.cochat.ai/papers/10.1007/s40747-021-00561-0
- The paper
- Papcunová, J., Martončik, M., Fedáková, D., Kentoš, M., Bozogáňová, M., Srba, I., Moro, R., Pikuliak, M., Šimko, M., & Adamkovič, M. (2023). Hate speech operationalization: a preliminary examination of hate speech indicators and their structure. Complex & Intelligent Systems. https://doi.org/10.1007/s40747-021-00561-0
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.