A Survey on Automatic Detection of Hate Speech in Text
Open accessThe DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index.
Abstract
The scientific study of hate speech, from a computer science point of view, is recent. This survey organizes and describes the current state of the field, providing a structured overview of previous approaches, including core algorithms, methods, and main features used. This work also discusses the complexity of the concept of hate speech, defined in many platforms and contexts, and provides a unifying definition. This area has an unquestionable potential for societal impact, particularly in online communities and digital media platforms. The development and systematization of shared resources, such as guidelines, annotated datasets in multiple languages, and algorithms, is a crucial step in advancing the automatic detection of hate speech.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recent studies(Jurgens etal. 2019; Fortuna and Nunes 2018) that have looked into tackling and proposing subtle hate detection suggest the consideration of making all subtle forms of discrimination, even jokes, as hate speech since they negatively affect some people psychological even though they are considered harmless(Douglass etal. 2016).”
From Misogynoir: challenges in detecting intersectional hate (2022) · cites this paper as Fortuna and Nunes 2018
Citations
Cites
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,312
Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
Powers, D. M. W. (2020) · arXiv · doi:10.48550/arxiv.2010.16061 · Cited by 5,440
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Waseem, Z., & Hovy, D. (2016) · Proceedings of the NAACL Student Research Workshop · doi:10.18653/v1/n16-2013 · Cited by 1,655 · cited 1×
“Nevertheless, detecting hate speech is a very important task, because it is connected with actual hate crimes [42, 65, 77], and automatic hate speech detection in text can also provide data about this phenomenon.”
BackgroundA Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A., & Wiegand, M. (2017) · Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media · doi:10.18653/v1/w17-1101 · Cited by 1,395 · cited 4×
“A problem that is referred in hate speech detection is that sentences must be classified and not words [66].”
BackgroundInfluentialAbusive Language Detection in Online User Content
Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., & Chang, Y. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883062 · Cited by 1,122 · cited 9×
“[10], abusive language [58], discrimination [69], profanity [23], toxicity [43], flaming [35], extremism [55, 62], and radicalization [2].”
BackgroundMethodInfluentialDetecting Hate Speech on the World Wide Web
Warner, W. B., & Hirschberg, J. (2012) · Cited by 964
Deep Learning for Hate Speech Detection in Tweets
Badjatiya, P., Gupta, S., Gupta, M., & Varma, V. (2017) · Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion · doi:10.1145/3041021.3054223 · Cited by 849 · cited 2×
“N-grams are one of the most used techniques in hate speech automatic detection and related tasks [4, 9, 19, 33, 52, 58, 77].”
MethodStanford typed dependencies manual
de Marnee, M.-C., & Manning, C. D. (2010) · Cited by 717
Hate Speech Detection with Comment Embeddings
Djuric, N., Zhou, J., Morris, R., Grbovic, M., Radosavljevic, V., & Bhamidipati, N. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2740908.2742760 · Cited by 712 · cited 2×
“Some authors [25] use a paragraph2vec approach to classify language on user comments as abusive or clean and also to predict the central word in the message [25].”
BackgroundBurnap, P., & Williams, M. L. (2015) · Policy & Internet · doi:10.1002/poi3.85 · Cited by 616 · cited 2×
“On one hand, on the internet, and social networks in particular, people are more likely to adopt an aggressive behavior because of the anonymity provided by these environments [8].”
BackgroundMethodAre You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter
Waseem, Z. (2016) · Proceedings of the First Workshop on NLP and Computational Social Science · doi:10.18653/v1/w16-5618 · Cited by 569 · cited 1×
“This is notorious in the low agreement found between annotators in the process of building new collections [65, 75].”
BackgroundModeling the Detection of Textual Cyberbullying
Dinakar, K., Reichart, R., & Lieberman, H. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i3.14209 · Cited by 512 · cited 5×
“The TF-IDF (term frequency-inverse document frequency) was also used in this kind of classification problems [24].”
BackgroundMethodInfluentialA Lexicon-based Approach for Hate Speech Detection
Gitari, N. D., Zhang, Z., Damien, H., & Long, J. (2015) · International Journal of Multimedia and Ubiquitous Engineering · doi:10.14257/ijmue.2015.10.4.21 · Cited by 502 · cited 5×
“Bearing in mind that hate speech has a negative polarity, authors have been computing the sentiment as a feature for hate speech detection [3, 19, 22, 31, 52, 53].”
BackgroundMethodInfluentialLocate the Hate: Detecting Tweets against Blacks
Kwok, I., & Wang, Y. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8539 · Cited by 446 · cited 1×
“Outside of context, these words bear no racial undertones of their own [50].”
BackgroundUs and them: identifying cyber hate on Twitter across multiple protected characteristics
Burnap, P., & Williams, M. L. (2016) · EPJ Data Science · doi:10.1140/epjds/s13688-016-0072-6 · Cited by 396 · cited 1×
“Automated techniques aim to programmatically classify text as hate speech, making its detection easier and faster for the ones that have the responsibility to protect the public [9, 65].”
BackgroundHate Me, Hate Me Not: Hate Speech Detection on Facebook.
Vigna, F. D., Cimino, A., Dell’Orletta⋄, F., Petrocchi, M., & Tesconi, M. (2017) · CNR ExploRA · Cited by 366
Yarowsky, D. (1994) · Proceedings of the 32nd annual meeting on Association for Computational Linguistics - · doi:10.3115/981732.981745 · Cited by 356
Adaptive least squares support vector machines filter for hand tremor canceling in microsurgery
Liu, Z., Wu, Q., Zhang, Y., & Philip Chen, C. L. (2011) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-011-0012-5 · Cited by 214
Do Characters Abuse More Than Words?
Mehdad, Y., & Tetreault, J. (2016) · Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w16-3638 · Cited by 198 · cited 1×
“Character N-gram features proved to be more predictive than token N-gram features, for the specific problem of abusive language detection [56].”
BackgroundClassifying racist texts using a support vector machine
Greevy, E., & Smeaton, A. F. (2004) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1008992.1009074 · Cited by 194 · cited 4×
“Another model similar to dictionaries is the bag-of-words [9, 33, 50].”
BackgroundMethodInfluentialImproved cyberbullying detection using gender information
Dadvar, M., de Jong, F. M. G., Ordelman, R., & Trieschnigg, D. (2012) · University of Twente Research Information · Cited by 191
“They were just making jokes”: Ethnic/racial teasing and discrimination among adolescents.
Douglass, S., Mirpuri, S., English, D., & Yip, T. (2016) · Cultural Diversity & Ethnic Minority Psychology · doi:10.1037/cdp0000041 · Cited by 160 · cited 1×
“Moreover, repeating these jokes can become a way of reinforcing racist attitudes [45] and, although they are considered harmless, they also have negative psychological effects for some people [26].”
BackgroundPerverts and sodomites: homophobia as hate speech in Africa
Reddy, V. (2002) · Southern African Linguistics and Applied Language Studies · doi:10.2989/16073610209486308 · Cited by 142
Using KNN and SVM Based One-Class Classifier for Detecting Online Radicalization on Twitter
Agarwal, S., & Sureka, A. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-14977-6_47 · Cited by 130 · cited 3×
“Radicalization Online radicalization is similar to the extremism concept and has been studied on multiple topics and domains, such as terrorism, anti-black communities, or nationalism [2].”
BackgroundMcNamee, L. G., Peterson, B. L., & Peña, J. F. (2010) · Communication Monographs · doi:10.1080/03637751003758227 · Cited by 121 · cited 3×
“However, these discourses focus other topics as well [55], such as new members recruitment, government and social media demonization of the in-group and persuasion [62].”
BackgroundBurnap, P., & Williams, M. L. (2014) · ORCA Online Research @Cardiff (Cardiff University) · Cited by 108
Anti-Discriminatory Practice: Equality, Diversity and Social Justice
Thompson, N. (2017) · Medical Entomology and Zoology · Cited by 100
Cyberbullying Detection and Classification Using Information Retrieval Algorithm
Nandhini, B. S., & Sheeba, J. I. (2015) · International Conference on Advanced Research in Computer Science Engineering & Technology (ICARCSET) · doi:10.1145/2743065.2743085 · Cited by 90
The problem of identifying misogynist language on Twitter (and other online social spaces)
Hewitt, S., Tiropanis, T., & Bokhove, C. (2016) · ACM Web Science Conference (WebSci) · doi:10.1145/2908131.2908183 · Cited by 80 · cited 2×
“In a study about sexism [41], a very simplistic approach was conducted.”
BackgroundThompson, N. (2016) · doi:10.1007/978-1-137-58666-7 · Cited by 79
Agarwal, S., & Sureka, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.04931 · Cited by 64 · cited 3×
“Characteristics of the message were also considered such as hashtags, mentions, retweets, URLs, number of tags, terms used in the tags, number of notes (reblog and like count), and link to multimedia content, such as image, video, or audio attached to the post [3].”
BackgroundMethodA Two Phase Deep Learning Model for Identifying Discrimination from Tweets
Yuan, S., Wu, X., & Xiang, Y. (2016) · Movebank · doi:10.5441/002/edbt.2016.92 · Cited by 63 · cited 1×
“Deep learning techniques are also recently being used in text classification and sentiment analysis, with high accuracy [83].”
MethodThe seriousness of ethnic jokes: Ethnic humor and social change in the Netherlands, 1995–2012
Kuipers, G., & van der Ent, B. (2016) · Humor - International Journal of Humor Research · doi:10.1515/humor-2016-0013 · Cited by 61
Prentice, S., Taylor, P. J., Rayson, P., Hoskins, A., & O’Loughlin, B. (2010) · Information Systems Frontiers · doi:10.1007/s10796-010-9272-y · Cited by 57 · cited 2×
“However, these discourses focus other topics as well [55], such as new members recruitment, government and social media demonization of the in-group and persuasion [62].”
BackgroundBartlett, J., Norrie, R., Patel, S., Rumpel, R., & Wibberley, S. (2014) · Cited by 57
New Classification Models for Detecting Hate and Violence Web Content
Liu, S., & Forss, T. (2015) · International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management · doi:10.5220/0005636704870495 · Cited by 50
Combining N-gram based Similarity Analysis with Sentiment Analysis in Web Content Classification
Liu, S., & Forss, T. (2014) · International Conference on Knowledge Discovery and Information Retrieval · doi:10.5220/0005170305300537 · Cited by 48
Cyberbullying Identification Using Participant-Vocabulary Consistency
Raisi, E., & Huang, B. (2016) · arXiv · doi:10.48550/arxiv.1606.08084 · Cited by 38 · cited 3×
“In a study about cyberbullying [63], this method is used to characterize the tendency of each user to harass or to be harassed, and the tendency of a key phrase to be indicative of harassment.”
BackgroundMethodInfluentialProceedings of the 17th Dutch-Belgian Information Retrieval WorkshopPreprint
Brandsen, A., Dirkson, A. R., Kraaij, W., Lamers, W., Verberne, S., de Vos, H., & Wiggers, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.04265 · Cited by 30
Text Classification Using Association Rules, Dependency Pruning and HyperonymizationPreprint
Haralambous, Y., & Lenca, P. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.7357 · Cited by 29 · cited 1×
“Rule-based methods do not involve learning and typically rely on a pre-compiled list or dictionary of subjectivity clues [36].”
BackgroundAnalysis of cyberbullying tweets in trending world events
Cortis, K., & Handschuh, S. (2015) · International Conference on Knowledge Management and Knowledge Technologies · doi:10.1145/2809563.2809605 · Cited by 27
Using a Semi-automatic Keyword Dictionary for Improving Violent Web Site Filtering
Guermazi, R., Hammami, M., & Hamadou, A. B. (2007) · International Conference on Signal-Image Technology & Internet-Based Systems (SITIS) · doi:10.1109/sitis.2007.137 · Cited by 22 · cited 2×
“Flaming Flaming are hostile, profane and intimidating comments that can disrupt participation in a community [35] Hate speech can occur in any context, whereas flaming is aimed toward a participant in the specific context of a discussion.”
BackgroundDETECTING OFFENSIVE LANGUAGE IN SOCIAL MEDIAS FOR PROTECTION OF ADOLESCENT ONLINE SAFETY
Chen, Y. (2011) · Cited by 16
Giglietto, F., & Lee, Y. (2015) · SOAS Research Online (SOAS University of London) · Cited by 15
Dashti, A., Al-Kandari, A. A., & Al-Abdullah, H. H. (2014) · Telematics and Informatics · doi:10.1016/j.tele.2014.08.007 · Cited by 12 · cited 1×
“It describes “Our” characteristics as superior to “Theirs,” which are inferior, undeserving, and incompatible [17].”
BackgroundMapping Racist Tweets in Response to President Obama's Re-election
Zook, M. (2012) · Cited by 11
Hate speech: A quality of service challenge
Oboler, A., & Connelly, K. (2014) · IEEE Conference on e-Learning, e-Management and e-Services (IC3e) · doi:10.1109/ic3e.2014.7081252 · Cited by 10 · cited 1×
“Social media companies provide a service [59]: They ease the communication between its users.”
BackgroundKompatsiaris, P. (2016) · Social Identities · doi:10.1080/13504630.2016.1207513 · Cited by 10 · cited 1×
“Moreover, repeating these jokes can become a way of reinforcing racist attitudes [45] and, although they are considered harmless, they also have negative psychological effects for some people [26].”
BackgroundAutomatic text categorisation of racist webpages
Greevy, E. (2004) · Dublin City University Open Access Institutional Repository (Dublin City University) · Cited by 9
Use of regular expressions for multi-lingual detection of hate speech in Kenya
Maloba, W. J. (2013) · SU+ Digital Repository (Strathmore University) · Cited by 7
Research works printed but not linked: 18
- ACL., ALW1: 1st workshop on abusive language online.
- Burnap Peter, Proceedings of the Conference on the Internet, Policy 8 Politics. 1--18
- CHI2017. 2017. 2017 Workshop on online harassment. Retrieved from http://social.umd.edu/woh/. CHI2017. 2017. 2017 Workshop on online harassment. Retrieved from http://social.umd.edu/woh/.
- CONTACT., 2017, Interdisciplinary conference on Hate speech. Definitions, Interpretations and Practices.
- Davidson Thomas, 2017, Automated hate speech detection and the problem of offensive language. arXiv Preprint arXiv:1703.04009 · arXiv
- De Marneffe Marie-Catherine, 2008, Manning
- Pauw Guy De, 2016, First workshop on text analytics for cybersecurity and online safety (TA-COS
- Vigna Fabio Del, 2017, Proceedings of the 1st Italian Conference on Cybersecurity. 86--95
- Cambridge Dictionary. 2017. Profanity. Retrieved from https://dictionary.cambridge.org/dictionary/english/profanity. Cambridge Dictionary. 2017. Profanity. Retrieved from https://dictionary.cambridge.org/dictionary/english/profanity.
- Giblietto Fabio, 2015, Proceedings of the 4th Workshop on Making Sense of Microposts (#Microposts’14)
- Reporting Hate. 2017. Hate speech conference I.H.D.I.P. Retrieved from http://reportinghate.eu/contact2017/. Reporting Hate. 2017. Hate speech conference I.H.D.I.P. Retrieved from http://reportinghate.eu/contact2017/.
- Kwok Irene, 2013, Proceedings of the Association for the Advancement of Artificial Intelligence.
- Ross Bjorn, 2017, Measuring the reliability of hate speech annotations: The case of the european refugee crisis. arXiv Preprint arXiv:1701.08118 · arXiv
- Silva Leandro, 2016, Analyzing the targets of hate in online social media. arXiv Preprint arXiv:1603.07709 · arXiv
- Natalya Tarasova. 2016. Classification of Hate Tweets and Their Reasons using SVM. Master’s thesis. Uppsala Universitet. Natalya Tarasova. 2016. Classification of Hate Tweets and Their Reasons using SVM. Master’s thesis. Uppsala Universitet.
- Tulkens Stéphan, 2016, A dictionary-based approach to racism detection in dutch social media. arXiv Preprint arXiv:1608.08738 · arXiv
- Warner William, 2012, Proceedings of the Second Workshop on Language in Social Media. Association for Computational Linguistics, 19--26
- Yuan Shuhan, 2016, Proceedings of the International Conference on Extending Database Technology. 696--697
Cited by
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764
“…create a range of harms, such as promoting social stereotypes that perpetuate the derogatory representation or unfair treat-ment of marginalised groups [22], inciting hate or violence [57], causing profound offence [199], or reinforcing social norms that exclude or marginalise identities [15, 58].”
BackgroundHate speech detection: Challenges and solutions
MacAvaney, S., Yao, H.-R., Yang, E., Russell, K., Goharian, N., & Frieder, O. (2019) · PLOS ONE · doi:10.1371/journal.pone.0221152 · Cited by 566
“’s definition is based on an analysis of the following characteristics from other definitions [8]:”
BackgroundHateXplain: A Benchmark Dataset for Explainable Hate Speech Detection
Mathew, B., Saha, P., Yimam, S. M., Biemann, C., Goyal, P., & Mukherjee, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i17.17745 · Cited by 513
Ethical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477
Racial Bias in Hate Speech and Abusive Language Detection Datasets
Davidson, T., Bhattacharya, D., & Weber, I. (2019) · Proceedings of the Third Workshop on Abusive Language Online · doi:10.18653/v1/w19-3504 · Cited by 414
“While hate speech and abusive language detection has become an important area for natural language processing research (Schmidt and Wiegand, 2017; Waseem et al., 2017; Fortuna and Nunes, 2018), there has been little work addressing the potential for these systems to be biased.”
Background
Metadata
- Authors
- Fortuna, P., & Nunes, S.
- Affiliations
- INESC TEC (PT)
- Universidade do Porto (PT)
- Venue
- ACM Computing Surveys
- Published
- 2018-07-31
- Volume
- 51
- Issue
- 4
- Pages
- 1-30
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W2887782043
- Cited by
- 1,241
- References
- 52
- Citations by year
- 2026: 139 · 2025: 195 · 2024: 141 · 2023: 185 · 2022: 158 · 2021: 199 · 2020: 144 · 2019: 71 · 2018: 8
- FWCI
- 59.53
- Type
- journal article
- Language
- en
- Open access
- Locations
- ACM Computing Surveys (publishedVersion)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, other-oa)
- Funders
- FourEyes, a research line within project TEC4Growth—Pervasive Intelligence
- North Portugal Regional Operational Programme
- European Regional Development Fund
- Enhancers and Proofs of Concept with Industrial (Impact/NORTE-01- 0145-FEDER-000020)
- PORTUGAL 2020 Partnership Agreement
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- hate speech detection, text classification, definition of hate speech, annotated dataset, multilingual dataset, online communities, digital media platforms, social impact
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.