A Survey on Automatic Detection of Hate Speech in Text
Open accessThe DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index.
Abstract
The scientific study of hate speech, from a computer science point of view, is recent. This survey organizes and describes the current state of the field, providing a structured overview of previous approaches, including core algorithms, methods, and main features used. This work also discusses the complexity of the concept of hate speech, defined in many platforms and contexts, and provides a unifying definition. This area has an unquestionable potential for societal impact, particularly in online communities and digital media platforms. The development and systematization of shared resources, such as guidelines, annotated datasets in multiple languages, and algorithms, is a crucial step in advancing the automatic detection of hate speech.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recent studies(Jurgens etal. 2019; Fortuna and Nunes 2018) that have looked into tackling and proposing subtle hate detection suggest the consideration of making all subtle forms of discrimination, even jokes, as hate speech since they negatively affect some people psychological even though they are considered harmless(Douglass etal. 2016).”
From Misogynoir: challenges in detecting intersectional hate (2022) · cites this paper as Fortuna and Nunes 2018
Citations
Cites
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,312
Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
Powers, D. M. W. (2020) · arXiv · doi:10.48550/arxiv.2010.16061 · Cited by 5,440
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Waseem, Z., & Hovy, D. (2016) · Proceedings of the NAACL Student Research Workshop · doi:10.18653/v1/n16-2013 · Cited by 1,655 · cited 1×
“Nevertheless, detecting hate speech is a very important task, because it is connected with actual hate crimes [42, 65, 77], and automatic hate speech detection in text can also provide data about this phenomenon.”
BackgroundA Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A., & Wiegand, M. (2017) · Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media · doi:10.18653/v1/w17-1101 · Cited by 1,395 · cited 4×
“A problem that is referred in hate speech detection is that sentences must be classified and not words [66].”
BackgroundInfluentialAbusive Language Detection in Online User Content
Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., & Chang, Y. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883062 · Cited by 1,122 · cited 9×
“[10], abusive language [58], discrimination [69], profanity [23], toxicity [43], flaming [35], extremism [55, 62], and radicalization [2].”
BackgroundMethodInfluentialDetecting Hate Speech on the World Wide Web
Warner, W. B., & Hirschberg, J. (2012) · Cited by 964
Deep Learning for Hate Speech Detection in Tweets
Badjatiya, P., Gupta, S., Gupta, M., & Varma, V. (2017) · Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion · doi:10.1145/3041021.3054223 · Cited by 849 · cited 2×
“N-grams are one of the most used techniques in hate speech automatic detection and related tasks [4, 9, 19, 33, 52, 58, 77].”
MethodStanford typed dependencies manual
de Marnee, M.-C., & Manning, C. D. (2010) · Cited by 717
Hate Speech Detection with Comment Embeddings
Djuric, N., Zhou, J., Morris, R., Grbovic, M., Radosavljevic, V., & Bhamidipati, N. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2740908.2742760 · Cited by 712 · cited 2×
“Some authors [25] use a paragraph2vec approach to classify language on user comments as abusive or clean and also to predict the central word in the message [25].”
Background
Research works printed but not linked: 18
- ACL., ALW1: 1st workshop on abusive language online.
- Burnap Peter, Proceedings of the Conference on the Internet, Policy 8 Politics. 1--18
- CHI2017. 2017. 2017 Workshop on online harassment. Retrieved from http://social.umd.edu/woh/. CHI2017. 2017. 2017 Workshop on online harassment. Retrieved from http://social.umd.edu/woh/.
- CONTACT., 2017, Interdisciplinary conference on Hate speech. Definitions, Interpretations and Practices.
- Davidson Thomas, 2017, Automated hate speech detection and the problem of offensive language. arXiv Preprint arXiv:1703.04009 · arXiv
- De Marneffe Marie-Catherine, 2008, Manning
- Pauw Guy De, 2016, First workshop on text analytics for cybersecurity and online safety (TA-COS
- Vigna Fabio Del, 2017, Proceedings of the 1st Italian Conference on Cybersecurity. 86--95
- Cambridge Dictionary. 2017. Profanity. Retrieved from https://dictionary.cambridge.org/dictionary/english/profanity. Cambridge Dictionary. 2017. Profanity. Retrieved from https://dictionary.cambridge.org/dictionary/english/profanity.
- Giblietto Fabio, 2015, Proceedings of the 4th Workshop on Making Sense of Microposts (#Microposts’14)
- Reporting Hate. 2017. Hate speech conference I.H.D.I.P. Retrieved from http://reportinghate.eu/contact2017/. Reporting Hate. 2017. Hate speech conference I.H.D.I.P. Retrieved from http://reportinghate.eu/contact2017/.
- Kwok Irene, 2013, Proceedings of the Association for the Advancement of Artificial Intelligence.
- Ross Bjorn, 2017, Measuring the reliability of hate speech annotations: The case of the european refugee crisis. arXiv Preprint arXiv:1701.08118 · arXiv
- Silva Leandro, 2016, Analyzing the targets of hate in online social media. arXiv Preprint arXiv:1603.07709 · arXiv
- Natalya Tarasova. 2016. Classification of Hate Tweets and Their Reasons using SVM. Master’s thesis. Uppsala Universitet. Natalya Tarasova. 2016. Classification of Hate Tweets and Their Reasons using SVM. Master’s thesis. Uppsala Universitet.
- Tulkens Stéphan, 2016, A dictionary-based approach to racism detection in dutch social media. arXiv Preprint arXiv:1608.08738 · arXiv
- Warner William, 2012, Proceedings of the Second Workshop on Language in Social Media. Association for Computational Linguistics, 19--26
- Yuan Shuhan, 2016, Proceedings of the International Conference on Extending Database Technology. 696--697
Cited by
Items 401–500 of 1,004
Sentir el odio: análisis de la gravedad percibida de los discursos de odio en la población española
Brändle, G., Zapatero, M. D. C., & Rebollo, M. A. P. (2024) · Revista Española de Sociología · doi:10.22325/fes/res.2024.219 · Cited by 15
Fonseca, A. J. F., Pontes, C., Moro, S., Batista, F., Ribeiro, R., Guerra, R., Carvalho, P. C. Q. D. F., Marques, C. M. V. A., & Silva, C. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e32246 · Cited by 15
“A comprehensive survey of this subject can be found in recent papers such as [26].”
BackgroundTen Natural Language Processing Tasks with Generative Artificial Intelligence
Golec, J., & Hachaj, T. (2025) · Applied Sciences · doi:10.3390/app15169057 · Cited by 19
Duluth at SemEval-2019 Task 6: Lexical Approaches to Identify and Categorize Offensive Tweets
Ted Pedersen, A. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2106 · Cited by 15
“Detecting such language has proven to be a challenging problem, at least in part because it remains difficult to make distinctions between the casual and even friendly use of profanity versus more serious forms of offensive language (Fortuna and Nunes, 2018; Schmidt and Wiegand, 2017).”
BackgroundYou too Brutus! Trapping Hateful Users in Social Media: Challenges, Solutions & Insights
Das, M., Saha, P., Dutt, R., Goyal, P., Mukherjee, A., & Mathew, B. (2021) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3465336.3475106 · Cited by 24
“Hate speech detection: Hate speech is a complex phenomenon, intrinsically associated to relationships among groups, and also relying on linguistic nuances [10].”
BackgroundA Framework to Understand Attitudes towards Immigration through Twitter
Freire-Vidal, Y., Graells-Garrido, E., & Rowe, F. (2021) · Applied Sciences · doi:10.3390/app11209689 · Cited by 18
“When the analysis is about groups of people, Twitter data highlights its potential use in the detection of hate speech [49], i.”
Deep Learning for Hate Speech Detection in social media
Kumar, A., Tyagi, V., & Das, S. (2021) · IEEE International Conference on Computing, Power and Communication Technologies (IC2PCT) · doi:10.1109/gucon50781.2021.9573687 · Cited by 15
Early Prediction of Hate Speech Propagation
Lin, K., Lee, R. K.-W., Gao, W. G., & Peng, W.-C. (2021) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw53433.2021.00126 · Cited by 23
A Survey of Recent Neural Network Models on Code-Mixed Indian Hate Speech Data
Dowlagar, S., & Mamidi, R. (2021) · Forum for Information Retrieval Evaluation · doi:10.1145/3503162.3503168 · Cited by 17
“In the survey by Paula Fortuna and Sérgio Nunes [14], the authors present detailed definitions of hate speech on various social media websites such as Youtube, Facebook, Twitter.”
BackgroundAn actor-based approach to understanding radical right viral tweets in the UK
Sprejer, L., Margetts, H. Z., Oliveira, K. A., O’Sullivan, D. J. P., & Vidgen, B. (2022) · Journal of Policing Intelligence and Counter Terrorism · doi:10.1080/18335330.2022.2086440 · Cited by 15
Gong, H., Valido, A., Ingram, K. M., Fanti, G. C., Bhat, S., & Espelage, D. L. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i17.17738 · Cited by 20
Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hate
Botelho, A., Hale, S. A., & Vidgen, B. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.166 · Cited by 24
Merging datasets for hate speech classification in Italian
Fortuna, P., Bonavita, I., & Nunes, S. (2018) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.4752 · Cited by 15
“Two surveys summarize previous research and conclude that the approaches rely frequently on Machine Learning and classification (Schmidt and Wiegand, 2017; Fortuna and Nunes, 2018).”
MethodSelf-citationBenito, D., Araque, Ó., & Iglesias, C. Á. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2070 · Cited by 15
“Based on previous hate speech statements (Fortuna and Nunes, 2018; Schmidt and Wiegand, 2017), this phenomenon could be de-fined as offensive or humorist content in form of text, video, or images that attacks, diminishes, in-cites violence or hate against groups or individuals, based on actual or…”
BackgroundTowards Ethics by Design in Online Abusive Content DetectionPreprint
Kiritchenko, S., & Nejadgholi, I. (2020) · arXiv (Cornell University) · doi:10.4224/40002027 · Cited by 18
Wiegand, M., Eder, E., & Ruppenhofer, J. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.410 · Cited by 27
“detection follows a one-size-fits-all approach (Fortuna and Nunes, 2018).”
Quantifying the impact of context on the quality of manual hate speech annotation
Ljubešić, N., Mozetič, I., & Novak, P. K. (2022) · Natural Language Engineering · doi:10.1017/s1351324922000353 · Cited by 14
“…unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited. primarily because of its frequent use in the literature, we are aware of its terminological ambiguity and refer the interested reader to a nuanced discussion in (Fortuna and Nunes 2018).”
BackgroundGarcía-Díaz, J. A., Pan, R., & Valencia‐García, R. (2023) · Mathematics · doi:10.3390/math11245004 · Cited by 23
Exploring hate speech detection: challenges, resources, current research and future directions
Geetanjali, G., & Kumar, M. (2025) · Multimedia Tools and Applications · doi:10.1007/s11042-025-20716-2 · Cited by 20
“[47, 48].”
Hadan, H., Mogavi, R. H., Zhang-Kennedy, L., & Nacke, L. E. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2549073 · Cited by 17
“Even worse, AI can internalize and reproduce hate speech, offensive language, or violent incitement found online (Fortuna & Nunes, 2019; Gorwa et al., 2020; Shelby et al., 2023).”
Does Moral Code have a Moral Code? Probing Delphi’s Moral Philosophy
Fraser, K., Kiritchenko, S., & Balkır, E. (2022) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2022.trustnlp-1.3 · Cited by 40
Automatic hate speech detection in audio using machine learning algorithms
Imbwaga, J. L., Chittaragi, N. B., & Koolagudi, S. G. (2024) · International Journal of Speech Technology · doi:10.1007/s10772-024-10116-6 · Cited by 14
“Later on, in 2018, another review article discussed, in detail, the definition of hate speech and how it has evolved (Fortuna & Nunes, 2018).”
Italian Counter Narrative Generation to Fight Online Hate Speech
Chung, Y.-L., Tekiroğlu, S. S., & Guerini, M. (2020) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.8378 · Cited by 14
Hateful Memes Classification using Machine Learning
Badour, J., & Brown, J. A. (2021) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci50451.2021.9659896 · Cited by 14
Tawalbeh, S. K., Hammad, M., & AL-Smadi, M. (2020) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2020.semeval-1.269 · Cited by 14
The Drivers and Virality of Hate Speech OnlinePreprint
Ananthakrishnan, U. M., & Tucker, C. E. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3793801 · Cited by 15
Hate Speech Detection in Saudi Twittersphere: A Deep Learning Ap-proach
2020, T. 2. I. C. O. C. L., Al-Khalifa, H., & Alshalan, R. (2020) · Underline Science Inc. · doi:10.48448/wp7z-j094 · Cited by 14
Hate Speech Detection in Limited Data Contexts Using Synthetic Data Generation
Khullar, A., Nkemelu, D., Nguyen, V. C., & Best, M. L. (2023) · ACM Journal on Computing and Sustainable Societies · doi:10.1145/3625679 · Cited by 13
“The increase in hateful content online has motivated research in automatic approaches for detecting hate speech [25, 58].”
BackgroundAuditing Image-based NSFW Classifiers for Content Filtering
Leu, W., Nakashima, Y., & García, N. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658963 · Cited by 20
“While NSFW or toxicity detectors have been extensively studied in the natural language domain for text inputs [3, 13, 20, 42, 43], the efficacy of image-only NSFW classifiers and their limitations are not well-studied.”
BackgroundNative vs Non-native Language Prompting: A Comparative Analysis
Kmainasi, M. B., Khan, R., Shahroor, A. E., Bendou, B., Hasanain, M., & Alam, F. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0576-7_30 · Cited by 27
Yadav, A., Khan, F. A., & Singh, V. (2024) · Applied Sciences · doi:10.3390/app142311206 · Cited by 13
Gender bias detection on hate speech classification: an analysis at feature-level
Nascimento, F. R. S., Cavalcanti, G. D. C., & Costa‐Abreu, M. D. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10841-8 · Cited by 14
Lopez, H., & Kübler, S. (2025) · Discourse Context & Media · doi:10.1016/j.dcm.2024.100848 · Cited by 16
Paul, J., Mallick, S., Mitra, A., Roy, A., & Sil, J. (2025) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3713077 · Cited by 13
“Recent works, such as those by Fortuna and Nunes [23], use advanced DL architectures to perform multi-modal hate speech detection.”
Hate Speech Dataset from Turkish Tweets
Mayda, İ., Demir, Y. E., Dalyan, T., & Diri, B. (2021) · Innovations in Intelligent Systems and Applications Conference (ASYU) · doi:10.1109/asyu52992.2021.9599042 · Cited by 13
Interpretable and High-Performance Hate and Offensive Speech Detection
Babaeianjelodar, M., Prudhvi, G. P., Lorenz, S., Chen, K., Mondal, S., Dey, S., & Kumar, N. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21707-4_18 · Cited by 18
“Most of these models are based on supervised machine learning systems such as support vector machines, random forest, logistic regression and decision trees [10].”
MethodMitigating Harmful Content on Social Media Using an Interactive User Interface
Shahi, G. K., & Tsoplefack, W. K. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19097-1_34 · Cited by 13
Content Moderation for Evolving Policies using Binary Question Answering
Mullick, S. S., Bhambhani, M., Sinha, S., Mathur, A., Gupta, S., & Shah, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-industry.54 · Cited by 26
“(Fortuna and Nunes, 2018; MacAvaney et al., 2019).”
Feng, Y., & Kim, J. (2024) · Journal of Current Issues & Research in Advertising · doi:10.1080/10641734.2024.2328576 · Cited by 13
“Outside of academic disciplines, social media companies, such as Facebook, Twitter, and YouTube, have made individual efforts to define hate or harmful speech tailored to their own platform (Del Vigna et al. 2017; Fortuna and Nunes 2018).”
de Oliveira, A. B., Baptista, C. D. S., Firmino, A. A., & de Paiva, A. C. (2024) · ACM Symposium on Applied Computing · doi:10.1145/3605098.3635964 · Cited by 17
“Hate speech, characterized by expressions that endorse violence and target specific demographic groups based on physical appearance, religion, ethnicity, gender, and more, has attracted significant attention in computer research [13, 22].”
BackgroundHate Speech Detection with Generalizable Target-aware Fairness
Chen, T., Wang, D., Liang, X., Risius, M., Demartini, G., & Yin, H. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671821 · Cited by 20
“To prevent the negative socio-economical impact from hateful content, accurate algorithms for hate speech detection (HSD) have been heavily investigated by both industry practitioners [12] and research communities [14].”
BackgroundWhat Is Abusive Language? Integrating Different Views on Abusive Language for Machine Learning
Niemann, M., Riehle, D. M., Brunk, J., & Becker, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-39627-5_6 · Cited by 15
AbuseAnalyzer: Abuse Detection, Severity and Target Prediction for Gab Posts
Chandra, M., Pathak, A., Dutta, E., Jain, P., Gupta, M., Shrivastava, M., & Kumaraguru, P. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.552 · Cited by 13
Leveraging cross-platform data to improve automated hate speech detectionPreprint
Gallacher, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.04895 · Cited by 14
“Recent approaches using these techniques have shown promise (Fortuna & Nunes, 2018; Pitsilis, Ramampiaro, & Langseth, 2018; Rizoiu, Wang, Ferraro, & Suominen, 2019).”
The Unseen Targets of Hate: A Systematic Review of Hateful Communication Datasets
Yu, Z., Sen, I., Assenmacher, D., Samory, M., Fröhling, L., Dahn, C., Nozza, D., & Wagner, C. (2024) · Social Science Computer Review · doi:10.1177/08944393241258771 · Cited by 19
“Fortuna and Nunes (2018) provided an analysis of the status of hate speech by presenting a summary of approaches, covering algorithms, methodologies, and main features.”
Daouadi, K. E., Boualleg, Y., & Guehairia, O. (2024) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3674970 · Cited by 12
“This has spurred the text classiication community to undertake research focused on hate speech detection, with a particular emphasis on languages other than English [2].”
BackgroundYadav, A., & Singh, V. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3512573 · Cited by 16
Ünlü, A., Truong, S., Sawhney, N., Tammi, T., & Kotonen, T. (2025) · New Media & Society · doi:10.1177/14614448241312900 · Cited by 13
“…should reflect different types of speech, such as offensive, hateful, racist, and sexist, rather than collapsing them into a binary classification (Fortuna and Nunes, 2019) or adding additional measurements such as offensive-ness (Roß et al., 2016) and aggressiveness (Basile et al., 2019; Poletto…”
Wirz, D. S., & Blassnig, S. (2025) · Information Communication & Society · doi:10.1080/1369118x.2025.2461646 · Cited by 12
“Fortuna and Nunes (2019) define hate speech based on four characteristics: 1) it refers to specific individuals or groups, 2) intends to incite violence or hatred, 3) attacks or denigrates, and 4) can be subtle or humorous.”
Pan, R., García-Díaz, J. A., & Valencia‐García, R. (2025) · Computer Standards & Interfaces · doi:10.1016/j.csi.2025.103990 · Cited by 14
UVA Wahoos at SemEval-2019 Task 6: Hate Speech Identification using Ensemble Machine Learning
Ramakrishnan, M., Zadrozny, W., & Tabari, N. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2141 · Cited by 12
“Papers published in the last two years include the surveys by Schmidt and Wiegand (2017) and Fortuna and Nunes (2018) where the authors extract features from the text like sentiment, linguistic features, utilize different lexical resources to tag an offensive tweet, and another paper by Davidson et…”
BackgroundThe Titans at SemEval-2019 Task 5: Detection of hate speech against immigrants and women in Twitter
Garain, A., & Basu, A. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2088 · Cited by 12
“Papers which have been published in the last two years include the surveys by (Schmidt and Wie-gand, 2017) and (Fortuna and Nunes, 2018), the paper by (Davidson et al., 2017) presenting the Hate Speech Detection dataset used in (Malmasi and Zampieri, 2017) and a few other recent papers such as…”
MethodOnline Troll Reviewer Detection Using Deep Learning Techniques
Al-Adhaileh, M. H., Aldhyani, T. H. H., & Alghamdi, A. D. (2022) · Applied Bionics and Biomechanics · doi:10.1155/2022/4637594 · Cited by 12
RetractedAre you a hero or a villain? A semantic role labelling approach for detecting harmful memes.
Fharook, S., Ahmed, S. S., Rithika, G., Budde, S. S., Saumya, S., & Biradar, S. (2022) · doi:10.18653/v1/2022.constraint-1.3 · Cited by 12
“People create offensive content and aggressively spread it over social media (P. Fortuna, 2018; T. Davidson, 2017).”
BackgroundKhorramrouz, A., Dutta, S., & KhudaBukhsh, A. R. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/667 · Cited by 19
Evaluation of Different Machine Learning and Deep Learning Techniques for Hate Speech Detection
Shawkat, N., Saquer, J. M., & Shatnawi, H. (2024) · ACM Southeast Regional Conference (ACMSE) · doi:10.1145/3603287.3651218 · Cited by 14
“NLP techniques and models are used in existing research to identify and categorize abusive language that targets a person based on their race, religion, ethnicity, national origin, disability, sex, sexual orientation, or gender identity [11, 20, 29].”
BackgroundMethodInfluentialFrom hate to harmony: Leveraging large language models for safer speech in times of COVID-19 crisis
Chao, A. F. Y., Wang, C.-S., Li, B.-Y., & Chen, H.-Y. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e35468 · Cited by 15
“Building upon prior studies that suggest hate speech falls under the umbrella of abusive language [40,28,26,41], this study adopts this correlation for classification purposes.”
BackgroundMethodResultInfluentialWiedemann, G., Ruppert, E., & Biemann, C. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2137 · Cited by 12
“Two recent survey papers, Schmidt and Wiegand (2017) and Fortuna and Nunes (2018), summarize the current state of the art in offensive language detection and related tasks such as hate speech or abusive language detection.”
BackgroundLearning to combine classifiers outputs with the transformer for text classification
Bugueño, M., & Mendoza, M. (2020) · Intelligent Data Analysis · doi:10.3233/ida-200007 · Cited by 13
“For more details on all hate speech detection variants, the reader is recommended to review the Fortuna and Nunes survey [21].”
Machine Learning Approach for the Detection of Hate Speech in Sinhala Unicode Text
Samarasinghe, S. W. A. M. D., Meegama, R. G. N., & Punchimudiyanse, M. (2020) · International Conference on Advances in ICT for Emerging Regions · doi:10.1109/icter51097.2020.9325493 · Cited by 12
“…hate against groups, based on specific characteristics such as physical appearance, religion, descent, national or ethnic origin, sexual orientation, gender identity or other and it can occur with different linguistic styles, even in subtle forms or when humour is used” [9]. levels in hate speech.”
BackgroundMultilingual Counter Narrative Type Classification
Chung, Y.-L., Guerini, M., & Agerri, R. (2021) · Workshop on Argument Mining (ArgMining) · doi:10.18653/v1/2021.argmining-1.12 · Cited by 12
Sustainable Hate: How Gab Built a Durable “Platform for the People”
Munn, L. (2022) · Canadian Journal of Communication · doi:10.22230/cjc.2022v47n1a4037 · Cited by 12
Shibly, F. H. A., Sharma, U., & Naleer, H. M. M. (2022) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3563775 · Cited by 12
“A (3) Numbers Removal (4) Special Characters Removal (5) Convert Emojis into Texts (6) Contraction Expansion (7) White Space Removal (8) Single Word Text Removal (9) Correct misspelled words (10) Duplicate Texts Removal (11) Drop empty lines”
BackgroundAltın, L. S. M., Serrano, À. B., & Saggion, H. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2120 · Cited by 12
“Previous research on the issue has been carried out with approaches from different perspectives such as abusive language (Waseem et al., 2017) (Chu et al., 2017), hate speech (Davidson et al., 2017) (Schmidt and Wie-gand, 2017) (Fortuna and Nunes, 2018) and cyber-bullying (Hee et al., 2018).”
BackgroundDetection of Hate and Offensive Speech in Text
Wani, A. H., Molvi, N. S., & Ashraf, S. I. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-44689-5_8 · Cited by 12
Hwang, T. (2017) · Cambridge University Press eBooks · doi:10.1017/9781108890960.012 · Cited by 13
Leveraging posts’ and authors’ metadata to spot several forms of abusive comments in Twitter
Casavantes, M., Aragón, M. E., González, L. C., & Montes-y-Gómez, M. (2023) · Journal of Intelligent Information Systems · doi:10.1007/s10844-023-00779-z · Cited by 11
“Most of the works on detecting abusive language have been modeled as text categorization problems (Schmidt & Wiegand, 2017; Fortuna & Nunes, 2018), that is, posts, comments, or documents are assigned to one or more predefined categories based solely on their content.”
BackgroundAnatomy of Hate Speech Datasets: Composition Analysis and Cross-dataset Classification
Guimarães, S., Kakizaki, G., Melo, P. D. F., Silva, M., Murai, F., Reis, J. C. S., & Benevenuto, F. (2023) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3603163.3609158 · Cited by 17
“accepted definition of hate speech [19].”
BackgroundToward Affective Empathy via Personalized Analogy Generation: A Case Study on Microaggression
Ju, H., Lee, J., Yang, S., Ok, J., & Hwang, I. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714122 · Cited by 16
“EmoEden [137] demonstrates the capability of LLMs to provide personalized emotional understanding and generate emotion-inducing contents to help children with autism learn specific emotions.”
Kim, S.-S., Kim, S., & Kim, H. (2025) · Journal of Management Information Systems · doi:10.1080/07421222.2025.2520173 · Cited by 11
“This erosion can subsequently lead to decreased user engagement and substantial advertiser attrition [22, 102].”
Plaza-del-Arco, F. M., Molina-González, M. D., Martín, M., & López, L. A. U. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2084 · Cited by 12
“Recently, a growing number of researchers have started to focus on studying the task of automatic detection of hateful language online (Fortuna and Nunes, 2018; Fersini et al., 2018b), moreover, some academic events and shared tasks have taken place focusing on this issue (Fersini et al., 2018a).”
BackgroundTECHSSN at SemEval-2020 Task 12: Offensive Language Detection Using BERT Embeddings
Sivanaiah, R., Suseelan, A., Rajendram, S. M., & Mirnalinee, T. T. (2020) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2020.semeval-1.291 · Cited by 11
Social Sensing in IoT Applications: A Review
Pandharipande, A. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3049714 · Cited by 12
“A structured review of methods for hate speech detection can be found in [56].”
BackgroundTHE PORTRAYAL OF DEFAMATION CASE DEFENDANT IN COURT VERDICT
Zifana, M., Lukmana, I., & Sudana, D. (2021) · Indonesian Journal of Applied Linguistics · doi:10.17509/ijal.v11i1.34672 · Cited by 11
Syntax and prejudice: ethically-charged biases of a syntax-based hate speech recognizer unveiled
Mastromattei, M., Ranaldi, L., Fallucchi, F., & Zanzotto, F. M. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.859 · Cited by 18
“Indeed, the major social networks and public entities give different definitions of it ( Fortuna & Nunes, 2018 ) and the boundary among hate speech comments and similar concepts (such as offensiveness, toxicity , aggressiveness , etc. ) is not obvious ( Poletto et al., 2021 ).”
Persily, N., & Tucker, J. A. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.002 · Cited by 11
UMUTeam@LT-EDI-ACL2022: Detecting homophobic and transphobic comments in Tamil
García-Díaz, J. A., Caparros-Laiz, C., & Valencia‐García, R. (2022) · Workshop on Language Technology for Equality, Diversity, Inclusion (LT-EDI) · doi:10.18653/v1/2022.ltedi-1.16 · Cited by 11
“For instance, the works described at (Fortuna and Nunes, 2018) Our research group has experience dealing with hate-speech.”
BackgroundAnti-Asian Hate Speech Detection via Data Augmented Semantic Relation Inference
Li, J., & Ning, Y. (2022) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v16i1.19319 · Cited by 15
A longitudinal study of the top 1% toxic Twitter profiles
Qayyum, H., Zhao, B. Z. H., Wood, I., Ikram, M., Kourtellis, N., & Kâafar, M. A. (2023) · ACM Web Science Conference (WebSci) · doi:10.1145/3578503.3583619 · Cited by 13
“Verbal misbehavior and toxicity on Online Social Networks (OSNs) are receiving a huge amount of attention in the research community, with efforts to identify [5, 7, 19, 60, 64], characterize [6, 8–10, 14], and automatically detect [3, 11, 17] online misbehavior, especially on Twitter.”
BackgroundMultimodal Deep Learning with Discriminant Descriptors for Offensive Memes Detection
Alzu’bi, A., Younis, L. B., Abuarqoub, A., & Hammoudeh, M. A. A. (2023) · Journal of Data and Information Quality · doi:10.1145/3597308 · Cited by 15
“One area of application for S-WoT is the detection of ofensive content on social media platforms, in recent years, there has been increasing concern over the spread of ofensive and hate speech on online social media [20].”
BackgroundConversation Modeling to Predict Derailment
Yuan, J., & Singh, M. P. (2023) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v17i1.22200 · Cited by 11
Jurgens, D., Seth, A., Sargent, J., Aghighi, A., & Geraci, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.616 · Cited by 19
Detection of Indonesian Hate Speech on Twitter Using Hybrid CNN-RNN
Riyadi, S., Andriyani, A. D., Masyhur, A. M., Damarjati, C., & Solihin, M. I. (2023) · International Conference on Information Technology and Computing (ICITCOM) · doi:10.1109/icitcom60176.2023.10442041 · Cited by 11
HarmonyNet: Navigating hate speech detection
Raza, S., & Chatrath, V. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100098 · Cited by 13
Dietrich, F. (2024) · AI and Ethics · doi:10.1007/s43681-024-00610-7 · Cited by 18
“A major problem for the detection of hate speech machine learning systems face is the variability and context-depen-dency of linguistic meaning [22, 23].”
AI_ML_NIT_Patna @ TRAC - 2: Deep Learning Approach for Multi-lingual Aggression Identification.
Kumari, K., & Singh, J. P. (2020) · Cited by 11
Oriola, O. (2020) · International Journal of Computer Applications · doi:10.5120/ijca2020920503 · Cited by 11
“It is better than simple bag of words that relies on the frequency of words[16].”
BackgroundCombating the hate speech in Thai textual memes
Mookdarsanit, L., & Mookdarsanit, P. (2021) · Indonesian Journal of Electrical Engineering and Computer Science · doi:10.11591/ijeecs.v21.i3.pp1493-1502 · Cited by 12
“Those spamming characters are filtered out using pyThaiNLP library [73], except for frequent misspelling words.”
BackgroundInfluentialRationalizing predictions by adversarial information calibration
Sha, L., Camburu, O.-M., & Lukasiewicz, T. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2022.103828 · Cited by 14
“This dataset contains 9,055 posts from Twitter [26, 34] and 11,093 posts from Gab [79, 86, 145].”
BackgroundThe Regulation of Content Moderation
Galli, F., Loreggia, A., & Sartor, G. (2023) · Law, governance and technology series · doi:10.1007/978-3-031-40516-7_5 · Cited by 17
Research Journey of Hate Content Detection From Cyberspace
Ghosal, S., & Jain, A. (2020) · Advances in business information systems and analytics book series · doi:10.4018/978-1-7998-4240-8.ch009 · Cited by 11
Toxic Speech Classification using Machine Learning Algorithms
Sumanth, P., Samiuddin, S., Jamal, K., Domakonda, S., & Shivani, P. (2022) · International Conference on Electronic Systems and Intelligent Computing (ICESIC) · doi:10.1109/icesic53714.2022.9783475 · Cited by 11
“Fortune and Nunes[3] investigated the motives for doing hate speech detection research and offered a comprehensive review of the field's future possibilities.”
BackgroundWhat Did You Learn To Hate? A Topic-Oriented Analysis of Generalization in Hate Speech Detection
Bourgeade, T., Chiril, P., Benamara, F., & Moriceau, V. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.254 · Cited by 11
“These issues have been broadly acknowledged in the relevant literature (Schmidt and Wiegand, 2017; Fortuna and Nunes, 2018; van Aken et al., 2018).”
BackgroundInfluentialText Style Transfer: A Review and Experiment Evaluation.Preprint
Hu, Z., Lee, R. K.-W., & Aggarwal, C. C. (2020) · arXiv (Cornell University) · Cited by 18
NDDSM: Novel Deep Decision-Support Model for Hate Speech Detection
Kumar, A., & Kumar, S. (2023) · SN Computer Science · doi:10.1007/s42979-023-02382-z · Cited by 10
“Out of these, six were obtained from Twitter [1, 2, 5, 6, 34, 35], two from YouTube/Facebook (Chung et al. [36] and Salminen et al. [37]), one from Gab (Kennedy et al. [139]), and the last one was from Reddit (Karrek et al. [39]) are shown in Table 1.”
Context-aware and expert data resources for Brazilian Portuguese hate speech detection
Vargas, F., Carvalho, I., Pardo, T. A. S., & Benevenuto, F. (2024) · Natural language processing. · doi:10.1017/nlp.2024.18 · Cited by 19
Measuring Online Hate on 4chan Using Pre-Trained Deep Learning Models
Bermudez-Villalva, A., Mehrnezhad, M., & Toreini, E. (2025) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2025.3549931 · Cited by 11
“However, 4chan imposes a maximum limit of 300 times a post can be bumped to the top, after that, the thread is moved to the archive [18].”
A Survey on Stereotype Detection in Natural Language Processing
Cignarella, A. T., Giachanou, A., & Lefever, E. (2025) · ACM Computing Surveys · doi:10.1145/3770754 · Cited by 22
“While Fortuna and Nunes [30] and Poletto et al. [62] provide foundational overviews of HaSpeeDe2 and its resources, they do not isolate stereotypes as a unique linguistic phenomenon.”
Iftikhar, U., Ali, S. F., Mustafa, G., Bahar, N., & Ishaq, K. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.3214 · Cited by 12
Violence Detection in Social Media-Review
Dikwatta, U., & Fernando, T. G. I. (2019) · Vidyodaya Journal of Science · doi:10.31357/vjs.v22i2.4385 · Cited by 10
Metadata
- Authors
- Fortuna, P., & Nunes, S.
- Affiliations
- INESC TEC (PT)
- Universidade do Porto (PT)
- Venue
- ACM Computing Surveys
- Published
- 2018-07-31
- Volume
- 51
- Issue
- 4
- Pages
- 1-30
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W2887782043
- Cited by
- 1,241
- References
- 52
- Citations by year
- 2026: 139 · 2025: 195 · 2024: 141 · 2023: 185 · 2022: 158 · 2021: 199 · 2020: 144 · 2019: 71 · 2018: 8
- FWCI
- 59.53
- Type
- journal article
- Language
- en
- Open access
- Locations
- ACM Computing Surveys (publishedVersion)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, other-oa)
- Funders
- FourEyes, a research line within project TEC4Growth—Pervasive Intelligence
- North Portugal Regional Operational Programme
- European Regional Development Fund
- Enhancers and Proofs of Concept with Industrial (Impact/NORTE-01- 0145-FEDER-000020)
- PORTUGAL 2020 Partnership Agreement
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- hate speech detection, text classification, definition of hate speech, annotated dataset, multilingual dataset, online communities, digital media platforms, social impact
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.