A Survey on Automatic Detection of Hate Speech in Text
Open accessThe DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index.
Abstract
The scientific study of hate speech, from a computer science point of view, is recent. This survey organizes and describes the current state of the field, providing a structured overview of previous approaches, including core algorithms, methods, and main features used. This work also discusses the complexity of the concept of hate speech, defined in many platforms and contexts, and provides a unifying definition. This area has an unquestionable potential for societal impact, particularly in online communities and digital media platforms. The development and systematization of shared resources, such as guidelines, annotated datasets in multiple languages, and algorithms, is a crucial step in advancing the automatic detection of hate speech.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recent studies(Jurgens etal. 2019; Fortuna and Nunes 2018) that have looked into tackling and proposing subtle hate detection suggest the consideration of making all subtle forms of discrimination, even jokes, as hate speech since they negatively affect some people psychological even though they are considered harmless(Douglass etal. 2016).”
From Misogynoir: challenges in detecting intersectional hate (2022) · cites this paper as Fortuna and Nunes 2018
Citations
Cites
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,312
Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
Powers, D. M. W. (2020) · arXiv · doi:10.48550/arxiv.2010.16061 · Cited by 5,440
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Waseem, Z., & Hovy, D. (2016) · Proceedings of the NAACL Student Research Workshop · doi:10.18653/v1/n16-2013 · Cited by 1,655 · cited 1×
“Nevertheless, detecting hate speech is a very important task, because it is connected with actual hate crimes [42, 65, 77], and automatic hate speech detection in text can also provide data about this phenomenon.”
BackgroundA Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A., & Wiegand, M. (2017) · Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media · doi:10.18653/v1/w17-1101 · Cited by 1,395 · cited 4×
“A problem that is referred in hate speech detection is that sentences must be classified and not words [66].”
BackgroundInfluentialAbusive Language Detection in Online User Content
Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., & Chang, Y. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883062 · Cited by 1,122 · cited 9×
“[10], abusive language [58], discrimination [69], profanity [23], toxicity [43], flaming [35], extremism [55, 62], and radicalization [2].”
BackgroundMethodInfluentialDetecting Hate Speech on the World Wide Web
Warner, W. B., & Hirschberg, J. (2012) · Cited by 964
Deep Learning for Hate Speech Detection in Tweets
Badjatiya, P., Gupta, S., Gupta, M., & Varma, V. (2017) · Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion · doi:10.1145/3041021.3054223 · Cited by 849 · cited 2×
“N-grams are one of the most used techniques in hate speech automatic detection and related tasks [4, 9, 19, 33, 52, 58, 77].”
MethodStanford typed dependencies manual
de Marnee, M.-C., & Manning, C. D. (2010) · Cited by 717
Hate Speech Detection with Comment Embeddings
Djuric, N., Zhou, J., Morris, R., Grbovic, M., Radosavljevic, V., & Bhamidipati, N. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2740908.2742760 · Cited by 712 · cited 2×
“Some authors [25] use a paragraph2vec approach to classify language on user comments as abusive or clean and also to predict the central word in the message [25].”
Background
Research works printed but not linked: 18
- ACL., ALW1: 1st workshop on abusive language online.
- Burnap Peter, Proceedings of the Conference on the Internet, Policy 8 Politics. 1--18
- CHI2017. 2017. 2017 Workshop on online harassment. Retrieved from http://social.umd.edu/woh/. CHI2017. 2017. 2017 Workshop on online harassment. Retrieved from http://social.umd.edu/woh/.
- CONTACT., 2017, Interdisciplinary conference on Hate speech. Definitions, Interpretations and Practices.
- Davidson Thomas, 2017, Automated hate speech detection and the problem of offensive language. arXiv Preprint arXiv:1703.04009 · arXiv
- De Marneffe Marie-Catherine, 2008, Manning
- Pauw Guy De, 2016, First workshop on text analytics for cybersecurity and online safety (TA-COS
- Vigna Fabio Del, 2017, Proceedings of the 1st Italian Conference on Cybersecurity. 86--95
- Cambridge Dictionary. 2017. Profanity. Retrieved from https://dictionary.cambridge.org/dictionary/english/profanity. Cambridge Dictionary. 2017. Profanity. Retrieved from https://dictionary.cambridge.org/dictionary/english/profanity.
- Giblietto Fabio, 2015, Proceedings of the 4th Workshop on Making Sense of Microposts (#Microposts’14)
- Reporting Hate. 2017. Hate speech conference I.H.D.I.P. Retrieved from http://reportinghate.eu/contact2017/. Reporting Hate. 2017. Hate speech conference I.H.D.I.P. Retrieved from http://reportinghate.eu/contact2017/.
- Kwok Irene, 2013, Proceedings of the Association for the Advancement of Artificial Intelligence.
- Ross Bjorn, 2017, Measuring the reliability of hate speech annotations: The case of the european refugee crisis. arXiv Preprint arXiv:1701.08118 · arXiv
- Silva Leandro, 2016, Analyzing the targets of hate in online social media. arXiv Preprint arXiv:1603.07709 · arXiv
- Natalya Tarasova. 2016. Classification of Hate Tweets and Their Reasons using SVM. Master’s thesis. Uppsala Universitet. Natalya Tarasova. 2016. Classification of Hate Tweets and Their Reasons using SVM. Master’s thesis. Uppsala Universitet.
- Tulkens Stéphan, 2016, A dictionary-based approach to racism detection in dutch social media. arXiv Preprint arXiv:1608.08738 · arXiv
- Warner William, 2012, Proceedings of the Second Workshop on Language in Social Media. Association for Computational Linguistics, 19--26
- Yuan Shuhan, 2016, Proceedings of the International Conference on Extending Database Technology. 696--697
Cited by
Items 501–600 of 1,004
Attanasio, G., & Pastor, E. (2020) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.6807 · Cited by 13
“Many recent works in the NLP community show effective results in online monitoring of hate speech (Fortuna and Nunes, 2018) and misogynous contents (Pamungkas et al.”
BackgroundTEAM HUB@LT-EDI-EACL2021: Hope Speech Detection Based On Pre-trained Language Model
Huang, B., & Bai, Y. (2021) · Cited by 10
Offence Detection in Dravidian Languages Using Code-Mixing Index-Based Focal Loss
Tula, D., Ms, S., Reddy, V., Sahu, P., Doddapaneni, S., Potluri, P., Sukumaran, R., & Patwa, P. (2022) · SN Computer Science · doi:10.1007/s42979-022-01190-1 · Cited by 10
“Fortuna and Nunes, 2018 [14] discuss the nuances of hate speech and its potential for negative cultural effects, specifically in online communities and social media platforms.”
BackgroundMixed frequency composite indicators for measuring public sentiment in the EU
Mattera, R., Misuraca, M., Spano, M., & Scepi, G. (2022) · Quality & Quantity · doi:10.1007/s11135-022-01468-9 · Cited by 10
Real-Time Detection of COVID-19 Events From Twitter: A Spatial-Temporally Bursty-Aware Method
Fei, G., Cheng, Y., Ma, W., Chen, C., Wen, S., & Hu, G. (2022) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3169742 · Cited by 10
“Tweets usually contain incomplete and even wrongly structured sentences due to abbreviations, irregular expressions, abnormal words, and slang terms [36], [37].”
BackgroundFetahi, E., Hamiti, M., Susuri, A., Shehu, V., & Besimi, A. (2023) · Mediterranean Conference on Embedded Computing (MECO) · doi:10.1109/meco58584.2023.10155070 · Cited by 10
What role do social media play in the production and circulation of hate speech?
Longhi, J., Vernet, S., & Carey-Libbrecht, L. (2023) · Réseaux · doi:10.3917/res.241.0011 · Cited by 10
Modeling Users and Online Communities for Abuse Detection: A Position on Ethics and Explainability
Mishra, P., Yannakoudakis, H., & Shutova, E. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.287 · Cited by 14
“of the field (Schmidt and Wiegand, 2017; Fortuna and Nunes, 2018) for a detailed literature review.”
BackgroundVitiugin, F., Senarath, Y., & Purohit, H. (2021) · ACM Web Science Conference (WebSci) · doi:10.1145/3447535.3462495 · Cited by 16
“” [18] Some definitions above consider hate towards a group while others consider attacks on an individual.”
BackgroundUnified and Multilingual Author Profiling for Detecting Haters
Schlicht, I. B., & de Paula, A. F. M. (2021) · RiuNet (Universitat Politècnica de València) · doi:10.48550/arxiv.2109.09233 · Cited by 10
“AI systems are encouraged for easing the process and understanding the rationales behind hate speech dissemination [7, 8].”
BackgroundSahnan, D., Dahiya, S., Goel, V., Bandhakavi, A., & Chakraborty, T. (2021) · Proceedings · doi:10.1109/icdm51629.2021.00066 · Cited by 10
Reinforcement Guided Multi-Task Learning Framework for Low-Resource Stereotype Detection
Pujari, R., Oveson, E., Kulkarni, P., & Nouri, E. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.462 · Cited by 16
Ollagnier, A., Cabrio, E., & Villata, S. (2023) · Social Network Analysis and Mining · doi:10.1007/s13278-023-01061-4 · Cited by 10
“Several surveys describe the current state of the field providing a structured overview of existing datasets (Fortuna and Nunes 2018; Vidgen and Derczynski 2021; Poletto et al. 2021).”
BackgroundNejadgholi, I., Balkır, E., Fraser, K., & Kiritchenko, S. (2022) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2022.blackboxnlp-1.18 · Cited by 13
Procházka, O. (2019) · Signs and Society · doi:10.1086/704763 · Cited by 11
Recalibrating classifiers for interpretable abusive content detection
Vidgen, B., Hale, S. A., Staton, S., Melham, T., Margetts, H. Z., Kammar, O., & Szymczak, M. (2020) · Workshop on Natural Language Processing and Computational Social Science (NLP+CSS) · doi:10.18653/v1/2020.nlpcss-1.14 · Cited by 13
“Computational tools for automatically detecting and categorizing abusive online content are now widely used for content moderation, to enforce and monitor regulatory and legal standards, and to study the dynamics of online abuse (Williams, 2019; Vid-gen et al., 2019; Fortuna and Nunes, 2018).”
MethodDoostmohammadi, E., Sameti, H., & Saffar, A. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2110 · Cited by 10
“, and their methodology including (Schmidt and Wiegand, 2017) and (Fortuna and Nunes, 2018).”
MethodEmpirical Study of Text Augmentation on Social Media Text in VietnamesePreprint
Luu, S. T., Nguyen, K. V., & Nguyen, N. L.-T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.12319 · Cited by 10
“In recent years, the growth of hate speech has become a crime, not only face-to-face action but also online communication (Fortuna and Nunes, 2018).”
BackgroundMazumder, D., Kumar, A., & Patro, J. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3726866 · Cited by 11
“Further, a small fraction of these studies focused on code-mixed setups where the hate is uttered altering more than one language.”
Real-Time Multi-Lingual Hate and Offensive Speech Detection in Social Networks Using Meta-Learning
Prasad, D., Kadambari, K. V., Mukati, R., & Singariya, S. (2023) · IEEE Region 10 Conference (TENCON) · doi:10.1109/tencon58879.2023.10322364 · Cited by 9
“Since the majority of existing solutions [1], [2] and [3] are focused on the English language there is a need for models that can detect hate 1 Corresponding author : kadambari@nitw.ac.in speech in other regional languages.”
BackgroundLingualyzer: A computational linguistic tool for multilingual and multidimensional text analysis
Linders, G. M., & Louwerse, M. M. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02284-1 · Cited by 9
“…such as the truth value of political statements (Mihalcea & Strapparava, 2009; Rashkin et al., 2017), whether social media texts contains humor or irony (Barbieri & Saggion, 2014; Reyes et al., 2012) or hate speech (Fortuna & Nunes, 2018), and the readability of a text (McNamara et al., 2012).”
BackgroundViTHSD: exploiting hatred by targets for hate speech detection on Vietnamese social media texts
Vo, C. N., Huynh, K. B., Luu, S. T., & Do, T.-H. (2025) · Journal of Computational Social Science · doi:10.1007/s42001-024-00348-6 · Cited by 9
“Fortuna and Nunes [2] defined hate speech as violence that attacks and expresses hatred toward a group of people based on their ethnic origin, nationality, religion, and gender even in subtle or rumor form.”
A visual approach to tracking emotional sentiment dynamics in social network commentaries
Hossain, I., Puppala, S., Alam, M. J., & Talukder, S. K. (2024) · Social Network Analysis and Mining · doi:10.1007/s13278-024-01332-8 · Cited by 10
“These studies typically analyzed content in isolation and retrospectively (Fortuna and Nunes 2018).”
Hybrid Feature and Optimized Deep Learning Model Fusion for Detecting Hateful Arabic Content
Gasmi, K., Ltaifa, I. B., Abdalrahman, A. E. M., Hamid, O. H., Altaieb, M. O., Ali, S., Ammar, L. B., & Mrabet, M. (2025) · IEEE Access · doi:10.1109/access.2025.3591673 · Cited by 9
“These approaches investigated Various types of characteristics, including lexical features, linguistic features, syntactic features, and knowledge-based features [30].”
Mitigating Text Toxicity with Counterfactual Generation
Bhan, M., Vittaut, J.-N., Achache, N., Legrand, V., Blangero, A., Chesneau, N., Murris, J., & Lesot, M. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08330-2_7 · Cited by 10
Indurthi, V., Syed, B., Shrivastava, M., Gupta, M., & Varma, V. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2109 · Cited by 9
“Papers published in the last two years include the surveys by (Schmidt and Wiegand, 2017) and (Fortuna and Nunes, 2018), the paper by (Davidson et al.”
Building an Integrated Comment Moderation System – Towards a Semi-automatic Moderation Tool
Riehle, D. M., Niemann, M., Brunk, J., Assenmacher, D., Trautmann, H., & Becker, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-49576-3_6 · Cited by 11
Hateful meme detection with multimodal deep neural networks
Constantin, M. G., Pârvu, D.-Ș., Stanciu, C. L., Ionaşcu, D., & Ionescu, B. E. (2021) · International Symposium on Signals, Circuits and Systems · doi:10.1109/isscs52333.2021.9497374 · Cited by 9
“For a thorough overview of these methods, several literature review papers are available [10], [11].”
A Semi-Supervised Approach to Detect Toxic Comments
Saraiva, G. D., Anchiêta, R. T., Neto, F. A. R., & Moura, R. S. (2021) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-072-4_142 · Cited by 14
“Also, it may occur with different linguistic styles, even in subtle forms or when humour is used (Fortuna and Nunes, 2018).”
BackgroundApplication-specific word embeddings for hate and offensive language detection
Soto, C. P., Nunes, G. M. S., Gomes, J. G. R. C., & Nedjah, N. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11880-2 · Cited by 9
“In[14],the authors provide a detailed study of the problem of hate speech automatic detection and describe most of the resources available.”
BackgroundHate Speech Detection in Portuguese with Naïve Bayes, SVM, MLP and Logistic Regression
da Silva, A. L., & Roman, N. T. (2020) · doi:10.5753/eniac.2020.12112 · Cited by 9
“This need has, in turn, led the community to study the problem, mostly constrained to the English language [Fortuna and Nunes 2018].”
BackgroundDetecting the Role of an Entity in Harmful Memes: Techniques and their Limitations
Nandi, R. N., Alam, F., & Nakov, P. (2022) · doi:10.18653/v1/2022.constraint-1.6 · Cited by 9
Towards automatic conceptual metaphor detection for psychological tasks
Panicheva, P., Mamaev, I. D., & Litvinova, T. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103191 · Cited by 11
PEACE: Cross-Platform Hate Speech Detection - A Causality-Guided Framework
Sheth, P., Kumarage, T., Moraffah, R., Chadha, A. R., & Liu, H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43412-9_33 · Cited by 16
Distinguishing Online Hate Speech from Aggressive Speech: A Five-Factor Annotation Model
Ermida, I. (2023) · Springer eBooks · doi:10.1007/978-3-031-38248-2_2 · Cited by 9
Monitoring Dynamics of Emotional Sentiment in Social Network Commentaries
Hossain, I., Puppala, S., Alam, M. J., & Talukder, S. K. (2023) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1145/3625007.3627730 · Cited by 11
“These studies typically analyze the content in isolation and retrospectively [14].”
BackgroundSridharan, M., & Swapna, T. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2097 · Cited by 9
“(Fortuna and Nunes, 2018) has elaborately surveyed several approaches employed for automatic detection of hate speech.”
BackgroundKebriaei, E., Karimi, S., Sabri, N., & Shakery, A. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2107 · Cited by 9
“Based on a survey conducted by Fortuna and Nunes (2018), the majority of previous works are on English and the researchers mainly use machine learning for this task and most proposed methods on abusive content detection have modeled the problem as a binary classification task.”
BackgroundSeeds of SEED: NMT-Stroke: Diverting Neural Machine Translation through Hardware-based Faults
Cai, K., Chowdhuryy, M. H. I., Zhang, Z., & Yao, F. (2021) · International Symposium on Secure and Private Execution Environment Design (SEED) · doi:10.1109/seed51797.2021.00019 · Cited by 9
“Machine comprehension of natural languages has long been considered as one of the core NLP tasks, which is increasingly employed in security-sensitive application scenarios as toxic comment detection [2], text generation [3] and question answering [4].”
BackgroundPinpointing Fine-Grained Relationships between Hateful Tweets and Replies
Albanyan, A., & Blanco, E. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21284 · Cited by 13
“Hate speech in user-generated content has received substantial attention in recent years (Fortuna and Nunes 2018).”
BackgroundOn the Robustness of Offensive Language Classifiers
Rusert, J., Shafiq, Z., & Srinivasan, P. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.513 · Cited by 9
Analysis and Detection of Multilingual Hate Speech Using Transformer Based Deep LearningPreprint
Das, A., Nandy, S., Saha, R., Das, S., & Saha, D. (2024) · doi:10.36227/techrxiv.170629868.84167256/v1 · Cited by 9
Hossain, M. P. (2023) · Journal of Cyber Policy · doi:10.1080/23738871.2024.2337129 · Cited by 8
The Role of Gender in Hate Speech Targeting Politicians: Evidence from Finnish Twitter
Meriläinen, J. (2024) · International Journal of Politics Culture and Society · doi:10.1007/s10767-024-09476-3 · Cited by 11
Detecting Hate Speech for Hindi-English Code-Mix Text Data Using Dual Contrastive Learning
Sharma, A., & Bhalla, R. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.03.304 · Cited by 8
Social Media Is a Threat for Democracy! A Political Perspective for Analysing and Diminishing Harm
Castelló, I., Colleoni, E., Scherer, A. G., & Trittin‐Ulbrich, H. (2025) · Journal of Management Studies · doi:10.1111/joms.70053 · Cited by 9
Garain, A., & Basu, A. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2133 · Cited by 8
Türkçe Tweetler üzerinde Makine Öğrenmesi ile Nefret Söylemi Tespiti
Mayda, İ., Diri, B., & Yıldız, T. (2021) · European Journal of Science and Technology · doi:10.31590/ejosat.903854 · Cited by 8
“Nefret söyleminin otomatik olarak tespit edilmesine yönelik son yıllarda çok sayıda yayın yapılmıştır (Fortuna & Nunes, 2018; Schmidt & Wiegand, 2017).”
Enhanced Task-based Knowledge for Lexicon-based Approach in Vietnamese Hate Speech Detection
Hoang, S. N., Nguyen, B. T., Nguyen, N. P., Luu, S. T., Phan, H. T., & Nguyen, H. D. (2022) · International Conference on Knowledge and Systems Engineering/Proceedings · doi:10.1109/kse56063.2022.9953615 · Cited by 8
“According to [8], a comment is recognized as hate speech in a case of containing these features: • Containing toxic and hateful languages such as profane and swear words, even if it is in humor cases.”
BackgroundWatimin, N. H., Zanuddin, H. B., Rahamad, M. S., & Yadegaridehkordi, E. (2023) · PLoS ONE · doi:10.1371/journal.pone.0287367 · Cited by 13
Kakati, P., & Dandotiya, D. (2024) · Social Network Analysis and Mining · doi:10.1007/s13278-024-01264-3 · Cited by 12
“Machine learning techniques have demonstrated efficiency in detecting hate speech (Davidson et al. 2017; Fortuna and Nunes 2018; Jaki and De Smedt 2019; Waseem and Hovy 2016).”
Üzelgün, M. A., Giannouli, I., Archontaki, I., Odstrčilová, K., Thomaß, B., & Álvares, C. (2024) · Central European Journal of Communication · doi:10.51480/1899-5101.17.1(35).711 · Cited by 8
“In the same vein, Kumar et al. (2018) propose a typology of verbal aggression by looking into both how it is expressed (overt or covert) and the nature of aggression (physical threat, sexual threat, identity threat, etc.) (see also Fortuna & Nunes, 2018).”
BackgroundHussain, A., & Aslam, A. (2024) · Journal of Applied Research and Technology · doi:10.22201/icat.24486736e.2024.22.4.2466 · Cited by 8
A Literature Survey on Multimodal and Multilingual Sexism Detection
Luo, X., Liang, B., Wang, Q., Li, J., Cambria, E., Xiao-jun, Z., He, Y., Yang, M., & Xu, R. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2025.3561921 · Cited by 11
“While there are a bunch of surveys on hate speech detection [1], [2], [3], [4], [5], [6], [7] where sexism is a subcategory within, there is a lack of literature surveys focusing specifically on sexism detection [8], [9], [10].”
Detecting caste and migration hate speech in low-resource Tamil language
Chakravarthi, B. R., Saranya, R., Ponnusamy, R., Sivagnanam, B., Thakare, S. Y., & Thangasamy, S. (2025) · Language Resources and Evaluation · doi:10.1007/s10579-025-09848-x · Cited by 8
“Researchers have been investigating many methods, such as machine learning, deep learning, and transformer-based models, to effectively address the increase in hate speech events (Fortuna & Nunes, 2018).”
Challenges and Perspectives of Hate Speech Research
Strippel, C., Paasch‐Colberg, S., Emmer, M., & Trebbe, J. (2023) · Weizenbaum Library (Weizenbaum Institute) · doi:10.48541/dcr.v12.0 · Cited by 25
Fortuna, P., & Nunes, S. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2131 · Cited by 8
“Two surveys summarize previous research and conclude that the approaches rely frequently on Machine Learning techniques (Schmidt and Wiegand, 2017; Fortuna and Nunes, 2018).”
Self-citationBuilding a formal model for hate detection in French corpora
Battistelli, D., Bruneau, C., & Dragos, V. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.09.299 · Cited by 8
Detection of hate speech in Social media memes: A comparative Analysis
Nayak, A., & Agrawal, A. (2022) · International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT) · doi:10.1109/icicict54557.2022.9917633 · Cited by 8
“It can be multimedia content or simple text content which may include nudity, pornography [13,5], hate speech [6] and misinformation [7].”
BackgroundWang, B., Zhou, X., & Zhang, X. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2143 · Cited by 8
“The field of Hate Speech Automatic Detection in the text has unquestion-able social impact potential, especially in online communities and digital media platforms (Fortuna and Nunes, 2018).”
BackgroundOnline Xenophobic Behavior Amid the COVID-19 Pandemic: A Commentary
Lee, R. K.-W., & Li, Z. (2020) · Digital Government Research and Practice · doi:10.1145/3428091 · Cited by 8
“The analysis and detection of online xenophobic behaviors such as hate speeches [10, 24] has been widely studied by the social, computer, and data science research community.”
BackgroundMultilingual and Multimodal Hate Speech Analysis in Twitter
Sarracén, G. L. D. L. P. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441668 · Cited by 8
Tula, D., Potluri, P., Ms, S., Doddapaneni, S., Sahu, P., Sukumaran, R., & Patwa, P. (2021) · Cited by 8
HamiltonDinggg at SemEval-2021 Task 5: Investigating Toxic Span Detection using RoBERTa Pre-training
Ding, H., & Jurgens, D. (2021) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2021.semeval-1.31 · Cited by 10
“Toxic messages remain a small but persistent part of online communications (Fortuna and Nunes, 2018; Jurgens et al., 2019).”
BackgroundLuu, S. T., Nguyen, K. V., & Nguyen, N. L.-T. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16968-5 · Cited by 8
“Therefore, there are several penitential approaches for this task, including lexical-based, word features, sentiment analysis, linguistic features, and knowledge-base features [3, 6].”
Addressing Hate Speech with Data Science: An Overview from Computer Science Perspective
Srba, I., Lenzini, G., Pikuliak, M., & Pecar, S. (2021) · Springer eBooks · doi:10.1007/978-3-658-31793-5_14 · Cited by 10
Temporal effects of Unmoderated Hate speech in Gab.Preprint
Mathew, B., Illendula, A., Saha, P., Sarkar, S., Goyal, P., & Mukherjee, A. (2019) · arXiv (Cornell University) · Cited by 8
Automated Detection of Cyberbullying Against Women and Immigrants and Cross-domain AdaptabilityPreprint
Atapattu, T., Herath, M., Zhang, G., & Falkner, K. E. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.02565 · Cited by 11
“Despite recent research advancement in hate speech detection (Fortuna and Nunes, 2018), automated identification of cyberbullying attempts (i.e. repetitive hate speech against an individual or a group) remains as a challenging subtask of NLP. Due to diverse variants of language (e.g. hate,…”
TAR on Social Media: A Framework for Online Content ModerationPreprint
Yang, E., Lewis, D. D., & Frieder, O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.12752 · Cited by 11
“shtml [22, 31, 33, 34, 35, 36] or offensive language in general [23, 37, 38, 39, 40, 41, 42].”
BackgroundApplying Deep Learning for Extremism Detection
Mussiraliyeva, S., Omarov, B., Bolatbek, M., Ospanov, R., Baispay, G., Medetbek, Z., & Yeltay, Z. (2021) · Communications in Computer and Information Science · doi:10.1007/978-981-16-3660-8_56 · Cited by 7
A Markovian Kernel-based Approach for itaLIan Speech acT labEliNg
Croce, D., & Basili, R. (2018) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.4661 · Cited by 7
Towards Youth-Sensitive Hateful Content Reporting: An Inclusive Focus Group Study in Germany
Bäumler, J., Bader, H., Kaufhold, M., & Reuter, C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713542 · Cited by 7
“While a universally accepted defnition of hate speech remains elusive [79, 93], we follow the view that it attacks or diminishes groups or individuals based on actual or ascribed group afliations or identities [30, 92].”
Explainable AI for Hate Speech Moderation: A Stakeholder‐Centered and Sociotechnical Review
Qureshi, M. D. M., Qureshi, M. A., & Rashwan, W. (2026) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70076 · Cited by 12
The problem of varying annotations to identify abusive language in social media content
Seemann, N., Lee, Y. S., Höllig, J., & Geierhos, M. (2023) · Natural Language Engineering · doi:10.1017/s1351324923000098 · Cited by 7
“Furthermore, popular social media platforms such as Facebook, YouTube, and Twitter support critical differences in definitions of abusive language (Fortuna and Nunes 2018).”
BackgroundOnline Learning From Evolving Feature Spaces With Deep Variational Models
Lian, H., Wu, D., Hou, B., Wu, J. F., & He, Y. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3326365 · Cited by 8
“Examples include multimedia retrieval [4], [5], online speech analytics [6], [7], recommender systems [8], [9], [10], [11], [12], to just name a few.”
BackgroundNewsCom-TOX: a corpus of comments on news articles annotated for toxicity in Spanish
Taulé, M., Nofre, M., Bargiela, V., & Bonet, X. (2024) · Language Resources and Evaluation · doi:10.1007/s10579-023-09711-x · Cited by 14
“Several proposals for a definition have been made, which are materialized in different terms used to refer to it, such as hate speech, offensive, abusive, aggressive or toxic language (Fortuna & Nunes, 2018; Poletto et al., 2021) and (Vidgen & Derczynski, 2021).”
Fairness-Aware Multi-Group Target Detection in Online Discussion
Gupta, S. S., De‐Arteaga, M., & Lease, M. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806441 · Cited by 7
“Similarly, many datasets [12, 16] treat group identification as binary or mutually exclusive, assuming each post targets only one group.”
Sun, Y., Pendyala, V., Lian, R., Xin, H., Patel, P., Bucy, E. P., & Shah, D. V. (2025) · Visual Communication Quarterly · doi:10.1080/15551393.2025.2455495 · Cited by 8
“…media, sometimes fused in the form of memes, yet most studies focus on textual or verbal analysis, ignoring the visual side of hate speech due to the challenges of large-Visual Volume 32 January – March 2025 scale image analysis (see Fortuna & Nunes, 2018; Watanabe, Bouazizi, & Ohtsuki, 2018).”
Suryanto, E., Mukhibun, A., Rahmawati, A., Putri, F. H. D. K., Junear, K. A. G., & Rohmah, R. D. (2025) · Humanities Horizon · doi:10.63373/3047-8014/36 · Cited by 7
Dezhboro, A., Ramírez-Márquez, J. E., & Krstikj, A. (2026) · Risk Analysis · doi:10.1111/risa.70214 · Cited by 8
“Second, supervised models trained on labeled data consistently demonstrate higher accuracy than unsupervised approaches, particularly for nuanced classification tasks such as hate speech detection (Davidson et al., 2017 ; Fortuna and Nunes, 2018 ).”
Error analysis in a hate speech detection task: The case of Haspeede-TW at Evalita 2018
Francesconi, C., Bosco, C., Poletto, F., & Sanguinetti, M. (2019) · UNICA IRIS Institutional Research Information System (University of Cagliari) · Cited by 7
HateSense: Tackling Ambiguity in Hate Speech Detection
Kumaresan, K., & Vidanage, K. (2019) · National Information Technology Conference (NITC) · doi:10.1109/nitc48475.2019.9114528 · Cited by 7
“Yet, it has been identified that in the domain of hate speech detection, a standard dataset as mentioned above does not exist [3], instead there are few openly available data sets from previous researches and these will be utilized in order to determine the accuracy of the system.”
BackgroundNjovangwa, G., & Justo, G. (2021) · Tanzania Journal of Science · doi:10.4314/tjs.v47i4.2 · Cited by 7
“The rule based approach has already been successfully used for word classification in some applications (Fortuna and Nunes 2018).”
BackgroundMethodInfluentialConvolutional Graph Neural Networks for Hate Speech Detection in Data-Poor Settings
Sarracén, G. L. D. L. P., & Rosso, P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-08473-7_2 · Cited by 7
Goldzycher, J., Preisig, M., Amrhein, C., & Schneider, G. (2023) · doi:10.18653/v1/2023.woah-1.19 · Cited by 9
Fillies, J., Hoffmann, M. P., & Paschke, A. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386244 · Cited by 10
“Notably, among these variations, the definition of Fortuna and Nunes [14] appears to be the most widely used by researchers and has been adopted for the purpose of this study.”
BackgroundHate Cannot Drive Out Hate: Forecasting Conversation Incivility following Replies to Hate Speech
Yu, X., Blanco, E., & Hong, L. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31422 · Cited by 7
A Visual Approach to Tracking Emotional Sentiment Dynamics in Social Network Commentaries
Hossain, I., Puppala, S., Alam, M. J., Talukder, S. K., & Talukder, Z. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31337 · Cited by 11
Brunk, J., Niemann, M., & Riehle, D. M. (2019) · IEEE Conference on Business Informatics · doi:10.1109/cbi.2019.00061 · Cited by 7
“Studies like the ones by [36], [26] and [27] can help researchers and practitioners to get into the topic without having to read through all forms of law books, cases and existing studies.”
BackgroundAbusive Comments in Online Media and How to Fight Them: State of the Domain and a Call to Action
Niemann, M., Welsing, J., Riehle, D. M., Brunk, J., Assenmacher, D., & Becker, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61841-4_9 · Cited by 7
Hate Speech Detection: A Bird’s-Eye View
Vadesara, A., Tanna, P., & Joshi, H. (2020) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-15-4474-3_26 · Cited by 7
Fedtke, C., & Wiedemann, G. (2020) · Ars digitalis · doi:10.1007/978-3-658-30702-8_5 · Cited by 7
Novel Approach for Generating Hybrid Features Set to Effectively Identify Hate Speech
Shruthi, P., & M, A. K. K. (2020) · INTELIGENCIA ARTIFICIAL · doi:10.4114/intartif.vol23iss66pp97-111 · Cited by 7
Targeted Identity Group Prediction in Hate Speech Corpora
Sachdeva, P., Barreto, R., von Vacano, C., & Kennedy, C. J. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.22 · Cited by 7
“Given the scale of online hate speech, much effort has been made toward the development of automated approaches to classify or measure it given raw text (Fortuna and Nunes, 2018; Tontodimamma et al., 2021).”
BackgroundMulti-label Emotion Classification of COVID–19 Tweets with Deep Learning and Topic Modelling
Anuratha, K., & Parvathy, M. (2022) · Computer Systems Science and Engineering · doi:10.32604/csse.2023.031553 · Cited by 7
Sarin, G., Kumar, P., & Mukund, M. (2023) · Benchmarking An International Journal · doi:10.1108/bij-07-2022-0454 · Cited by 8
Hateful Comment Detection and Hate Target Type Prediction for Video Comments
Gupta, S., Priyadarshi, P., & Gupta, M. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615260 · Cited by 7
Trigger Warnings: Bootstrapping a Violence Detector for Fan Fiction
Wolska, M. A., Wiegmann, M., Schröder, C., Borchardt, O., Stein, B., & Potthast, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.41 · Cited by 9
Singh, P., & Chand, S. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2128 · Cited by 7
“Examples include (Wie-gand et al., 2018), (ElSherief et al., 2018) and (Fortuna and Nunes, 2018).”
Metadata
- Authors
- Fortuna, P., & Nunes, S.
- Affiliations
- INESC TEC (PT)
- Universidade do Porto (PT)
- Venue
- ACM Computing Surveys
- Published
- 2018-07-31
- Volume
- 51
- Issue
- 4
- Pages
- 1-30
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W2887782043
- Cited by
- 1,241
- References
- 52
- Citations by year
- 2026: 139 · 2025: 195 · 2024: 141 · 2023: 185 · 2022: 158 · 2021: 199 · 2020: 144 · 2019: 71 · 2018: 8
- FWCI
- 59.53
- Type
- journal article
- Language
- en
- Open access
- Locations
- ACM Computing Surveys (publishedVersion)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, other-oa)
- Funders
- FourEyes, a research line within project TEC4Growth—Pervasive Intelligence
- North Portugal Regional Operational Programme
- European Regional Development Fund
- Enhancers and Proofs of Concept with Industrial (Impact/NORTE-01- 0145-FEDER-000020)
- PORTUGAL 2020 Partnership Agreement
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- hate speech detection, text classification, definition of hate speech, annotated dataset, multilingual dataset, online communities, digital media platforms, social impact
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.