A Survey on Automatic Detection of Hate Speech in Text
Open accessThe DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index.
Abstract
The scientific study of hate speech, from a computer science point of view, is recent. This survey organizes and describes the current state of the field, providing a structured overview of previous approaches, including core algorithms, methods, and main features used. This work also discusses the complexity of the concept of hate speech, defined in many platforms and contexts, and provides a unifying definition. This area has an unquestionable potential for societal impact, particularly in online communities and digital media platforms. The development and systematization of shared resources, such as guidelines, annotated datasets in multiple languages, and algorithms, is a crucial step in advancing the automatic detection of hate speech.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recent studies(Jurgens etal. 2019; Fortuna and Nunes 2018) that have looked into tackling and proposing subtle hate detection suggest the consideration of making all subtle forms of discrimination, even jokes, as hate speech since they negatively affect some people psychological even though they are considered harmless(Douglass etal. 2016).”
From Misogynoir: challenges in detecting intersectional hate (2022) · cites this paper as Fortuna and Nunes 2018
Citations
Cites
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,312
Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
Powers, D. M. W. (2020) · arXiv · doi:10.48550/arxiv.2010.16061 · Cited by 5,440
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Waseem, Z., & Hovy, D. (2016) · Proceedings of the NAACL Student Research Workshop · doi:10.18653/v1/n16-2013 · Cited by 1,655 · cited 1×
“Nevertheless, detecting hate speech is a very important task, because it is connected with actual hate crimes [42, 65, 77], and automatic hate speech detection in text can also provide data about this phenomenon.”
BackgroundA Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A., & Wiegand, M. (2017) · Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media · doi:10.18653/v1/w17-1101 · Cited by 1,395 · cited 4×
“A problem that is referred in hate speech detection is that sentences must be classified and not words [66].”
BackgroundInfluentialAbusive Language Detection in Online User Content
Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., & Chang, Y. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883062 · Cited by 1,122 · cited 9×
“[10], abusive language [58], discrimination [69], profanity [23], toxicity [43], flaming [35], extremism [55, 62], and radicalization [2].”
BackgroundMethodInfluentialDetecting Hate Speech on the World Wide Web
Warner, W. B., & Hirschberg, J. (2012) · Cited by 964
Deep Learning for Hate Speech Detection in Tweets
Badjatiya, P., Gupta, S., Gupta, M., & Varma, V. (2017) · Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion · doi:10.1145/3041021.3054223 · Cited by 849 · cited 2×
“N-grams are one of the most used techniques in hate speech automatic detection and related tasks [4, 9, 19, 33, 52, 58, 77].”
MethodStanford typed dependencies manual
de Marnee, M.-C., & Manning, C. D. (2010) · Cited by 717
Hate Speech Detection with Comment Embeddings
Djuric, N., Zhou, J., Morris, R., Grbovic, M., Radosavljevic, V., & Bhamidipati, N. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2740908.2742760 · Cited by 712 · cited 2×
“Some authors [25] use a paragraph2vec approach to classify language on user comments as abusive or clean and also to predict the central word in the message [25].”
Background
Research works printed but not linked: 18
- ACL., ALW1: 1st workshop on abusive language online.
- Burnap Peter, Proceedings of the Conference on the Internet, Policy 8 Politics. 1--18
- CHI2017. 2017. 2017 Workshop on online harassment. Retrieved from http://social.umd.edu/woh/. CHI2017. 2017. 2017 Workshop on online harassment. Retrieved from http://social.umd.edu/woh/.
- CONTACT., 2017, Interdisciplinary conference on Hate speech. Definitions, Interpretations and Practices.
- Davidson Thomas, 2017, Automated hate speech detection and the problem of offensive language. arXiv Preprint arXiv:1703.04009 · arXiv
- De Marneffe Marie-Catherine, 2008, Manning
- Pauw Guy De, 2016, First workshop on text analytics for cybersecurity and online safety (TA-COS
- Vigna Fabio Del, 2017, Proceedings of the 1st Italian Conference on Cybersecurity. 86--95
- Cambridge Dictionary. 2017. Profanity. Retrieved from https://dictionary.cambridge.org/dictionary/english/profanity. Cambridge Dictionary. 2017. Profanity. Retrieved from https://dictionary.cambridge.org/dictionary/english/profanity.
- Giblietto Fabio, 2015, Proceedings of the 4th Workshop on Making Sense of Microposts (#Microposts’14)
- Reporting Hate. 2017. Hate speech conference I.H.D.I.P. Retrieved from http://reportinghate.eu/contact2017/. Reporting Hate. 2017. Hate speech conference I.H.D.I.P. Retrieved from http://reportinghate.eu/contact2017/.
- Kwok Irene, 2013, Proceedings of the Association for the Advancement of Artificial Intelligence.
- Ross Bjorn, 2017, Measuring the reliability of hate speech annotations: The case of the european refugee crisis. arXiv Preprint arXiv:1701.08118 · arXiv
- Silva Leandro, 2016, Analyzing the targets of hate in online social media. arXiv Preprint arXiv:1603.07709 · arXiv
- Natalya Tarasova. 2016. Classification of Hate Tweets and Their Reasons using SVM. Master’s thesis. Uppsala Universitet. Natalya Tarasova. 2016. Classification of Hate Tweets and Their Reasons using SVM. Master’s thesis. Uppsala Universitet.
- Tulkens Stéphan, 2016, A dictionary-based approach to racism detection in dutch social media. arXiv Preprint arXiv:1608.08738 · arXiv
- Warner William, 2012, Proceedings of the Second Workshop on Language in Social Media. Association for Computational Linguistics, 19--26
- Yuan Shuhan, 2016, Proceedings of the International Conference on Extending Database Technology. 696--697
Cited by
Items 801–900 of 1,004
Multilingual and Severity-Aware Cyberbullying Detection with Emotion Fusion and Transformer Models
Vetal, N., & Vasgi, B. P. (2025) · IEEE Pune Section International Conference (PuneCon) · doi:10.1109/punecon67554.2025.11379022 · Cited by 2
Mahouachi, R. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.131854 · Cited by 3
Attitudes, Imagined Roles, and Governance Boundaries for AI in Decentralized Social Media
Zhang, Z., Zhao, J., Wang, G., Viswanathan, S., Ross, T., Johnston, S., Liu, D., Noh, H., Kleek, M. G. V., & Shadbolt, N. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790295 · Cited by 2
“Major platforms deploy machine-learning classifiers, keyword filters, and hashing systems to detect hate speech, harassment, misinformation, spam, and child sexual abuse material (CSAM), making such systems central to contemporary governance infrastructures [2, 34, 67, 89].”
How can hate narratives be tracked in online environments? A corpus-based study
Almagro, M., Vieites, C., Chakkour, H., & Sancho, A. (2026) · Frontiers in Communication · doi:10.3389/fcomm.2026.1743196 · Cited by 2
Forensic Linguistics Study: Online Hate Speech On Social Media Instagram (Indonesian Government)
Clara, E. D., Hutauruk, B. S., & Munthe, M. V. R. (2026) · Jurnal Ilmu Sosial dan Humaniora · doi:10.58540/isihumor.v4i2.1627 · Cited by 2
“Fortuna and Nunes (2019) explain that hate speech is closely related to the speaker’s intention expressed through language.”
Garain, A. (2020) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2020.semeval-1.257 · Cited by 2
Rodrigues, L., Junior, L. C. C. L., Jacob, A. F. L., & Lobato, F. M. F. (2021) · Anais do Brazilian Workshop on Social Network Analysis and Mining (BraSNAM) · doi:10.5753/brasnam.2021.16128 · Cited by 2
Silva, W. P. E. (2021) · Revista de Estudos da Linguagem · doi:10.17851/2237-2083.29.4.2367-2395 · Cited by 2
Applying Machine Learning to Online Data? Beware! Computational Social Science Requires Care
Bayram, U. (2022) · Advances in human and social aspects of technology book series · doi:10.4018/978-1-7998-8553-5.ch005 · Cited by 2
Masud, S., Pinkesh, P., Das, A., Gupta, M., Nakov, P., & Chakraborty, T. (2022) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3488560.3501392 · Cited by 2
“Subsequently, neural models based on CNNs, RNNs, and Transformers were used [3, 5, 7, 8, 15, 16, 20, 22, 24, 28, 33, 38, 39, 44, 46, 51, 53, 58–61].”
Virtuelle Welten und künstliche Intelligenzen als Herausforderungen und Chancen digitaler Gegenrede
Schmitt, J. B. (2022) · Interkulturelle Studien · doi:10.1007/978-3-658-36540-0_3 · Cited by 2
Leveraging external resources for offensive content detection in social media
Kovács, G., Alonso, P., Saini, R., & Liwicki, M. (2022) · AI Communications · doi:10.3233/aic-210138 · Cited by 5
“The task of detecting hateful and offensive language comes with many difficult challenges, the reader can gain an overview of by for example reading the works of MacAvaney et al. [59], Fortuna et al. [26,27] or Kovács et al. [51].”
(Semi-)Automatische Kommentarmoderation zur Erhaltung Konstruktiver Diskurse
Niemann, M., Assenmacher, D., Brunk, J., Riehle, D. M., Trautmann, H., & Becker, J. (2022) · Aktivismus- und Propagandaforschung · doi:10.1007/978-3-658-35658-3_13 · Cited by 2
VIARS: An Intelligent Voice Agent to Prevent Inappropriate Content Display for Restricted Users
Kang, R., & Rau, P. P. (2021) · Interacting with Computers · doi:10.1093/iwc/iwac019 · Cited by 2
A Survey on Detecting and Preventing Hateful Comments on Social Media Using Deep Learning
Karthika, I., Boomika, G., Nisha, R., Shalini, M., & Srivarshini, S. P. (2022) · Smart innovation, systems and technologies · doi:10.1007/978-981-19-3575-6_30 · Cited by 2
BİRİCİK, Z. (2022) · Turk Turizm Arastirmalari Dergisi · doi:10.26677/tr1010.2022.1106 · Cited by 2
Terrorism Detection and Approaches to Fairness: A Brief Literature Review
Salaam, C., & Rawat, D. B. (2022) · IEEE Global Humanitarian Technology Conference (GHTC) · doi:10.1109/ghtc55712.2022.9910986 · Cited by 2
“Some surveys have reviewed NLP contributions to topics that are conceptually related to extrem-ism/terrorism, such as hate speech [14] or law enforcement [15].”
BackgroundSharma, G., Brar, G. S., Singh, P., Gupta, N., Kalra, N., & Parashar, A. (2022) · Lecture notes in networks and systems · doi:10.1007/978-981-19-3679-1_45 · Cited by 4
Vrunda, K. S., & Jayasri, B. S. (2022) · IEEE Global Conference for Advancement in Technology (GCAT) · doi:10.1109/gcat55367.2022.9971980 · Cited by 2
“[3] is a survey paper where we can find what is Hate speech and why we have to study Hate Speech.”
BackgroundEvaluation of Different ML and Text Processing Techniques for Hate Speech Detection
Shawkat, N., Simpson, J. R., & Saquer, J. M. (2022) · International Conference on Data Intelligence and Security (ICDIS) · doi:10.1109/icdis55630.2022.00040 · Cited by 6
“In previous works [1,5,8,18,20], researchers used different techniques for preprocessing, but all were infused with TF-IDF and n-grams.”
BackgroundMethodInfluentialFeature Representation Techniques for Hate Speech Detection on Social Media: A Comparative Study
Gongane, V. U., Munot, M. V., & Anuse, A. D. (2022) · International Conference on Signal and Information Processing · doi:10.1109/iconsip49665.2022.10007458 · Cited by 3
“Literature has reported various terminologies related to hate speech as offensive [3], abusive [4], cyberbullying [2], toxic, racist, and sexist [2].”
BackgroundInfluentialHate Speech Prediction on Social Media
Aouchiche, I. R. A., Boumahdi, F., Madani, A., & Remmide, M. A. (2023) · SN Computer Science · doi:10.1007/s42979-023-01668-6 · Cited by 2
“Indeed, the detection of this phenomenon named hate speech has become a crucial task and a major challenge to overcome [1, 4, 12].”
BackgroundPurwitasari, D., Navastara, D. A., Findawati, Y., Pramana, K. A., & Raharjo, A. B. (2023) · International Conference on Computer Science, Information Technology and Engineering (ICCoSITE) · doi:10.1109/iccosite57641.2023.10127774 · Cited by 2
“Hate speech is abusive speech that targets specific group characteristics, such as ethnic origin, religion, gender, or sexual orientation [1].”
BackgroundMorabito, R., Kabbara, J., & Emami, A. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.280 · Cited by 10
LITHUANIAN HATE SPEECH CLASSIFICATION USING DEEP LEARNING METHODS
Kankevičiūtė, E., Songailaitė, M., Zhyhun, B., & Mandravickaitė, J. (2023) · Automation technological and business processes · doi:10.15673/atbp.v15i3.2621 · Cited by 2
“Similarly, [54] found that most of the publicly available datasets are incompatible due to different definitions attributed to similar concepts.”
BackgroundMethodEuphemistic Abuse – A New Dataset and Classification Experiments for Implicitly Abusive Language
Wiegand, M., Kampfmeier, J., Eder, E., & Ruppenhofer, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1012 · Cited by 16
Hate Speech Recognition: The Role of Empathy and Awareness of Social Media Influence
Melchiori, F. M., Martucci, S., Destro, C. L., & Benvenuto, G. (2023) · Journal of Educational Cultural and Psychological Studies (ECPS Journal) · doi:10.7358/ecps-2023-028-melc · Cited by 3
Noisy Self-Training with Data Augmentations for Offensive and Hate Speech Detection Tasks
Leite, J. A., Scarton, C., & Silva, D. F. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_068 · Cited by 6
MERMAID: A Dataset and Framework for Multimodal Meme Semantic Understanding
Toh, S., Kuek, A., Chong, W.-H., & Lee, R. K.-W. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386279 · Cited by 6
“Despite the widespread use and popularity of memes, research in this area has primarily focused on analyzing their spread and influence [1]–[3] rather than their meaning.”
BackgroundSaxena, S., Dixit, A. K., & Memoria, M. (2023) · International Conference on Contemporary Computing and Informatics (IC3I) · doi:10.1109/ic3i59117.2023.10397997 · Cited by 2
Transformer-based models for hate speech classification
Jain, D., Arora, S., Jha, C. K., & Malik, G. (2024) · AIP conference proceedings · doi:10.1063/5.0198822 · Cited by 3
Xenophobia Meter: Defining and Measuring Online Sentiment toward Foreigners on Twitter
Umarova, K., Okorafor, O., Lu, P., Shan, S., Xu, A., Zhou, R., Otiono, J., Lyon, B., & Leshed, G. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31406 · Cited by 4
“Different world organizations and social media platforms have their own definitions of hate speech (Fortuna and Nunes 2018).”
BackgroundHate Speech Detection Using Machine Learning and Deep Learning Techniques
Singh, D. P., Gupta, S., & Baghel, R. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-62217-5_10 · Cited by 2
Twitter Hate Speech Detection using Machine Learning
Janardhan, G., Saikiran, B., Reddy, I., & Abhishek, M. (2024) · International Conference on Pervasive Computing and Social Networking (ICPCSN) · doi:10.1109/icpcsn62568.2024.00051 · Cited by 2
YOSHIDA, R., Yoshida, S., & Muneyasu, M. (2024) · IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences · doi:10.1587/transfun.2024smp0004 · Cited by 2
“Many researchers have conducted studies on malicious behavior detection (MBD)–in particular, to detect malicious posting behavior–from textual information using natural language processing (NLP) methods [1].”
Offensive Language Detection of Arabic Tweets Using Deep Learning Algorithm
Alsukhni, E. M., Alazzam, I., & Hanandeh, S. (2024) · International Conference on Information and Communication Systems (ICICS) · doi:10.1109/icics63486.2024.10638282 · Cited by 2
“Additionally, this sort of online activity undermines user confidence in the online platform (Fortuna et al., 2018).”
BackgroundNo room for hate: What research about socially unacceptable discourse taught us about collaboration?
Šulc, A., & de Maiti, K. P. (2020) · Digital Humanities in the Nordic and Baltic Countries Publications · doi:10.5617/dhnbpub.11228 · Cited by 2
“The problem is usually regarded as a machine learning classification task in which researchers develop algorithms or produce descriptive statistics (Fortuna & Nunes, 2018).”
BackgroundCultural Violence and Peace Interventions in Social Media
Kaufhold, M., Haunschild, J., & Reuter, C. (2024) · Technology, peace and security · doi:10.1007/978-3-658-44810-3_18 · Cited by 2
Attention-based Hybrid Deep Learning Model for Detecting Hateful Tweets
Rawat, A., Kumar, S., & Samant, S. S. (2024) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt61001.2024.10725312 · Cited by 2
“Similarly, by employing a comparable architecture to identify hate speech in Portuguese social media, [10] demonstrated how adaptable the technique is across other datasets and languages.”
BackgroundGouliev, Z., & Jaiswal, R. R. (2024) · Conference on Human Centred Artificial Intelligence - Education and Practice (HCAIep) · doi:10.1145/3701268.3701269 · Cited by 3
“Online spaces are filled with negative content like hate speech and cyberbullying [11, 12].”
Ferreira, P. C., Pereira, N., Rosa, H., Oliveira, S., Coheur, L., Francisco, S. M., de Souza, S. B., Ribeiro, R., Carvalho, J. P., Paulino, P., et al. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3518587 · Cited by 2
“Abusiveness and toxicity can be considered rude, hurtful, profane and derogatory language [18], [19].”
An Investigation of Hate Speech in Italian: Use, Identification, and Perception
Cruschina, S., Gianollo, C., Bianchi, C., Zuloaga, M. B., Safina, E. S., Pascale, S. D., Cavirani, E., Marzo, S., Zingaretti, M., Garraffa, M., et al. (2024) · Helsinki University Press eBooks · doi:10.33134/hup-27 · Cited by 3
Cómo entrenar a un algoritmo para detectar el lenguaje de odio en el conflicto Israel-Palestina
Sulayes, A. R. (2025) · European Public & Social Innovation Review · doi:10.31637/epsir-2025-1199 · Cited by 2
Detection of Inappropriate Language on Social Media Platforms Using Machine Learning Algorithms
Mishra, S. O., Ahmer, M., Mittal, N., Maurya, A. K., Singh, A. K., & Kumar, A. (2024) · International Conference on Computational and Characterization Techniques in Engineering & Sciences (IC3TES) · doi:10.1109/ic3tes62412.2024.10877429 · Cited by 4
Bäumler, J., Voronin, G., & Kaufhold, M. (2025) · i-com · doi:10.1515/icom-2024-0062 · Cited by 2
Cyberbullying Detection on Social Media using AI
Begum, A. R. J. (2025) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2025.70045 · Cited by 2
Detecting Hate Speech in Tweets with Advanced Machine Learning Techniques
Chaitrika, D. K., Lalitha, C., Gnanasai, E., Keerthi, D., & Mudduswamy, K. (2025) · International Journal of Scientific Research in Science Engineering and Technology · doi:10.32628/ijsrset2512312 · Cited by 2
Promoting Constructive Online Debates Through Toxic Comment Classifier
Jotheeswaran, J., Geetha, V., Iyyappan, M., & Srinivasa, K. G. (2025) · International Conference on IT Innovation and Knowledge Discovery (ITIKD) · doi:10.1109/itikd63574.2025.11004955 · Cited by 2
“Toxic comments, characterized by abusive language, hate speech, harassment, or offensive content, can have severe consequences for individuals and communities, leading to psychological distress, decreased participation, and the perpetuation of harmful stereotypes and biases [4-5].”
Prabowo, B. A., Maryani, T., Mulyani, W., & Sulityawardhana, N. K. (2025) · LITERACY International Scientific Journals of Social Education Humanities · doi:10.56910/literacy.v4i2.2464 · Cited by 3
Applied Artificial Intelligence in Detecting Hate Speech
Cincović, J., Radenković, U., Mićović, M., Milaković, A., Jocović, V., & Drašković, D. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-99201-8_11 · Cited by 2
Min, H. J. (2025) · IEEE Access · doi:10.1109/access.2025.3596551 · Cited by 2
Thandil, R. K., Muneer, V. K., Houlath, K., & Sabique, P. V. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-06268-8 · Cited by 2
Squadrone, L. (2018) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.4854 · Cited by 3
Govor mržnje u hrvatskom medijskom prostoru
Poljak, M., Hadžić, J., & Martinić, M. (2020) · In medias res · doi:10.46640/imr.9.17.8 · Cited by 8
Pachón, V., Vázquez, J. M., Betanzos, J. M. L., & Fernández, J. L. A. (2020) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2020.semeval-1.259 · Cited by 3
“We decided to test BERT (Devlin et al. 2018), RoBERTa (Liu et al. 2019), DistilBERT (Sanh et al. 2019), ALBERT (Lan et al. 2019), XLM-RoBERTa (Conneau et al. 2019b), Transformer XL (Dai et al. 2019), XLM (Lample and Conneau 2019), XLMNet (Yang et al. 2019) and XLM-RoBERTa (Conneau et al. 2019a)…”
MethodInfluentialIIITG-ADBU at SemEval-2020 Task 12: Comparison of BERT and BiLSTM in Detecting Offensive Language
Baruah, A., Das, K., Barbhuiya, F. A., & Dey, K. (2020) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2020.semeval-1.204 · Cited by 4
“Fortuna and Nunes (2018) defines these related terms. Waseem and Hovy (2016) used a logistic regression classifier to detect racist and sexist tweets. Logistic regression and multi-layer perceptron classifiers were used in the study performed by Wulczyn et al. (2017). This study was performed to detect insults in Wiki…”
BackgroundMethodAchieving Hate Speech Detection in a Low Resource Setting
Li, P. (2021) · Digital Commons - USU (Utah State University) · doi:10.26076/1ed8-eb08 · Cited by 2
“Since it incorporates at some degree the context of each word, the classifiers’ performance is improved [34].”
BackgroundABL-MICRO: Opportunities for Affective AI Built Using a Multimodal Microaggression Dataset.
Washington, G. J., Mance, G., Aryal, S. K., & Kengni, M. (2021) · National Conference on Artificial Intelligence · Cited by 9
Data Visualization for Supporting Linguists in the Analysis of Toxic Messages
Kavaz, E., Puig, A., Rodríguez, I., Taulé, M., & Nofre, M. (2021) · Computer Science Research Notes · doi:10.24132/csrn.2021.3002.7 · Cited by 2
Classism Hate Speech in Katherine Mansfield's Short Story ‘The Doll’s House’: A Pragmatic Study
Jassim, I. N., & Ahmed, M. S. (2021) · Journal of College of Education for Women · doi:10.36231/coedw.v32i3.1521 · Cited by 2
“Such an investigation involves applying Fortuna and Nunes’s (2018) criteria to each utterance found within selected short stories, such as: 1) exclusion language, 2) arrogance language, 3) belittled language, 4) negative stereotypes, or 5) racial slurs, together with the other pragmatic theories selected as models of…”
BackgroundMethodInfluentialKuş, O., & ÖZTÜRK, İ. D. (2021) · Türkiye İletişim Araştırmaları Dergisi · doi:10.17829/turcom.933550 · Cited by 2
Barkhashree, & Dhaliwal, P. (2021) · International Conference on Advances in Computing, Communication Control and Networking (ICAC3N) · doi:10.1109/icac3n53548.2021.9725393 · Cited by 2
“Fortuna et al. [7] demonstrated a detailed study discussing about the several definitions.”
BackgroundAn Introduction to Detection of Hate Speech and Offensive Language in Slovak
Sokolová, Z., Staš, J., & Hládek, D. (2022) · International Conference on Advanced Computer Information Technologies (ACIT) · doi:10.1109/acit54803.2022.9913104 · Cited by 2
“Hate speech can occur in the form of different language styles, it can be hidden behind attempts at humor or satire [2].”
BackgroundCahyana, R., Tresnawati, D., Fitriani, L., Setiawan, R., Satria, E., Sutedi, A., Rahayu, S., Nashrulloh, M. R., Kurniawati, R., & Utami, Y. H. (2022) · JMM (Jurnal Masyarakat Mandiri) · doi:10.31764/jmm.v6i5.10121 · Cited by 2
“Sebagian di antara julukan tersebut merupakan kata atau istilah yang menyinggung karena menyerang karakteristik yang dilindungi oleh negara, seperti agama, etnis, jenis kelamin dan lain sebagainya (Chetty & Alathur, 2018; CohenAlmagor, 2018; Fortuna & Nunes, 2018; Zhang & Luo, 2019).”
BackgroundCombating high variance in Data-Scarce Implicit Hate Speech Classification
Pal, D., Chaudhari, K., & Sharma, H. (2022) · IEEE Region 10 Conference (TENCON) · doi:10.1109/tencon55691.2022.9977681 · Cited by 2
“In such cases, the hate is delivered through sarcasm [1], humor [2], or some other literary device [3].”
BackgroundLa structure actantielle des discours de haine dans les plateformes participatives en ligne
Monnier, A., & Boursier, A. (2022) · Cahiers de narratologie · doi:10.4000/narratologie.13848 · Cited by 2
A Social-Aware Deep Learning Approach for Hate-Speech Detection
Apostolopoulos, G. C., Liakos, P., & Delis, A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25158-0_43 · Cited by 2
An Approach of Hate Speech Identification on Twitter Corpus
Kumari, K., & Jamatia, A. (2023) · Smart innovation, systems and technologies · doi:10.1007/978-981-19-7513-4_11 · Cited by 2
dos Santos, P. D., Penteado, C. L. D. C., de Almeida, L. D., & Goya, D. (2023) · Revista Debates · doi:10.22456/1982-5269.129776 · Cited by 5
Cross-community Adapter Learning (CAL) to Understand the Evolving Meanings of Norm Violation
dos Santos, T. F., Cranefield, S., Savarimuthu, B. T. R., Osman, N., & Schorlemmer, M. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/13 · Cited by 3
A Comparative Study of Automatic Hate Speech Detection Using Machine Learning
Siddiqui, J. A., Yuhaniz, S. S., & Memon, Z. A. (2024) · IEEE Karachi Section Humanitarian Technology Conference (KHI-HTC) · doi:10.1109/khi-htc60760.2024.10482049 · Cited by 2
“"Material that incites violence towards individuals or collectives on grounds of religion, gender, race or ethnic background, disability, veteran status, sexual orientation/gender identity" and age, is how they define hate speech [5].”
BackgroundDeciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hate
Botelho, A., Vidgen, B., & Hale, S. A. (2021) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.5123566 · Cited by 2
Impact of Data Augmentation on Hate Speech Detection in Roman Urdu
Maqbool, F., Spahiu, B., & Maurino, A. (2024) · BOA (University of Milano-Bicocca) · Cited by 2
Trawling for Trolling: A Dataset
Hitkul, Aggarwal, K., Bamdev, P., Mahata, D., Shah, R. R., & Kumaraguru, P. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3828501 · Cited by 2
Learning Neural Models for Natural Language Processing in the Face of Distributional ShiftPreprint
Michel, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.01558 · Cited by 3
A comparative study of deep learning algorithms for hate speech detection on Twitter
Mutanga, R. T. (2021) · DUT Open Scholar (Durban University of Technology) · doi:10.51415/10321/3776 · Cited by 2
The 7th Workshop on Online Abuse and Harms (WOAH)
(2023) · doi:10.18653/v1/2023.woah-1 · Cited by 2
Race and Digital Discrimination
Gangadharan, S. P. (2024) · Oxford University Press eBooks · doi:10.1093/acrefore/9780190228613.013.1258 · Cited by 2
BOISHOMMO: Holistic Approach for Bangla Hate SpeechPreprint
Kafi, M. A. A., Banshal, S. K., Shakib, M. I., Azam, S., & Tabashom, T. A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.08408 · Cited by 2
“To generate a multifaceted dataset, ten broader categories of hate speech were identified [Fortuna and Nunes, 2018].”
Wang, Y., Jobanputra, M., Lee, J.-U., Oh, S., Valera, I., & Demberg, V. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2509.22291 · Cited by 4
ViToxReduce: A Python library for Toxic Comment Detection and Reduction
Le, T., & Tran, T. K. (2026) · SoftwareX · doi:10.1016/j.softx.2026.102620 · Cited by 2
Hate speech and offensive language detection using transfer learning approaches
Mozafari, M. (2021) · theses.fr (ABES) · doi:10.70675/9fc685efz2a9cz4fa0z9041z5001b09e8195 · Cited by 6
The other side of compression: Measuring bias in pruned transformers
Proskurina, I., Metzler, G., & Velcin, J. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-30047-9_29 · Cited by 1
Taking Back Control of Social Media Feeds with Take Back Control
Salminen, J. O., Corporan, J., Jung, S., & Jansen, B. J. J. (2021) · International Conference on Innovations in Intelligent Systems and Applications (INISTA) · doi:10.1109/inista52262.2021.9548614 · Cited by 1
“Tasks such as toxicity detection and sentiment analysis have shown to be difficult [9,10,19,35,37,46], not only for machines but also for humans due to subjectivity and interpretation.”
BackgroundGenetic Algorithm‐Based Optimization for Speech Processing Applications
Ramya, R., Preethi, M., & Rajalakshmi, R. (2023) · Wiley eBooks · doi:10.1002/9781119906391.ch13 · Cited by 1
Nejadgholi, I., Kiritchenko, S., Fraser, K., & Balkır, E. (2023) · doi:10.18653/v1/2023.woah-1.14 · Cited by 4
Aporophobia: An Overlooked Type of Toxic Language Targeting the Poor
Kiritchenko, S., Curto, G., Nejadgholi, I., & Fraser, K. (2023) · doi:10.18653/v1/2023.woah-1.12 · Cited by 13
“Significant research effort has been devoted to addressing the problem of toxic language targeting several social groups, including women, immigrants, and ethnic minorities (Fortuna and Nunes, 2018; Kiritchenko et al., 2021).”
BackgroundComputer Processing of Language: Where Archaeological Discourse and Computers Meet
Martín-Rodilla, P. (2023) · Quantitative archaeology and archaeological modelling · doi:10.1007/978-3-031-37156-1_9 · Cited by 1
Finding Authentic Counterhate Arguments: A Case Study with Public Figures
Albanyan, A., Hassan, A. E., & Blanco, E. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.855 · Cited by 6
“Identifying hateful content in user-generated content has received substantial attention in recent years (Fortuna and Nunes, 2018).”
BackgroundMachine Learning for Predicting Intent of Radical Action in Text Data
Jurek, A., & Szabelska, A. (2024) · Collabra Psychology · doi:10.1525/collabra.94304 · Cited by 1
Offensive Language Detection from Arabic Texts
Awajan, A. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-62269-4_6 · Cited by 1
Hate Speech in Indian Cyber Space at the Intersection of Law and Technology: An Exploratory Study
Ojha, N. K., & Vaish, A. (2024) · Advances in human and social aspects of technology book series · doi:10.4018/979-8-3693-1127-1.ch009 · Cited by 1
Hate speech in higher education: Exploring student perspectives and its impact on democratic values
Canaj, K., Nimani, A., Husaj, S., & Canaj, B. (2025) · Journal of Infrastructure Policy and Development · doi:10.24294/jipd10918 · Cited by 1
“Fortuna and Nunes (2018) provide a systematic overview of automated hate speech detection, offering insights into computational advancements and challenges in identifying hateful content.”
Detecting Linguistic Bias in Government Documents Using Large language Models
de Swart, M., den Hengst, F., & Chen, J. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714526 · Cited by 5
“The definitions were designed to capture a broad spectrum of bias, ranging from subtle and possibly unintended language variations to more explicit forms, such as hate speech [11].”
Topic Modeling for Enhancing Transformers Hate Speech Detection
Santos, F. M. P., Sousa, J. M. C., & Vieira, S. M. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-74000-8_19 · Cited by 1
From invocation to indictment: Analyzing hate speech elements in a social media prayer
Surahmat, S., Arimi, S., Susilowati, N. E., & Imamah, F. M. (2025) · Rainbow Journal of Literature Linguistics and Cultural Studies · doi:10.15294/rainbow.v14i1.23626 · Cited by 1
“Based on the various existing definitions, Fortuna & Nunes (2018) identified that hate speech has the following elements: (1) carried out to incite violence or hatred, (2) carried out to attack or reduce, and (3) hate speech has a specific target.”
Barbosa, M., Arcila, C. E. R., & Holgado, P. S. (2025) · Empiria Revista de metodología de ciencias sociales · doi:10.5944/empiria.64.2025.45201 · Cited by 1
“En otros ámbitos, como la informática y la ingeniería, ha habido una trayectoria similar, puesto que antes de 2014 el discurso de odio recibía escasa atención (Fortuna & Nunes, 2019).”
Ramya, S., Akash, C., Gambhir, R., Keerthi Raj, S., & Hegde, S. (2025) · International Journal of Scientific Research in Computer Science Engineering and Information Technology · doi:10.32628/cseit25113312 · Cited by 1
Cellular Automata-based Toxicity Analysis for Social Media Comments
Elizabeth, M. J., & Hazari, R. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-94121-4_11 · Cited by 1
A Multi-Task LSTM Approach for Comprehensive Detection of Emotion, Hate Speech, and Violence in Text
Satapathy, S., Gopal, K. M., & Kar, B. (2025) · International Conference on Emerging Trends in Industry 4.0 Technologies (ICETI4T) · doi:10.1109/iceti4t63625.2025.11132222 · Cited by 1
Metadata
- Authors
- Fortuna, P., & Nunes, S.
- Affiliations
- INESC TEC (PT)
- Universidade do Porto (PT)
- Venue
- ACM Computing Surveys
- Published
- 2018-07-31
- Volume
- 51
- Issue
- 4
- Pages
- 1-30
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W2887782043
- Cited by
- 1,241
- References
- 52
- Citations by year
- 2026: 139 · 2025: 195 · 2024: 141 · 2023: 185 · 2022: 158 · 2021: 199 · 2020: 144 · 2019: 71 · 2018: 8
- FWCI
- 59.53
- Type
- journal article
- Language
- en
- Open access
- Locations
- ACM Computing Surveys (publishedVersion)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, other-oa)
- Funders
- FourEyes, a research line within project TEC4Growth—Pervasive Intelligence
- North Portugal Regional Operational Programme
- European Regional Development Fund
- Enhancers and Proofs of Concept with Industrial (Impact/NORTE-01- 0145-FEDER-000020)
- PORTUGAL 2020 Partnership Agreement
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- hate speech detection, text classification, definition of hate speech, annotated dataset, multilingual dataset, online communities, digital media platforms, social impact
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.