A Survey on Automatic Detection of Hate Speech in Text
Open accessThe DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index.
Abstract
The scientific study of hate speech, from a computer science point of view, is recent. This survey organizes and describes the current state of the field, providing a structured overview of previous approaches, including core algorithms, methods, and main features used. This work also discusses the complexity of the concept of hate speech, defined in many platforms and contexts, and provides a unifying definition. This area has an unquestionable potential for societal impact, particularly in online communities and digital media platforms. The development and systematization of shared resources, such as guidelines, annotated datasets in multiple languages, and algorithms, is a crucial step in advancing the automatic detection of hate speech.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recent studies(Jurgens etal. 2019; Fortuna and Nunes 2018) that have looked into tackling and proposing subtle hate detection suggest the consideration of making all subtle forms of discrimination, even jokes, as hate speech since they negatively affect some people psychological even though they are considered harmless(Douglass etal. 2016).”
From Misogynoir: challenges in detecting intersectional hate (2022) · cites this paper as Fortuna and Nunes 2018
Citations
Cites
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,312
Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
Powers, D. M. W. (2020) · arXiv · doi:10.48550/arxiv.2010.16061 · Cited by 5,440
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Waseem, Z., & Hovy, D. (2016) · Proceedings of the NAACL Student Research Workshop · doi:10.18653/v1/n16-2013 · Cited by 1,655 · cited 1×
“Nevertheless, detecting hate speech is a very important task, because it is connected with actual hate crimes [42, 65, 77], and automatic hate speech detection in text can also provide data about this phenomenon.”
BackgroundA Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A., & Wiegand, M. (2017) · Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media · doi:10.18653/v1/w17-1101 · Cited by 1,395 · cited 4×
“A problem that is referred in hate speech detection is that sentences must be classified and not words [66].”
BackgroundInfluentialAbusive Language Detection in Online User Content
Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., & Chang, Y. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883062 · Cited by 1,122 · cited 9×
“[10], abusive language [58], discrimination [69], profanity [23], toxicity [43], flaming [35], extremism [55, 62], and radicalization [2].”
BackgroundMethodInfluentialDetecting Hate Speech on the World Wide Web
Warner, W. B., & Hirschberg, J. (2012) · Cited by 964
Deep Learning for Hate Speech Detection in Tweets
Badjatiya, P., Gupta, S., Gupta, M., & Varma, V. (2017) · Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion · doi:10.1145/3041021.3054223 · Cited by 849 · cited 2×
“N-grams are one of the most used techniques in hate speech automatic detection and related tasks [4, 9, 19, 33, 52, 58, 77].”
MethodStanford typed dependencies manual
de Marnee, M.-C., & Manning, C. D. (2010) · Cited by 717
Hate Speech Detection with Comment Embeddings
Djuric, N., Zhou, J., Morris, R., Grbovic, M., Radosavljevic, V., & Bhamidipati, N. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2740908.2742760 · Cited by 712 · cited 2×
“Some authors [25] use a paragraph2vec approach to classify language on user comments as abusive or clean and also to predict the central word in the message [25].”
Background
Research works printed but not linked: 18
- ACL., ALW1: 1st workshop on abusive language online.
- Burnap Peter, Proceedings of the Conference on the Internet, Policy 8 Politics. 1--18
- CHI2017. 2017. 2017 Workshop on online harassment. Retrieved from http://social.umd.edu/woh/. CHI2017. 2017. 2017 Workshop on online harassment. Retrieved from http://social.umd.edu/woh/.
- CONTACT., 2017, Interdisciplinary conference on Hate speech. Definitions, Interpretations and Practices.
- Davidson Thomas, 2017, Automated hate speech detection and the problem of offensive language. arXiv Preprint arXiv:1703.04009 · arXiv
- De Marneffe Marie-Catherine, 2008, Manning
- Pauw Guy De, 2016, First workshop on text analytics for cybersecurity and online safety (TA-COS
- Vigna Fabio Del, 2017, Proceedings of the 1st Italian Conference on Cybersecurity. 86--95
- Cambridge Dictionary. 2017. Profanity. Retrieved from https://dictionary.cambridge.org/dictionary/english/profanity. Cambridge Dictionary. 2017. Profanity. Retrieved from https://dictionary.cambridge.org/dictionary/english/profanity.
- Giblietto Fabio, 2015, Proceedings of the 4th Workshop on Making Sense of Microposts (#Microposts’14)
- Reporting Hate. 2017. Hate speech conference I.H.D.I.P. Retrieved from http://reportinghate.eu/contact2017/. Reporting Hate. 2017. Hate speech conference I.H.D.I.P. Retrieved from http://reportinghate.eu/contact2017/.
- Kwok Irene, 2013, Proceedings of the Association for the Advancement of Artificial Intelligence.
- Ross Bjorn, 2017, Measuring the reliability of hate speech annotations: The case of the european refugee crisis. arXiv Preprint arXiv:1701.08118 · arXiv
- Silva Leandro, 2016, Analyzing the targets of hate in online social media. arXiv Preprint arXiv:1603.07709 · arXiv
- Natalya Tarasova. 2016. Classification of Hate Tweets and Their Reasons using SVM. Master’s thesis. Uppsala Universitet. Natalya Tarasova. 2016. Classification of Hate Tweets and Their Reasons using SVM. Master’s thesis. Uppsala Universitet.
- Tulkens Stéphan, 2016, A dictionary-based approach to racism detection in dutch social media. arXiv Preprint arXiv:1608.08738 · arXiv
- Warner William, 2012, Proceedings of the Second Workshop on Language in Social Media. Association for Computational Linguistics, 19--26
- Yuan Shuhan, 2016, Proceedings of the International Conference on Extending Database Technology. 696--697
Cited by
Items 301–400 of 1,004
Learning to Detect Online Harassment on Twitter with the Transformer
Bugueño, M., & Mendoza, M. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-43887-6_23 · Cited by 23
Social Media and Microblogs Credibility: Identification, Theory Driven Framework, and Recommendation
Qureshi, K. A., Malick, R. A. S., & Sabih, M. (2021) · IEEE Access · doi:10.1109/access.2021.3114417 · Cited by 25
“The dataset must also be enriched with hate speech, with its related concepts like abusive language, offensive language, general hate, cyberbullying, discrimination, flaming, harassment, profanity, toxic language or comment, extremism, radicalization, etc. [14].”
BackgroundAn Empirical Study of AI Generated Text Detection Tools
Akram, A. (2023) · arXiv · doi:10.48550/arxiv.2310.01423 · Cited by 22
“This is of the utmost importance in applications such as content moderation, where it is important to identify and remove hazardous information and automated spam [11].”
BackgroundUTFPR at SemEval-2019 Task 5: Hate Speech Identification with Recurrent Neural Networks
Paetzold, G. H., Zampieri, M., & Malmasi, S. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2093 · Cited by 22
“As evidenced in two recent surveys (Schmidt and Wiegand, 2017; Fortuna and Nunes, 2018) and in a number of other studies (Malmasi and Zampieri, 2017; Gambäck and Sikdar, 2017; ElSherief et al.”
ResultThe Thin Line Between Hate and Profanity
Madukwe, K. J., & Gao, X. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-35288-2_28 · Cited by 22
Hate Speech Criteria: A Modular Approach to Task-Specific Hate Speech Definitions
Khurana, U., Vermeulen, I., Nalisnick, E., van Noorloos, M., & Fokkens, A. S. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.17 · Cited by 48
Cross-Cultural Transfer Learning for Chinese Offensive Language Detection
Zhou, L., Cabello, L., Cao, Y., & Hershcovich, D. (2023) · Workshop on Cross-Cultural Considerations in NLP (C3NLP) · doi:10.18653/v1/2023.c3nlp-1.2 · Cited by 22
Cross-lingual Offensive Language Detection: A Systematic Review of Dataset, Approach and Challenge
Jiang, A., & Zubiaga, A. (2026) · ACM Computing Surveys · doi:10.1145/3801736 · Cited by 23
“This surge can be attributed to the anonymity provided by these platforms and the lack of strict regulations and effective mechanisms to restrain such behaviour [64].”
InfluentialManifestations of xenophobia in AI systems
Tomašev, N., Maynard, J. L., & Gabriel, I. (2024) · AI & Society · doi:10.1007/s00146-024-01893-4 · Cited by 25
“…speech” or “dangerous speech” has served as a focal point for efforts to address xenophobia, fuelling the development of content moderation systems to tackle the spread of hateful and dangerous speech in social media (Fortuna and Nunes 2018; Cao et al. 2020; Awal et al. 2021; Röttger et al. 2020).”
A survey on multi-lingual offensive language detection
Mnassri, K., Farahbakhsh, R., Chalehchaleh, R., Rajapaksha, P., Jafari, A. R., Li, G., & Crespi, N. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.1934 · Cited by 29
“Moreover, Fortuna & Nunes (2018) presented a systematic overview.”
BackgroundMethodHateThaiSent: Sentiment-Aided Hate Speech Detection in Thai Language
Maity, K., Poornash, A. S., Bhattacharya, S., Phosit, S., Kongsamlit, S., Saha, S., & Pasupa, K. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3376958 · Cited by 22
“Addressing the detection of offensive and hateful speech poses significant challenges, including the comprehension of contextual subtleties, mitigation of data bias, and handling multilingual and code-switching text [18].”
BackgroundAn automated approach to identify sarcasm in low-resource language
Khan, S., Qasim, I., Khan, W., Khan, A., Khan, J. A., Qahmash, A., & Ghadi, Y. Y. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307186 · Cited by 22
Hate Speech Detection using Attention-based LSTM
Sarracén, G. L. D. L. P., Gil-Pons, R., Cuza, C. E. M., & Rosso, P. (2018) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.4784 · Cited by 26
PROTOCOL: Online interventions for reducing hate speech and cyberhate: A systematic review
Windisch, S., Wiedlitzka, S., & Olaghere, A. (2021) · Campbell Systematic Reviews · doi:10.1002/cl2.1133 · Cited by 30
Detecting hate in diversity: a survey of multilingual code-mixed image and video analysis
Rehman, H. M. R. U., Saleem, M., Jhandir, M. Z., Alvarado, E. S., Garay, H., & Ashraf, I. (2025) · Journal Of Big Data · doi:10.1186/s40537-025-01167-w · Cited by 24
“The second study [71] emphasizes the complexity of hate speech by discussing nuances such as the use of humor and subtle forms of discrimination.”
Profiling Hate Speech Spreaders on Twitter Task at PAN 2021.
Rangel, F., Sarracén, G. L. D. L. P., Chulvi, B., Fersini, E., & Rosso, P. (2021) · RiuNet (Universitat Politècnica de València) · Cited by 21
Addressing religious hate online: from taxonomy creation to automated detection
Ramponi, A., Testa, B., Tonelli, S., & Ježek, E. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.1128 · Cited by 21
An approach to automatic classification of hate speech in sports domain on social media
Stanković, S. V., & Mladenović, M. (2023) · Journal Of Big Data · doi:10.1186/s40537-023-00766-9 · Cited by 27
“In computational linguistics, it is known as online hate [32], cyber hate [33], or HS [34].”
BackgroundMolero, J. M., Pérez‐Martín, J., Rodrigo, Á., & Peñas, A. (2023) · IEEE Access · doi:10.1109/access.2023.3310244 · Cited by 21
A novel transformer attention‐based approach for sarcasm detection
Khan, S., Qasim, I., Khan, W., Aurangzeb, K., Khan, J. A., & Anwar, M. S. (2024) · Expert Systems · doi:10.1111/exsy.13686 · Cited by 21
“Advancements in automatic sarcasm detection could significantly enhance sentiment analysis (Fortuna & Nunes, 2018; Joshi et al., 2015).”
Hate Speech and Target Community Detection in Nastaliq Urdu Using Transfer Learning Techniques
Malik, M. S. I., Nawaz, A., & Jamjoom, M. M. (2024) · IEEE Access · doi:10.1109/access.2024.3444188 · Cited by 21
On Analyzing Antisocial Behaviors Amid COVID-19 PandemicPreprint
Awal, M. R., Cao, R., Mitrović, S., & Lee, R. K.-W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.10712 · Cited by 21
“The analysis and detection of online antisocial behaviors such as hate speeches [28,11,29,30,7,13,16,10,3,12], cyberbullying [5,2], harassment [17], and usage of abusive and offensive languages [7,30,12,32,33], etc.”
BackgroundPlatform-agnostic Model to Detect Sinophobia on Social Media
Morgan, M., & Kulkarni, A. (2023) · ACM Southeast Regional Conference (ACMSE) · doi:10.1145/3564746.3587024 · Cited by 20
“Thus, a significant amount of research is happening to design natural language processing and machine learning techniques to automatically detect online abuse and prevent the spread of it [10, 13].”
BackgroundMITRE at SemEval-2019 Task 5: Transfer Learning for Multilingual Hate Speech Detection
Gertner, A. S., Henderson, J., Merkhofer, E., Marsh, A., Wellner, B., & Zarrella, G. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2080 · Cited by 20
“The identification of hate speech (Fortuna and Nunes, 2018; Schmidt and Wiegand, 2017) on platforms like Twitter is of particular interest for law enforcement and to social media companies who wish to remove accounts with offending content from their sites.”
BackgroundCross-lingual Capsule Network for Hate Speech Detection in Social Media
Jiang, A., & Zubiaga, A. (2021) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3465336.3475102 · Cited by 26
“Anonymity and lack of moderation provide benefits for social media, alongside negative effects such as production of harmful and hateful contents [18, 32, 39].”
Levantine hate speech detection in twitter
AbdelHamid, M., Jafar, A., & Rahhal, Y. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00950-4 · Cited by 20
“Following the definition provided by the authors in (Warner and Hirschberg 2012; Fortuna and Nunes 2018), ”Hate speech is any communicative act that used to express hatred toward a person or a group on the basis of some characteristic such as race, ethnicity, gender, sexual orientation,…”
MethodA review on abusive content automatic detection: approaches, challenges and opportunities
Alrashidi, B., Jamal, A. T., Khan, I., & Alkhathlan, A. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.1142 · Cited by 28
Neubauer, L., Straw, I., Mariconti, E., & Tanczer, L. M. (2023) · Journal of Family Violence · doi:10.1007/s10896-023-00517-7 · Cited by 20
“Computational text mining methodologies have been used to harness big data to research social phenomena in other domains, such as the study of online hate (Fortuna & Nunes, 2018), cyberbul-lying (Rosa et al., 2019) and child abuse victimisation (Shahi et al., 2021).”
MethodLang, J., Hong, R., Xu, J., Li, Y., Xu, X., & Zhou, F. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714560 · Cited by 34
“Hateful content in short videos often targets attributes like race, gender, or religion [10, 25, 45, 58, 66, 67] and can manifest through multiple modalities.”
Supporting Human Raters with the Detection of Harmful Content Using Large Language Models
Thomas, K., Kelley, P. G., Tao, D. D. J., Meiklejohn, S., Vallis, O., Tan, S., Bratanič, B., Ferreira, F. T., Eranti, V., & Bursztein, E. (2025) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp61157.2025.00082 · Cited by 29
“Platforms and researchers have developed a plethora of detection strategies for identifying harmful content, most often in the form of content classifiers for text [8], [57], [10], [27], [14], [35], [45] or hash matching solutions for imagery [11], [4].”
Sainudiin, R., Yogeeswaran, K., Nash, K. A., & Sahioun, R. (2019) · Social Network Analysis and Mining · doi:10.1007/s13278-019-0567-9 · Cited by 21
“(2017) for a recent example] or predictive [for e.g., hate speech detection (Schmidt and Wiegand 2017; Fortuna and Nunes 2018)].”
BackgroundSupervised Classifiers to Identify Hate Speech on English and Spanish Tweets
Almatarneh, S., Gamallo, P., Ribadas, F. J., & Alexeev, A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-34058-2_3 · Cited by 26
Investigating Multilingual Abusive Language Detection: A Cautionary Tale
Steimel, K., Dakota, D., Chen, Y., & Kübler, S. (2019) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-056-4_132 · Cited by 20
“For a comprehensive survey of NLP techniques to detect hate speech see (Schmidt and Wiegand, 2017; Fortuna and Nunes, 2018).”
BackgroundEnhancing deep neural networks with morphological information
Klemen, M., Krsnik, L., & Robnik‐Šikonja, M. (2022) · Natural Language Engineering · doi:10.1017/s1351324922000080 · Cited by 27
“Recent approaches involve variants of deep neural networks, though standard machine learning approaches are still popular as shown in the survey of Fortuna and Nunes [2018]. These approaches typically use features such as character n-grams, word n-grams, and sentiment of the sequence.”
BackgroundAziz, N. A. A., Maarof, M. A., & Zainal, A. B. (2021) · Cyber Resilience Conference · doi:10.1109/crc50527.2021.9392486 · Cited by 20
“Although no formal definition of hate speech is universally been accepted, Fortuna and Nunes [11] has concluded the definition of hate speech from the numerous definitions as “ language that attacks or diminishes, that incites violence or hate against groups, based on specific characteristics such…”
BackgroundLiu, X., Zheng, L., Jia, X., Qi, H., Yu, S., & Wang, X. (2021) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2021.3087346 · Cited by 20
Mohdeb, D., Laifa, M., Zerargui, F., & Benzaoui, O. (2022) · Aslib Journal of Information Management · doi:10.1108/ajim-10-2021-0293 · Cited by 20
Yoo, S., Jeon, E., Hyeon, J., & Cho, J. (2025) · Scientific Reports · doi:10.1038/s41598-025-88960-y · Cited by 19
Annotating for Hate Speech: The MaNeCo Corpus and Some Input from Critical Discourse Analysis
Assimakopoulos, S., Muskat, R. V., van der Plas, L., & Albert Gatt, A. (2020) · OAR@UM (University of Malta) · doi:10.48550/arxiv.2008.06222 · Cited by 19
“5092 In this respect, it is often acknowledged (Warner and Hirschberg, 2012; Gao and Huang, 2017; Schmidt and Wiegand, 2017; Fortuna and Nunes, 2018; Sanguinetti et al., 2018; Watanabe et al., 2018) that discourse context plays a crucial role in evaluating remarks, as there are several indirect strategies for expressi…”
BackgroundMethodInfluentialAn Interpretable Approach to Hateful Meme Detection
Deshpande, T., & Mani, N. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3479949 · Cited by 19
“Current state-of-the art approaches [3, 8, 10] have involved the standard natural language processing toolkit, including BERT and other embedding schemes.”
Cyber democracy in the digital age: Characterizing hate networks in the 2022 US midterm elections
Rozo, A. Z., Campo-Archbold, A., López, D. D., Gray, I. W., Pastor-Galindo, J., Nespoli, P., Mármol, F. G., & McCoy, D. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102459 · Cited by 19
OffensiveLang: A Community Based Implicit Offensive Language Dataset
Das, A., Rahgouy, M., Feng, D., Zhang, Z., Bhattacharya, T., Raychawdhary, N., Jamshidi, F., Jain, V., Chadha, A. R., Sandage, M. J., et al. (2024) · IEEE Access · doi:10.1109/access.2024.3513220 · Cited by 19
“Ad-ditionally, the structure of social networks creates an environment conducive to the rapid dissemination of offensive language (Fortuna and Nunes, 2018).”
BackgroundHybrid Machine Learning and Deep Learning Approaches for Insult Detection in Roman Urdu Text
Hussain, N., Qasim, A., Mehak, G., Kolesnikova, O., Gelbukh, A. F., & Sidorov, G. (2025) · AI · doi:10.3390/ai6020033 · Cited by 19
“Insults, being a distinct and nuanced subset of offensive language, often require more contextual understanding and cultural sensitivity to detect accurately [1,2].”
Hussain, N., Qasim, A., Mehak, G., Kolesnikova, O., Gelbukh, A. F., & Sidorov, G. (2025) · Information · doi:10.3390/info16020139 · Cited by 19
Sentimental analysis over twitter data using clustering based machine learning algorithm
Jacob, S. S., & Vijayakumar, R. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-020-02771-9 · Cited by 19
Genetic Programming Approach to Detect Hate Speech in Social Media
Aljero, M. K. A., & Dimililer, N. (2021) · IEEE Access · doi:10.1109/access.2021.3104535 · Cited by 19
“…machine learning approaches, including Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), K-Nearest Neighbors (KNN), Naïve Bayes (NB), Decision Trees (DT), and Convolutional Neural Network (CNN), have been employed to implement classifiers for hate speech detection [2].”
MethodPDHS: Pattern-Based Deep Hate Speech Detection With Improved Tweet Representation
Sharmila, P., Anbananthen, K. S. M., Chelliah, D., Parthasarathy, S., & Kannan, S. (2022) · IEEE Access · doi:10.1109/access.2022.3210177 · Cited by 19
“Fortune et al. [24] suggested a summary of how the automatic detection of hate language in the text had made significant progress in NLP. Karamy et al. [25] suggested a systematic basis for text analysis of Twitter.”
BackgroundDetection and Cross-domain Evaluation of Cyberbullying in Facebook Activity Contents for Turkish
Çoban, Ö., Özel, S. A., & İnan, A. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3580393 · Cited by 22
“However, each target group is often exposed to specific CB-related terms that complicate automatic classification, which depends on a particular language and its most frequent feature is context information [38, 41, 52].”
BackgroundRace and Religion in Online Abuse towards UK Politicians: Working PaperPreprint
Gorrell, G., Bakir, M. E., Greenwood, M., Roberts, I., & Bontcheva, K. (2019) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.48550/arxiv.1910.00920 · Cited by 23
“, 2011; Gonzalez-Bailon, Banchs, & Kaltenbrunner, 2010; Kaczmirek et al., 2013; Nulty, Theocharis, Popa, arnet, & Benoit, 2016; Weber, Garimella, & Borra, 2012). The effect of abuse and incivility in online political discussion has also been explored. Vargo and Hopp (2017) find greater levels of online abuse in contex…”
BackgroundMethodInfluentialMubeen, M., Muskan, A., Akram, A., Rashid, J., Alshalali, T. A. N., & Sarwar, N. (2025) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-025-00919-z · Cited by 18
“The authors in [11] developed a multilingual hate speech classification model that utilizes multiple algorithmic stages combined with textual features.”
Chen, C., Qu, W. D., Su, S., Feng, Y., & Li, T. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114689 · Cited by 28
Resources for Multilingual Hate Speech Detection
Arango, A., Pérez, J. E. P., Poblete, B., Saldaña, M., & Proust, V. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.12 · Cited by 18
Killing me softly: Creative and cognitive aspects of implicitness in abusive language online
Frenda, S., Patti, V., & Rosso, P. (2022) · Natural Language Engineering · doi:10.1017/s1351324922000316 · Cited by 31
“Moreover, the existing surveys on abusive language detection (Schmidt and Wiegand 2017; Fortuna and Nunes 2018) underline the necessity to computationally approach the implicitness of toxic discourses, especially in the cases where these are disguised by sarcasm, euphemism, rhetorical questions, litotes, or where ther…”
BackgroundAsian hate speech detection on Twitter during COVID-19
Toliyat, A., Levitan, S. I., Peng, Z., & Etemadpour, R. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.932381 · Cited by 18
“(Facebook, 2022) (Fortuna and Nunes, 2018).”
BackgroundPassive ship detection and classification using hybrid cepstrums and deep compound autoencoders
Kamalipour, M., Agahi, H., Khishe, M., & Mahmoodzadeh, A. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-08075-7 · Cited by 19
“These methods are simple yet effective for the signal processing tasks that occur in the homogeneous atmosphere propagation environment, including music and speech processing methods [59–61].”
MethodGraphBERT: Bridging Graph and Text for Malicious Behavior Detection on Social Media
Wu, J., Zhang, C., Liu, Z., Zhang, E., Wilson, S. L., & Zhang, C. (2022) · Proceedings · doi:10.1109/icdm54844.2022.00065 · Cited by 29
“Among those methods, Natural Language Processing (NLP) methods have been widely used to detect and reduce online malicious behaviors, especially hateful speech [9].”
MethodPersian offensive language detection
Kebriaei, E., Homayouni, A., Faraji, R., Razavi, A., Shakery, A., Faili, H., & Yaghoobzadeh, Y. (2023) · Machine Learning · doi:10.1007/s10994-023-06370-5 · Cited by 27
“The availability of high-quality data-sets is a crucial factor in advancing the field of offensive language detection for many languages (Fortuna and Nunes, 2018; Golbeck et al., 2017), including Persian (Alavi et al., 2021).”
Coded Term Discovery for Online Hate Speech Detection
Kikkisetti, D., Mustafa, R. U., Melillo, W., Corizzo, R., Boukouvalas, Z., Gill, J., & Japkowicz, N. (2024) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa61799.2024.10722816 · Cited by 18
“Previous work aims to detect hate speech from social media using various Machine Learning (ML) methods as documented by a number of surveys written in the last six years [3], [7], [15], [17].”
BackgroundAlShannaq, F. B., Shehab, M., Alshorman, A., Hammad, M., HAMMO, B. H., & Alomari, W. (2025) · Expert Systems · doi:10.1111/exsy.13826 · Cited by 18
“…or animosity toward individuals or communities based on particular traits, including their physical attributes, religion, descent, sexual identity Fortuna and Nunes (2018) Offensive or abusive language (Off) Employing offensive, disparaging, or harmful language can leave the recipient feeling…”
The unappreciated role of intent in algorithmic moderation of abusive content on social media
Wang, X., Koneru, S., Venkit, P. N., Frischmann, B. M., & Rajtmajer, S. M. (2025) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-180 · Cited by 20
Tracking Hate in Social Media: Evaluation, Challenges and Approaches
Modha, S., Mandl, T., Majumder, P., & Patel, D. D. (2020) · SN Computer Science · doi:10.1007/s42979-020-0082-0 · Cited by 22
“However, many definitions are offered in the literature [17, 35].”
BackgroundInfluentialKnowMeme: A Knowledge-enriched Graph Neural Network Solution to Offensive Meme Detection
Shang, L., Youn, C., Zha, Y., Zhang, Y., & Wang, D. (2021) · IEEE International Conference on eScience · doi:10.1109/escience51609.2021.00029 · Cited by 27
Shevtsov, A., Tzagkarakis, C., Antonakaki, D., & Ioannidis, S. (2022) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v16i1.19349 · Cited by 30
“It has been observed that Twitter bots can also be exploited to spread fake news, rumors and hate speech (Founta et al. 2018; Fortuna and Nunes 2018; Burnap and Williams 2015) by instantly republishing low credibility Twitter content (Shao et al.”
BackgroundWijaya, K., & Setiawan, E. B. (2023) · Jurnal Ilmiah Teknik Elektro Komputer dan Informatika · doi:10.26555/jiteki.v9i3.26532 · Cited by 18
Online Abuse toward Candidates during the UK General Election 2019: Working PaperPreprint
Gorrell, G., Bakir, M. E., Roberts, I., Greenwood, M., & Bontcheva, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.08686 · Cited by 22
“review of prior work and methods, as do Fortuna and Nunes [14].”
BackgroundMethodInfluentialWhat's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus
Luccioni, A. S., & Viviano, J. D. (2021) · arXiv · doi:10.48550/arxiv.2105.02732 · Cited by 17
“…sources such as Twitter (Founta et al., 2018; Basile et al., 2019; Waseem and Hovy, 2016) and Reddit (Tadesse et al., 2019; Farrell et al., 2019), with both ML-based (Badjatiya et al., 2017) and count-based approaches (Davidson et al., 2017) achieving comparable results (Fortuna and Nunes, 2018).”
ResultCapozzi, A. T. E., Lai, M., Basile, V., Musto, C., Polignano, M., Poletto, F., Sanguinetti, M., Bosco, C., Patti, V., Ruffo, G., et al. (2019) · Cited by 17
Aggression Identification in English, Hindi and Bangla Text using BERT, RoBERTa and SVM.
Baruah, A., Das, K., Barbhuiya, F. A., & Dey, K. (2020) · Cited by 17
Toxic language detection: A systematic review of Arabic datasets
Bensalem, I., Rosso, P., & Zitouni, H. (2024) · Expert Systems · doi:10.1111/exsy.13551 · Cited by 17
“Given our use of toxic language as an umbrella term encompassing various related concepts, we have strived to amass a wide array of associated terms sourced from surveys and taxonomies (Banko et al., 2020; Fortuna & Nunes, 2019; Jahan & Oussalah, 2023).”
BackgroundSievers, B., & Thornton, M. A. (2024) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsae014 · Cited by 18
“Speech can be automatically transcribed to text (Fortuna and Nunes, 2018), and the text converted to vectors representing semantic features (Devlin et al ., 2018).”
BackgroundTahir, I., & Ramadhan, M. G. F. (2024) · LLT Journal A Journal on Language and Language Teaching · doi:10.24071/llt.v27i1.8180 · Cited by 23
“The willingness of people to express their opinions online has contributed to the propagation of hate speech (Fortuna & Nunes, 2018).”
BackgroundExplainable hate speech detection using LIME
Imbwaga, J. L., Chittaragi, N. B., & Koolagudi, S. G. (2024) · International Journal of Speech Technology · doi:10.1007/s10772-024-10135-3 · Cited by 17
“These platforms have also acted as channels to propagate hate speech (Fortuna & Nunes, 2018) and other related social phenomena such as fake news (Reis et al., 2019), and cyber-bullying (Reynolds et al., 2011) due to the unregulated generated content.”
Wang, B., Ding, Y., Liu, S., & Zhou, X. (2019) · FIRE (Working Notes) · Cited by 17
Hybrid Emoji-Based Masked Language Models for Zero-Shot Abusive Language Detection
Corazza, M., Menini, S., Cabrio, E., Tonelli, S., & Villata, S. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.84 · Cited by 24
“An increasing number of approaches has been proposed to detect this kind of messages (for a survey on the task, see (Schmidt and Wiegand, 2017) and (Fortuna and Nunes, 2018)).”
A GA-Based Approach to Fine-Tuning BERT for Hate Speech Detection
Madukwe, K. J., Gao, X., & Xue, B. (2020) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308419 · Cited by 19
“The success of this detection and classification task varies amongst these methods, with some outperforming the others as seen in the literature [9]–[14].”
BackgroundResultBias and comparison framework for abusive language datasets
Wich, M., Eder, T., Kuwatly, H. A., & Groh, G. (2021) · AI and Ethics · doi:10.1007/s43681-021-00081-0 · Cited by 23
“These problems emerge from differences between the datasets, such as context, platform, sampling process, and labeling, with even the task definition being subtly different in many cases [14].”
BackgroundAlmatarneh, S., Gamallo, P., & Ribadas, F. J. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2068 · Cited by 17
“The scientific study of hate speech, from a computer science point of view, is recent, and the number of studies in the field is low (Fortuna and Nunes, 2018).”
BackgroundOn Analyzing Annotation Consistency in Online Abusive Behavior DatasetsPreprint
Awal, R., Cao, R., Lee, R. K.-W., & Mitrović, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.13507 · Cited by 17
“The recent surveys (Schmidt and Wiegand, 2017; Fortuna and Nunes, 2018) have comprehensively summarized these methods.”
BackgroundModeling the Second Player in Distributionally Robust OptimizationPreprint
Michel, P., Hashimoto, T., & Neubig, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.10282 · Cited by 26
“Identifying online abuse on the internet is a crucial challenge, and has garnered much interest in the NLP community (Schmidt & Wiegand, 2017; Fortuna & Nunes, 2018).”
A Large Language Model-Based Approach for Multilingual Hate Speech Detection on Social Media
Usman, M., Ahmad, M., Sidorov, G., Gelbukh, I., & Quintero, R. (2025) · Computers · doi:10.3390/computers14070279 · Cited by 16
Tackling Hate Speech in Low-resource Languages with Context Experts
Nkemelu, D., Shah, H. S., Best, M. L., & Essa, I. A. (2022) · International Conference on Information and Communication Technologies and Development (ICTD) · doi:10.1145/3572334.3572372 · Cited by 29
Hettiachchi, D., Holcombe-James, I., Livingstone, S., de Silva, A., Lease, M., Salim, F. D., & Sanderson, M. (2023) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v11i1.27546 · Cited by 29
Giorgi, S., Bellew, D., Habib, D. R. S., Sedoc, J., Smitterberg, C. W., Devoto, A., Himelein-Wachowiak, M., & Curtis, B. L. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31328 · Cited by 19
Detecting Online Hate Speech: Approaches Using Weak Supervision and Network Embedding Models
Ridenhour, M., Bagavathi, A., Raisi, E., & Krishnan, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61255-9_20 · Cited by 16
Capozzi, A. T. E., Lai, M., Basile, V., Poletto, F., Sanguinetti, M., Bosco, C., Patti, V., Ruffo, G., Musto, C., Polignano, M., et al. (2020) · Italian Journal of Computational Linguistics · doi:10.4000/ijcol.659 · Cited by 21
Targets and Aspects in Social Media Hate Speech
Shvets, A., Fortuna, P., Juan Soler-Company, & Wanner, L. (2021) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2021.woah-1.19 · Cited by 16
Self-citationMethods for Detoxification of Texts for the Russian Language
Dementieva, D., Moskovskiy, D., Logacheva, V., Dale, D., Kozlova, O., Semenov, N., & Panchenko, A. (2021) · Multimodal Technologies and Interaction · doi:10.3390/mti5090054 · Cited by 16
Online Deep Learning from Doubly-Streaming Data
Lian, H., Atwood, J. S., Hou, B., Wu, J., & He, Y. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548355 · Cited by 21
“Examples include multimedia retrieval [63, 64], online speech analytics [16, 18], recommender systems [11, 13, 66, 68, 69], to just name a few.”
BackgroundDetecting Abusive Language on Online Platforms: A Critical Analysis.Preprint
Nakov, P., Nayak, V., Dent, K., Bhatawdekar, A., Sarwar, S. M., Hardalov, M., Dinkov, Y., Zlatkova, D., Bouchard, G., & Augenstein, I. (2021) · arXiv (Cornell University) · Cited by 25
An Information Retrieval Approach to Building Datasets for Hate Speech DetectionPreprint
Rahman, M. M., Balakrishnan, D., Murthy, D., Kutlu, M., & Lease, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09775 · Cited by 23
“What constitutes hate speech is also fairly subjective, with a multitude of varying definitions found across legal, regional, platform, and personal contexts [7, 13, 47].”
BackgroundHighly Generalizable Models for Multilingual Hate Speech DetectionPreprint
Deshpande, N., Farris, N., & Kumar, V. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.11294 · Cited by 16
“Hate speech detection online has increasingly become an important domain of research in the past decade [15].”
In conversation with Artificial Intelligence: aligning language models with human values
Kasirzadeh, A., & Gabriel, I. (2022) · arXiv · doi:10.48550/arxiv.2209.00731 · Cited by 128
“38For a recent survey of attempts to reduce ‘toxic speech’, see Fortuna and Nunes [34]. consideration.”
BackgroundEfficient Hate Speech Detection: Evaluating 38 Models from Traditional Methods to Transformers
Abusaqer, M., Saquer, J. M., & Shatnawi, H. (2025) · ACM Southeast Regional Conference (ACMSE) · doi:10.1145/3696673.3723061 · Cited by 23
“Research in automated hate speech detection spans multiple disciplines, encompassing both social science and computer science perspectives [12].”
S, J. M., & A, A. P. J. (2020) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/2020.wnut-1.66 · Cited by 20
Abdelhakim, M., Liu, B., & Sun, C. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120888 · Cited by 15
Semi-meta-supervised hate speech detection
Putra, C. D., & Wang, H. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111386 · Cited by 15
Classification of intent in moderating online discussions: An empirical evaluation
Gehweiler, C., & Lobachev, O. (2024) · Decision Analytics Journal · doi:10.1016/j.dajour.2024.100418 · Cited by 19
Polli, C., & Sindoni, M. G. (2024) · Discourse Context & Media · doi:10.1016/j.dcm.2024.100755 · Cited by 17
Multi-Modal Hate Speech Recognition Through Machine Learning
Irfan, A., Azeem, D., Narejo, S., & Kumar, N. (2024) · IEEE Karachi Section Humanitarian Technology Conference (KHI-HTC) · doi:10.1109/khi-htc60760.2024.10482031 · Cited by 15
Metadata
- Authors
- Fortuna, P., & Nunes, S.
- Affiliations
- INESC TEC (PT)
- Universidade do Porto (PT)
- Venue
- ACM Computing Surveys
- Published
- 2018-07-31
- Volume
- 51
- Issue
- 4
- Pages
- 1-30
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W2887782043
- Cited by
- 1,241
- References
- 52
- Citations by year
- 2026: 139 · 2025: 195 · 2024: 141 · 2023: 185 · 2022: 158 · 2021: 199 · 2020: 144 · 2019: 71 · 2018: 8
- FWCI
- 59.53
- Type
- journal article
- Language
- en
- Open access
- Locations
- ACM Computing Surveys (publishedVersion)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, other-oa)
- Funders
- FourEyes, a research line within project TEC4Growth—Pervasive Intelligence
- North Portugal Regional Operational Programme
- European Regional Development Fund
- Enhancers and Proofs of Concept with Industrial (Impact/NORTE-01- 0145-FEDER-000020)
- PORTUGAL 2020 Partnership Agreement
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- hate speech detection, text classification, definition of hate speech, annotated dataset, multilingual dataset, online communities, digital media platforms, social impact
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.