Hate Lingo: A Target-Based Linguistic Analysis of Hate Speech in Social Media
Open accessAbstract
While social media empowers freedom of expression and individual voices, it also enables anti-social behavior, online harassment, cyberbullying, and hate speech. In this paper, we deepen our understanding of online hate speech by focusing on a largely neglected but crucial aspect of hate speech -- its target: either directed towards a specific person or entity, or generalized towards a group of people sharing a common protected characteristic. We perform the first linguistic and psycholinguistic analysis of these two forms of hate speech and reveal the presence of interesting markers that distinguish these types of hate speech. Our analysis reveals that Directed hate speech, in addition to being more personal and directed, is more informal, angrier, and often explicitly attacks the target (via name calling) with fewer analytic words and more words suggesting authority and influence. Generalized hate speech, on the other hand, is dominated by religious hate, is characterized by the use of lethal words such as murder, exterminate, and kill; and quantity words such as million and many. Altogether, our work provides a data-driven analysis of the nuances of online-hate speech that enables not only a deepened understanding of hate speech and its social implications, but also its detection.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The dataset used to train HateSonar was built using a Hatebase hate speech vocabulary, which is likely to create non-representative training data with other nuances of hate uncounted for.
“For example, the dataset used to train HateSonar was built using a Hatebase 13 hate speech vocabulary, including commonly used terms and phrases on the internet, which is likely to create non-representative training data with other nuances of hate uncounted for.”
From Misogynoir: challenges in detecting intersectional hate (2022) · cites this paper as [13]
The Data and Methods section (page unavailable) says the authors used Hatebase terms to build a keyphrase-based Twitter dataset and notes that filters based on specific hate terms can miss evasive spellings. It does not identify this as HateSonar’s training dataset or establish that HateSonar’s training data is non-representative. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610 · see citing works
Baker, C. F., Fillmore, C. J., & Lowe, J. B. (1998) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/980845.980860 · Cited by 2,565 · see citing works
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Talat, Z., & Hovy, D. (2016) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n16-2013 · Cited by 1,919 · see citing works
A Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A. G., & Wiegand, M. (2017) · International Workshop on Natural Language Processing for Social Media (SocialNLP) · doi:10.18653/v1/w17-1101 · Cited by 1,447 · see citing works
Named Entity Recognition in Tweets: An Experimental Study
Ritter, A., Clark, S., & Etzioni, O. (2011) · Cited by 1,192 · see citing works
Abusive Language Detection in Online User Content
Nobata, C., Tetreault, J., Thomas, A. O., Mehdad, Y., & Chang, Y. (2016) · International World Wide Web Conference (WWW) · doi:10.1145/2872427.2883062 · Cited by 1,160 · see citing works
FrameNet II: Extended theory and practice
Ruppenhofer, J., Ellsworth, M., Petruck, M. R. L., Johnson, C. R., & Scheffczyk, J. (2006) · Publication Server of the Institute for German Language (Institute for German Language) · Cited by 1,065 · see citing works
Is the Sample Good Enough? Comparing Data from Twitter's Streaming API with Twitter's Firehose
Morstatter, F., Pfeffer, J., Liu, H., & Carley, K. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14401 · Cited by 863 · see citing works
Tufekci, Z. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14517 · Cited by 780 · see citing works
Hate Speech Detection with Comment Embeddings
Djuric, N., Zhou, J. L., Morris, R., Grbovic, M., Radosavljević, V., & Bhamidipati, N. L. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2740908.2742760 · Cited by 725 · see citing works
Ex Machina: Personal Attacks Seen at Scale
Wulczyn, E., Thain, N., & Dixon, L. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052591 · Cited by 636 · see citing works
Burnap, P., & Williams, M. L. (2015) · Policy & Internet · doi:10.1002/poi3.85 · Cited by 615 · see citing works
A Lexicon-based Approach for Hate Speech Detection
Gitari, N. D., Zhang, Z., Hanyurwimfura, D., & Long, J. (2015) · International Journal of Multimedia and Ubiquitous Engineering · doi:10.14257/ijmue.2015.10.4.21 · Cited by 506 · see citing works
Detecting Hate Speech on the World Wide Web
Warner, W. B., & Hirschberg, J. (2012) · Cited by 505 · see citing works
Cheng, J., Bernstein, M. S., Danescu-Niculescu-Mizil, C., & Leskovec, J. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998213 · Cited by 499 · see citing works
Us versus them: social identity shapes neural responses to intergroup competition and harm
Cikara, M., Botvinick, M. M., & Fiske, S. T. (2011) · Psychological Science · doi:10.1177/0956797610397667 · Cited by 494 · see citing works
Understanding Abuse: A Typology of Abusive Language Detection Subtasks
Waseem, Z., Davidson, T., Warmsley, D., & Weber, I. G. (2017) · Workshop on Abusive Language Online (ALW) · doi:10.18653/v1/w17-3012 · Cited by 486 · see citing works
Common Sense Reasoning for Detection, Prevention, and Mitigation of Cyberbullying
Dinakar, K., Jones, B., Havasi, C., Lieberman, H. A., & Picard, R. W. (2012) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2362394.2362400 · Cited by 434 · see citing works
What to Expect When the Unexpected Happens
Olteanu, A., Vieweg, S. E., & Castillo, C. (2015) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2675133.2675242 · Cited by 403 · see citing works
What Are the Police Doing on Twitter? Social Media, the Police and the Public
Crump, J. (2011) · Policy & Internet · doi:10.2202/1944-2866.1130 · Cited by 296 · see citing works
Automated Hate Speech Detection and the Problem of Offensive LanguagePreprint
Davidson, T., Warmsley, D., Macy, M. W., & Weber, I. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04009 · Cited by 284 · see citing works
Sparse Additive Generative Models of Text
Eisenstein, J., Ahmed, A. A., & Xing, E. P. (2018) · Figshare · doi:10.1184/r1/6476342 · Cited by 279 · see citing works
Ex Machina: Personal Attacks Seen at ScalePreprint
Wulczyn, E., Thain, N., & Dixon, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.08914 · Cited by 260 · see citing works
Smokey: Automatic Recognition of Hostile Messages
Ellen, & Spertus (1997) · Cited by 217 · see citing works
Analyzing the Targets of Hate in Online Social MediaPreprint
da Silva, L. H. G., Mondal, M., Correa, D., Benevenuto, F., & Weber, I. G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.07709 · Cited by 204 · see citing works
Do Characters Abuse More Than Words?
Mehdad, Y., & Tetreault, J. (2016) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w16-3638 · Cited by 197 · see citing works
Smokey: automatic recognition of hostile messages
Spertus, E. (1997) · Cited by 170 · see citing works
Detecting tension in online communities with computational Twitter analysis
Burnap, P., RANA, O. F., Avis, N. J., Williams, M. L., Housley, W., Edwards, A., Morgan, J., & Sloan, L. (2013) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2013.04.013 · Cited by 170 · see citing works
Detection and Fine-Grained Classification of Cyberbullying Events
Hee, C. V., Lefever, E., Verhoeven, B., Mennes, J., Desmet, B., Pauw, G. D., Daelemans, W. M. P., & Hoste, V. (2015) · Ghent University Academic Bibliography (Ghent University) · Cited by 154 · see citing works
Automatic identification of personal insults on social news sites
Sood, S. O., Churchill, E. F., & Antin, J. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21690 · Cited by 153 · see citing works
Profanity use in online communities
Sood, S. O., Antin, J., & Churchill, E. F. (2012) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/2207676.2208610 · Cited by 145 · see citing works
A Web of Hate: Tackling Hateful Speech in Online Social Spaces
Saleem, H. M., Dillon, K. P., Benesch, S., & Ruths, D. (2017) · arXiv · doi:10.48550/arxiv.1709.10159 · Cited by 144 · see citing works
Sellars, A. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2882244 · Cited by 130 · see citing works
Intermediaries and Hate Speech: Fostering Digital Citizenship for our Information Age
Citron, D. K., & Norton, H. (2011) · Digital Commons at University of Maryland Carey Law (University of Maryland Francis King Carey School of Law) · Cited by 128 · see citing works
Anyone Can Become a Troll: Causes of Trolling Behavior in Online DiscussionsPreprint
Cheng, J., Bernstein, M. S., Danescu-Niculescu-Mizil, C., & Leskovec, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.01119 · Cited by 122 · see citing works
Hine, G. E., Onaolapo, J., Cristofaro, E. D., et al. (2017) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.841769 · Cited by 77 · see citing works
Hine, G. E., Onaolapo, J. O., Cristofaro, E. D., Kourtellis, N., Ilias Leontiadis, A., Samaras, R., Stringhini, G., & Blackburn, J. (2017) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.841768 · Cited by 74 · see citing works
(2015) · Cited by 69 · see citing works
SEMAFOR: Frame Argument Resolution with Log-Linear Models
Chen, D., Schneider, N., Das, D., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473675 · Cited by 46 · see citing works
Hate speech, sex speech, free speech
(1997) · Choice Reviews Online · doi:10.5860/choice.35-1939 · Cited by 28 · see citing works
Historical Analysis of Legal Opinions with a Sparse Mixed-Effects Latent Variable Model
Wang, W. Y., Mayfield, E., Naidu, S., & Dittmar, J. E. (2012) · London School of Economics and Political Science Research Online (London School of Economics and Political Science) · Cited by 22 · see citing works
Using Frame Semantics for Knowledge Extraction from Twitter
Søgaard, A., Plank, B., & Alonso, H. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9524 · Cited by 22 · see citing works
Discovering Factions in the Computational Linguistics Community
Sim, Y., Smith, N. A., & Smith, D. A. (2018) · Figshare · doi:10.1184/r1/6473315 · Cited by 18 · see citing works
Hate Speech, Sex Speech, Free Speech
Wolfson, N. (1997) · Praeger eBooks · doi:10.5040/9798400662003 · Cited by 14 · see citing works
Understanding Abuse: A Typology of Abusive Language Detection Subtasks
Waseem, Z., Davidson, T., Warmsley, D., & Weber, I. (2017) · arXiv · doi:10.48550/arxiv.1705.09899
Flagged first, then by citations
2 of the 47 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- [4]4. Chen, D.; Schneider, N.; Das, D.; and Smith, N. A. 2010. Se- mafor: Frame Argument Resolution with Log-linear Mod- els. In Proceedings of the 5th International Workshop on Semantic Evaluation.
Cited by
Sorted by citations
Social Media, Echo Chambers, and Political Polarization
Barberá, P. (2020) · Social Media and Democracy · doi:10.1017/9781108890960.004 · Cited by 373 · see citing works
Developing an online hate classifier for multiple social media platforms
Salminen, J. O., Hopf, M., Chowdhury, S. A., Jung, S., Almerekhi, H., & Jansen, B. J. J. (2020) · Human-centric Computing and Information Sciences · doi:10.1186/s13673-019-0205-6 · Cited by 295 · see citing works
Misinformation, Disinformation, and Online Propaganda
Guess, A. M., & Lyons, B. A. (2020) · Social Media and Democracy · doi:10.1017/9781108890960.003 · Cited by 284 · see citing works
Vulnerable community identification using hate speech detection on social media
Mossie, Z., & Wang, J. (2019) · Information Processing & Management · doi:10.1016/j.ipm.2019.102087 · Cited by 189 · see citing works
A Benchmark Dataset for Learning to Intervene in Online Hate Speech
Qian, J., Bethke, A., Liu, Y., Belding, E., & Wang, W. Y. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1482 · Cited by 180 · see citing works
Self-citationSoK: Hate, Harassment, and the Changing Landscape of Online Abuse
Thomas, K., Akhawe, D., Bailey, M., Boneh, D., Bursztein, E., Consolvo, S., Dell, N. L., Durumeric, Z., Kelley, P. G., Kumar, D., et al. (2021) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40001.2021.00028 · Cited by 159 · see citing works
A Literature Review of Textual Hate Speech Detection Methods and Datasets
Alkomah, F., & Ma, X. (2022) · Information · doi:10.3390/info13060273 · Cited by 152 · see citing works
‘Welcome to #GabFam’: Far-right virtual community on Gab
Jasser, G., McSwiney, J., Pertwee, E., & Zannettou, S. (2021) · New Media & Society · doi:10.1177/14614448211024546 · Cited by 138 · see citing works
A Framework for the Computational Linguistic Analysis of Dehumanization
Mendelsohn, J., Tsvetkov, Y., & Jurafsky, D. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00055 · Cited by 133 · see citing works
Facts and Where to Find Them: Empirical Research on Internet Platforms and Content Moderation
Keller, D., & Leerssen, P. (2020) · Data Archiving and Networked Services (DANS) · doi:10.1017/9781108890960
Siegel, A. A. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.005 · Cited by 117 · see citing works
DeepHateExplainer: Explainable Hate Speech Detection in Under-resourced Bengali Language
Karim, M. R., Dey, S. K., Islam, T., Sarker, S., Menon, M. H., Hossain, K., Hossain, M. A., & Decker, S. (2021) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa53316.2021.9564230 · Cited by 114 · see citing works
Automatic Classification of Sexism in Social Networks: An Empirical Study on Twitter Data
Rodríguez-Sanchez, F. J., Carrillo‐de‐Albornoz, J., & Plaza, L. (2020) · IEEE Access · doi:10.1109/access.2020.3042604 · Cited by 114 · see citing works
A Quantitative Approach to Understanding Online Antisemitism
Zannettou, S., Finkelstein, J., Bradlyn, B., & Blackburn, J. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7343 · Cited by 104 · see citing works
Rieger, D., Kümpel, A. S., Wich, M., Kiening, T., & Groh, G. (2021) · Social Media + Society · doi:10.1177/20563051211052906 · Cited by 102 · see citing works
Karim, M. R., Chakravarthi, B. R., McCrae, J. P., & Cochez, M. (2020) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa49011.2020.00053 · Cited by 105 · see citing works
Pamungkas, E. W., & Patti, V. (2019) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/p19-2051 · Cited by 98 · see citing works
Uyheng, J., & Carley, K. M. (2020) · Journal of Computational Social Science · doi:10.1007/s42001-020-00087-4 · Cited by 96 · see citing works
Mariconti, E., Suárez‐Tangil, G., Blackburn, J., Cristofaro, E. D., Kourtellis, N., Ilias Leontiadis, A., Serrano, J. L., & Stringhini, G. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359309 · Cited by 90 · see citing works
Gerts, D., Shelley, C. D., Parikh, N., Pitts, T., Ross, C. W., Fairchild, G. C., Chavez, N. Y. V., & Daughton, A. R. (2021) · JMIR Public Health and Surveillance · doi:10.2196/26527 · Cited by 84 · see citing works
Indurthi, V., Syed, B., Shrivastava, M., Chakravartula, N., Gupta, M., & Varma, V. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2009 · Cited by 83 · see citing works
Racism is a Virus: Anti-Asian Hate and Counterhate in Social Media during the COVID-19 CrisisPreprint
Ziems, C., He, B., Soni, S., & Kumar, S. (2020) · arXiv (Cornell University) · Cited by 81 · see citing works
Rudnicki, K., Vandebosch, H., Voué, P., & Poels, K. (2022) · Behaviour and Information Technology · doi:10.1080/0144929x.2022.2027013 · Cited by 80 · see citing works
Mansur, Z., Omar, N., & Tiun, S. B. (2023) · IEEE Access · doi:10.1109/access.2023.3239375 · Cited by 68 · see citing works
Hate speech detection in social media: Techniques, recent trends, and future challenges
Rawat, A., Kumar, S., & Samant, S. S. (2024) · Wiley Interdisciplinary Reviews Computational Statistics · doi:10.1002/wics.1648 · Cited by 68 · see citing works
Toxic language in online incel communities
Pelzer, B., Kaati, L., Cohen, K., & Fernquist, J. (2021) · SN Social Sciences · doi:10.1007/s43545-021-00220-8 · Cited by 67 · see citing works
Democratic Transparency in the Platform Society
Gorwa, R., & Ash, T. G. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.013 · Cited by 67 · see citing works
Nikolov-Radivchev at SemEval-2019 Task 6: Offensive Tweet Classification with BERT and Ensembles
Nikolov, A., & Radivchev, V. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2123 · Cited by 69 · see citing works
Misinformation and Its Correction
Wittenberg, C. E., & Berinsky, A. J. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.009 · Cited by 63 · see citing works
Multimodal Hate Speech Detection from Bengali Memes and Texts
Karim, M. R., Dey, S. K., Islam, T., Shajalal, M., & Chakravarthi, B. R. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-33231-9_21 · Cited by 60 · see citing works
Analyzing the hate and counter speech accounts on TwitterPreprint
Mathew, B., Kumar, N. R., Ravina, R., Goyal, P., & Mukherjee, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.02712 · Cited by 58 · see citing works
SoK: Content Moderation in Social Media, from Guidelines to Enforcement, and Research to Practice
Singhal, M., Ling, C., Paudel, P., Thota, P., Kumarswamy, N., Stringhini, G., & Nilizadeh, S. (2023) · 2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp57164.2023.00056 · Cited by 57 · see citing works
AngryBERT: Joint Learning Target and Emotion for Hate Speech Detection
Awal, R., Cao, R., Lee, R. K.-W., & Mitrović, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-75762-5_55 · Cited by 56 · see citing works
Characterizing network dynamics of online hate communities around the COVID-19 pandemic
Uyheng, J., & Carley, K. M. (2021) · Applied Network Science · doi:10.1007/s41109-021-00362-x · Cited by 52 · see citing works
del Valle-Cano, G., Quijano-Sánchez, L., Liberatore, F., & Gómez, J. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.119446 · Cited by 51 · see citing works
HateGAN: Adversarial Generative-Based Data Augmentation for Hate Speech Detection
Cao, R., & Lee, R. K.-W. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.557 · Cited by 51 · see citing works
Pavlopoulos, J., Thain, N., Dixon, L., & Androutsopoulos, I. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2102 · Cited by 49 · see citing works
Amores, J. J., Blanco‐Herrero, D., Holgado, P. S., & Frías-Vázquez, M. (2021) · Cuadernos info · doi:10.7764/cdi.49.27817 · Cited by 48 · see citing works
Aiello, L. M., Quercia, D., Zhou, K., Constantinides, M., Šćepanović, S., & Joglekar, S. P. (2021) · Humanities and Social Sciences Communications · doi:10.1057/s41599-021-00861-3 · Cited by 47 · see citing works
"A Virus Has No Religion": Analyzing Islamophobia on Twitter During the COVID-19 Outbreak
Chandra, M., Reddy, M., Sehgal, S., Gupta, S., Buduru, A. B., & Kumaraguru, P. (2021) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3465336.3475111 · Cited by 44 · see citing works
Challenges of Moderating Social Virtual Reality
Sabri, N., Chen, B., Teoh, A., Dow, S. P., Vaccaro, K., & ElSherief, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581329 · Cited by 43 · see citing works
Self-citationJeon, Y., Kim, B., Xiong, A., Lee, D., & Han, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479859 · Cited by 42 · see citing works
Understanding Visual Memes: An Empirical Analysis of Text Superimposed on Memes Shared on Twitter
Du, Y., Masood, M. A., & Joseph, K. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7287 · Cited by 40 · see citing works
A Deep Learning Framework for the Detection of Malay Hate Speech
Maity, K., Bhattacharya, S., Saha, S., & Seera, M. (2023) · IEEE Access · doi:10.1109/access.2023.3298808 · Cited by 39 · see citing works
Bots and Computational Propaganda: Automation for Communication and Control
Woolley, S. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.006 · Cited by 39 · see citing works
Towards a Comprehensive Taxonomy and Large-Scale Annotated Corpus for Online Slur Usage
Kurrek, J., Saleem, H. M., & Ruths, D. A. (2020) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2020.alw-1.17 · Cited by 38 · see citing works
Facts and Where to Find Them: Empirical Research on Internet Platforms and Content Moderation
Keller, D., & Leerssen, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.011 · Cited by 37 · see citing works
Sequential targeting: A continual learning approach for data imbalance in text classification
Jang, J., Kim, Y., Choi, K.-H., & Suh, S. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115067 · Cited by 40 · see citing works
A Deep Dive into Multilingual Hate Speech Classification
Aluru, S. S., Mathew, B., Saha, P., & Mukherjee, A. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-67670-4_26 · Cited by 37 · see citing works
Balayn, A., Yang, J., Szlávik, Z., & Bozzon, A. (2021) · ACM Transactions on Social Computing · doi:10.1145/3479158 · Cited by 33 · see citing works
Hate Speech and Counter Speech Detection: Conversational Context Does Matter
Yu, X., Blanco, E., & Hong, L. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.433 · Cited by 34 · see citing works
An Investigation of Large Language Models for Real-World Hate Speech Detection
Guo, K., Hu, A., Mu, J., Shi, Z., Zhao, Z., Vishwamitra, N., & Hu, H. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00237 · Cited by 33 · see citing works
Democratic Creative Destruction? The Effect of a Changing Media Landscape on Democracy
Nielsen, R. K., & Fletcher, R. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.008 · Cited by 35 · see citing works
Comparing the Language of QAnon-Related Content on Parler, Gab, and Twitter
Sipka, A., Hannák, A., & Urman, A. (2021) · Web Science Conference · doi:10.1145/3501247.3531550 · Cited by 36 · see citing works
Towards offensive language detection and reduction in four Software Engineering communities
Cheriyan, J., Savarimuthu, B. T. R., & Cranefield, S. (2021) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3463274.3463805 · Cited by 31 · see citing works
Solovev, K., & Pröllochs, N. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512261 · Cited by 31 · see citing works
Political Hate Speech Detection and Lexicon Building: A Study in Taiwan
Wang, C., Day, M.-Y., & Wu, C.-L. (2022) · IEEE Access · doi:10.1109/access.2022.3160712 · Cited by 33 · see citing works
Misogynoir: challenges in detecting intersectional hate
Kwarteng, J., Perfumi, S. C., Farrell, T., Third, A., & Fernandez, M. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00993-7 · Cited by 30 · see citing works
“including studies by (ElSherief et al. 2018; Davidson et al. 2019; Cao et al. 2020), HateSonar has been used in a number of hate speech detection research studies to evaluate and compare other datasets and classifiers in studies by (Kim et al.”
MethodSharma, D., Singh, A., & Singh, V. K. (2024) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3653017 · Cited by 34 · see citing works
Exploring hate speech dynamics: The emotional, linguistic, and thematic impact on social media users
Ghenai, A., Noorian, Z., Moradisani, H., Abadeh, P., Erentzen, C. A., & Zarrinkalam, F. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104079 · Cited by 34 · see citing works
Towards Measuring Adversarial Twitter Interactions against Candidates in the US Midterm Elections
Hua, Y., Ristenpart, T., & Naaman, M. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7298 · Cited by 29 · see citing works
Hate begets Hate: A Temporal Study of Hate SpeechPreprint
Mathew, B., Illendula, A., Saha, P., Sarkar, S., Goyal, P., & Mukherjee, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.10966 · Cited by 29 · see citing works
A Method to Analyze Multiple Social Identities in Twitter Bios
Pathak, A., Madani, N., & Joseph, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479502 · Cited by 29 · see citing works
The Economics of Content Moderation: Theory and Experimental Evidence from Hate Speech on TwitterPreprint
Jiménez-Durán, R. (2022) · Social Science Research Network · doi:10.2139/ssrn.4044098 · Cited by 35 · see citing works
Comparative Media Regulation in the United States and Europe
Fukuyama, F., & Grotto, A. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.010 · Cited by 26 · see citing works
Calderón, C. A., Amores, J. J., Holgado, P. S., Vrysis, L., Vryzas, N., & Alonso, M. O. (2022) · Sustainability · doi:10.3390/su142013094 · Cited by 26 · see citing works
Comparative Evaluation of Label-Agnostic Selection Bias in Multilingual Hate Speech Datasets
Ousidhoum, N. D., Song, Y., & Yeung, D. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.199 · Cited by 25 · see citing works
Online Political Advertising in the United States
Fowler, E. F., Franz, M. M., & Ridout, T. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.007 · Cited by 25 · see citing works
Interpretable Multi-Modal Hate Speech DetectionPreprint
Vijayaraghavan, P., Larochelle, H., & Roy, D. K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.01616 · Cited by 25 · see citing works
Towards Automatic Detection and Explanation of Hate Speech and Offensive Language
Dorris, W., Hu, R., Vishwamitra, N., Luo, F., & Costello, M. (2020) · doi:10.1145/3375708.3380312 · Cited by 24 · see citing works
Chiril, P., Moriceau, V., Benamara, F., Mari, A., Origgi, G., & Coulomb‐Gully, M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.373 · Cited by 25 · see citing works
Hate Speech as an Indicator for the State of the Society
Schäfer, S., Sülflow, M., & Reiners, L. (2021) · Journal of Media Psychology Theories Methods and Applications · doi:10.1027/1864-1105/a000294 · Cited by 28 · see citing works
Maity, S., Chakraborty, A., Goyal, P., & Mukherjee, A. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274386 · Cited by 24 · see citing works
A New Hate Speech Detection System based on Textual and Psychological Features
Alkomah, F., Salati, S., & Ma, X. (2022) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2022.01308100 · Cited by 23 · see citing works
Chinivar, S., Roopa, M. S., Arunalatha, J. S., & Venugopal, K. R. (2022) · Entertainment Computing · doi:10.1016/j.entcom.2022.100544 · Cited by 25 · see citing works
Raiyani, K., Gonçalves, T., Quaresma, P., & Nogueira, V. (2018) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · Cited by 23 · see citing works
Conclusion: The Challenges and Opportunities for Social Media Research
Persily, N., & Tucker, J. A. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.014 · Cited by 23 · see citing works
Us vs. Them: A Dataset of Populist Attitudes, News Bias and Emotions
Cabot, .-L. H., Abadi, D., Fischer, A. H., & Shutova, E. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.165 · Cited by 22 · see citing works
Indonesian Tweets Hate Speech Target Classification using Machine Learning
Kurniawan, S. V., & Budi, I. D. (2020) · International Conference on Informatics and Computing (ICIC) · doi:10.1109/icic50835.2020.9288515 · Cited by 21 · see citing works
From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models
Mendelsohn, J., Bras, R. J. L., Choi, Y., & Sap, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.845 · Cited by 21 · see citing works
MetaHate: A Dataset for Unifying Efforts on Hate Speech Detection
Piot, P., Martín-Rodilla, P., & Parapar, J. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31445 · Cited by 21 · see citing works
Platform-agnostic Model to Detect Sinophobia on Social Media
Morgan, M., & Kulkarni, A. (2023) · ACM Southeast Regional Conference (ACMSE) · doi:10.1145/3564746.3587024 · Cited by 19 · see citing works
jhan014 at SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media
Han, J., Wu, S., & Liu, X. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2116 · Cited by 18 · see citing works
Minarti, M., Rahmah, M. N., Khalilurrahman, K., Samsir, S., & Mardiana, M. (2023) · Nidhomul Haq Jurnal Manajemen Pendidikan Islam · doi:10.31538/ndh.v8i2.3930 · Cited by 18 · see citing works
Mohdeb, D., Laifa, M., Zerargui, F., & Benzaoui, O. (2022) · Aslib Journal of Information Management · doi:10.1108/ajim-10-2021-0293 · Cited by 18 · see citing works
The Economics of Content Moderation: Theory and Experimental Evidence from Hate Speech on TwitterPreprint
Jiménez-Durán, R. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4590147 · Cited by 18 · see citing works
KnowMeme: A Knowledge-enriched Graph Neural Network Solution to Offensive Meme Detection
Shang, L., Youn, C., Zha, Y., Zhang, Y., & Wang, D. (2021) · IEEE International Conference on eScience · doi:10.1109/escience51609.2021.00029 · Cited by 17 · see citing works
A review on abusive content automatic detection: approaches, challenges and opportunities
Alrashidi, B., Jamal, A. T., Khan, I., & Alkhathlan, A. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.1142 · Cited by 20 · see citing works
Using Deep Learning to Detect Islamophobia on Reddit
Aldreabi, E., Lee, J. M., & Blackburn, J. (2023) · Proceedings of the ... International Florida Artificial Intelligence Research Society Conference · doi:10.32473/flairs.36.133324 · Cited by 16 · see citing works
How Hate Speech Varies by Target Identity: A Computational Analysis
Yoder, M. M., Ng, L. H. X., Brown, D. W., & Carley, K. M. (2022) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2022.conll-1.3 · Cited by 20 · see citing works
Detecting Online Hate Speech: Approaches Using Weak Supervision and Network Embedding Models
Ridenhour, M., Bagavathi, A., Raisi, E., & Krishnan, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61255-9_20 · Cited by 15 · see citing works
Levels of hate in online environments
Berglind, T., Pelzer, B., & Kaati, L. (2019) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1145/3341161.3343521 · Cited by 15 · see citing works
Opinion Conflicts: An Effective Route to Detect Incivility in TwitterPreprint
Maity, S. K., Chakraborty, A., Goyal, P., & Mukherjee, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.00289 · Cited by 15 · see citing works
The Effects of Social Approval Signals on the Production of Online Hate: A Theoretical Explication
Walther, J. B. (2024) · Communication Research · doi:10.1177/00936502241278944 · Cited by 16 · see citing works
Does terrorism trigger online hate speech? On the association of events and time series
Scharwächter, E., & Müller, E. (2020) · The Annals of Applied Statistics · doi:10.1214/20-aoas1338 · Cited by 14 · see citing works
Where do angry birds tweet? Income inequality and online hate in Italy
Denti, D., & Faggian, A. (2021) · Cambridge Journal of Regions Economy and Society · doi:10.1093/cjres/rsab016 · Cited by 14 · see citing works
Pelzer, B., Kaati, L., & Akrami, N. (2018) · IEEE International Conference on Intelligence and Security Informatics (ISI) · doi:10.1109/isi.2018.8587396 · Cited by 14 · see citing works
Benito, D., Araque, Ó., & Iglesias, C. (2019) · International Workshop on Semantic Evaluation · doi:10.18653/v1/s19-2070 · Cited by 11 · see citing works
Combating Online Hate Speech: The Impact of Legislation on TwitterPreprint
Andres, R., & Slivko, O. (2021) · Social Science Research Network · doi:10.2139/ssrn.4013662 · Cited by 10 · see citing works
- Authors
- ElSherief, M., Kulkarni, V., Nguyen, D., Yang Wang, W., & Belding, E.
- Affiliations
- University of California, Santa Barbara (US)
- Venue
- Proceedings of the International AAAI Conference on Web and Social Media
- Published
- 2018-06-15
- Volume
- 12
- Issue
- 1
- ISSN
- 2162-3449 (ISSN-L)
- 2334-0770
- OpenAlex
- W2797630160
- Cited by
- 294
- References
- 47
- Citations by year
- 2026: 16 · 2025: 34 · 2024: 25 · 2023: 44 · 2022: 37 · 2021: 43 · 2020: 54 · 2019: 34 · 2018: 6
- FWCI
- 13.93
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the International AAAI Conference on Web and Social Media (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Keywords
- online hate speech, social media, linguistic markers, psycholinguistic analysis, cyberbullying, online harassment, name-calling, hate speech detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
