Hate Speech Detection with Comment Embeddings
The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
We address the problem of hate speech detection in online user comments. Hate speech, defined as an "abusive speech targeting specific group characteristics, such as ethnicity, religion, or gender", is an important problem plaguing websites that allow users to leave feedback, having a negative impact on their online business and overall user experience. We propose to learn distributed low-dimensional representations of comments using recently proposed neural language models, that can then be fed as inputs to a classification algorithm. Our approach addresses issues of high-dimensionality and sparsity that impact the current state-of-the-art, resulting in highly efficient and effective hate speech detectors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Opinion Mining and Sentiment Analysis
Pang, B., & Lee, L. (2008) · now publishers, Inc. eBooks · doi:10.1561/9781601981516 · Cited by 6,720
Opinion mining and sentiment analysis
Pang, B., & Lee, L. (2008) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000011 · Cited by 6,372
Distributed Representations of Sentences and Documents
Le, Q. V., & Mikolov, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.4053 · Cited by 4,934
Detecting Hate Speech on the World Wide Web
Warner, W. B., & Hirschberg, J. (2012) · Cited by 964
Locate the Hate: Detecting Tweets against Blacks
Kwok, I., & Wang, Y. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8539 · Cited by 446
Burnap, P., & Williams, M. L. (2014) · ORCA Online Research @Cardiff (Cardiff University) · Cited by 108
Filtering Offensive Language in Online Communities using Grammatical Relations
Xu, Z., & Zhu, S. (2010) · Cited by 60
Research works printed but not linked: 5
- Burnap P., 2014, IPP
- Kwok I., 2013, AAAI
- Massaro T. M., 1990, Equality and freedom of expression: The hate speech dilemma, Wm. & Mary L. Rev., 32, 211
- Warner W., 2012, Workshop on Language in Social Media at ACL, 19
- Xu Z., 2010, Anti-Abuse and Spam Conference
Cited by
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
A Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A., & Wiegand, M. (2017) · Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media · doi:10.18653/v1/w17-1101 · Cited by 1,395
A Survey on Automatic Detection of Hate Speech in Text
Fortuna, P., & Nunes, S. (2019) · ACM Computing Surveys · doi:10.1145/3232676 · Cited by 1,241
“Some authors [25] use a paragraph2vec approach to classify language on user comments as abusive or clean and also to predict the central word in the message [25].”
BackgroundAbusive Language Detection in Online User Content
Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., & Chang, Y. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883062 · Cited by 1,122
Deep Learning for Hate Speech Detection in Tweets
Badjatiya, P., Gupta, S., Gupta, M., & Varma, V. (2017) · Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion · doi:10.1145/3041021.3054223 · Cited by 849
Predicting the Type and Target of Offensive Posts in Social Media
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., & Kumar, R. (2019) · Proceedings of the 2019 Conference of the North · doi:10.18653/v1/n19-1144 · Cited by 720
SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., & Kumar, R. (2019) · Proceedings of the 13th International Workshop on Semantic Evaluation · doi:10.18653/v1/s19-2010 · Cited by 701
Detecting Hate Speech on Twitter Using a Convolution-GRU Based Deep Neural Network
Zhang, Z., Robinson, D., & Tepper, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-93417-4_48 · Cited by 638
Ranjbarzadeh, R., Bagherian Kasgari, A., Jafarzadeh Ghoushchi, S., Anari, S., Naseri, M., & Bendechache, M. (2021) · Scientific Reports · doi:10.1038/s41598-021-90428-8 · Cited by 634
Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior
Founta, A., Djouvas, C., Chatzakou, D., Leontiadis, I., Blackburn, J., Stringhini, G., Vakali, A., Sirivianos, M., & Kourtellis, N. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.14991 · Cited by 596
Understanding Abuse: A Typology of Abusive Language Detection Subtasks
Waseem, Z., Davidson, T., Warmsley, D., & Weber, I. (2017) · Proceedings of the First Workshop on Abusive Language Online · doi:10.18653/v1/w17-3012 · Cited by 474
Watanabe, H., Bouazizi, M., & Ohtsuki, T. (2018) · IEEE Access · doi:10.1109/access.2018.2806394 · Cited by 467
A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media
Mozafari, M., Farahbakhsh, R., & Crespi, N. (2020) · Studies in Computational Intelligence · doi:10.1007/978-3-030-36687-2_77 · Cited by 396
Deep Learning for Detecting Cyberbullying Across Multiple Social Media Platforms
Agrawal, S., & Awekar, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-76941-7_11 · Cited by 387
Social Media, Echo Chambers, and Political Polarization
Barberá, P. (2020) · Social Media and Democracy · doi:10.1017/9781108890960.004 · Cited by 373
Mean Birds: Detecting Aggression and Bullying on Twitter
Chatzakou, D., Kourtellis, N., Blackburn, J., De Cristofaro, E., Stringhini, G., & Vakali, A. (2017) · Proceedings of the 2017 ACM on Web Science Conference · doi:10.1145/3091478.3091487 · Cited by 369
One-step and Two-step Classification for Abusive Language Detection on Twitter
Park, J. H., & Fung, P. (2017) · Proceedings of the First Workshop on Abusive Language Online · doi:10.18653/v1/w17-3006 · Cited by 353
Spread of Hate Speech in Online Social Media
Mathew, B., Dutt, R., Goyal, P., & Mukherjee, A. (2019) · Proceedings of the 10th ACM Conference on Web Science · doi:10.1145/3292522.3326034 · Cited by 348
Effective hate-speech detection in Twitter data using recurrent neural networks
Pitsilis, G. K., Ramampiaro, H., & Langseth, H. (2018) · Applied Intelligence · doi:10.1007/s10489-018-1242-y · Cited by 347
Hate speech detection: A solved problem? The challenging case of long tail on Twitter
Zhang, Z., & Luo, L. (2019) · Semantic Web · doi:10.3233/sw-180338 · Cited by 325
Community Interaction and Conflict on the Web
Kumar, S., Hamilton, W. L., Leskovec, J., & Jurafsky, D. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186141 · Cited by 314
Social Media and Democracy: The State of the Field, Prospects for Reform
(2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960 · Cited by 295
Hate Lingo: A Target-Based Linguistic Analysis of Hate Speech in Social Media
ElSherief, M., Kulkarni, V., Nguyen, D., Yang Wang, W., & Belding, E. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.15041 · Cited by 294
Developing an online hate classifier for multiple social media platforms
Salminen, J., Hopf, M., Chowdhury, S. A., Jung, S.-G., Almerekhi, H., & Jansen, B. J. (2020) · Human-centric Computing and Information Sciences · doi:10.1186/s13673-019-0205-6 · Cited by 286
Detecting Hate Speech in Social Media
Harvard Medical School, B., Malmasi, S., Zampieri, M., & University of Wolverhampton, W. (2017) · RANLP 2017 - Recent Advances in Natural Language Processing Meet Deep Learning · doi:10.26615/978-954-452-049-6_062 · Cited by 285
Misinformation, Disinformation, and Online Propaganda
Guess, A. M., & Lyons, B. A. (2020) · Social Media and Democracy · doi:10.1017/9781108890960.003 · Cited by 284
A systematic review of hate speech automatic detection using natural language processing
Jahan, M. S., & Oussalah, M. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126232 · Cited by 279
A survey of Twitter research: Data model, graph structure, sentiment analysis and attacks
Antonakaki, D., Fragopoulou, P., & Ioannidis, S. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114006 · Cited by 265
Challenges in discriminating profanity from hate speech
Malmasi, S., & Zampieri, M. (2018) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2017.1409284 · Cited by 264
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M. W., & Weber, I. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04009 · Cited by 263
Hate speech detection and racial bias mitigation in social media based on BERT model
Mozafari, M., Farahbakhsh, R., & Crespi, N. (2020) · PLOS ONE · doi:10.1371/journal.pone.0237861 · Cited by 260
“[33] proposed a neural network-based model advantaging pagraph2vec embeddings to distinguish between hate speech and clean content.”
BackgroundDetecting Online Hate Speech Using Context Aware Models
Texas A&M University, U., Gao, L., & Huang, R. (2017) · RANLP 2017 - Recent Advances in Natural Language Processing Meet Deep Learning · doi:10.26615/978-954-452-049-6_036 · Cited by 235
“Djuric et al. (2015) used the paragraph2vec for joint modeling of comments and words, then the generated embeddings were used as feature in a logistic regression model.”
MethodDetecting and Monitoring Hate Speech in Twitter
Pereira-Kohatsu, J. C., Quijano-Sánchez, L., Liberatore, F., & Camacho-Collados, M. (2019) · Sensors · doi:10.3390/s19214654 · Cited by 214
“[52] 2015 BOW, TF, TF-IDF, paragraph2vec embeddings LR 951,736 Yahoo Finance user comments No 0.”
BackgroundMethodJha, A., & Mamidi, R. (2017) · Proceedings of the Second Workshop on NLP and Computational Social Science · doi:10.18653/v1/w17-2902 · Cited by 169
“Djuric et al. (2015) try to address this problem in online user comments.”
BackgroundRaj, C., Agarwal, A., Bharathy, G., Narayan, B., & Prasad, M. (2021) · Electronics · doi:10.3390/electronics10222810 · Cited by 128
CyberBERT: BERT for cyberbullying identification
Paul, S., & Saha, S. (2022) · Multimedia Systems · doi:10.1007/s00530-020-00710-4 · Cited by 116
“[5] came up with paragraph-to-vector-based distributed representations of comments over Yahoo social media that can easily detect hate-speech achieving a sound accuracy of 80%.”
BackgroundBiCHAT: BiLSTM with deep CNN and hierarchical attention for hate speech detection
Khan, S., Fazil, M., Sejwal, V. K., Alshara, M. A., Alotaibi, R. M., Kamal, A., & Baig, A. R. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2022.05.006 · Cited by 115
Lingiardi, V., Carone, N., Semeraro, G., Musto, C., D’Amico, M., & Brena, S. (2020) · Behaviour & Information Technology · doi:10.1080/0144929x.2019.1607903 · Cited by 113
Understanding the Effect of Deplatforming on Social Networks
Ali, S., Saeed, M. H., Aldreabi, E., Blackburn, J., De Cristofaro, E., Zannettou, S., & Stringhini, G. (2021) · 13th ACM Web Science Conference 2021 · doi:10.1145/3447535.3462637 · Cited by 109
Offensive, aggressive, and hate speech analysis: From data-centric to human-centered approach
Kocoń, J., Figas, A., Gruza, M., Puchalska, D., Kajdanowicz, T., & Kazienko, P. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102643 · Cited by 103
Understanding the creation of trust in cryptocurrencies: the case of Bitcoin
Marella, V., Upreti, B., Merikivi, J., & Tuunainen, V. K. (2020) · Electronic Markets · doi:10.1007/s12525-019-00392-5 · Cited by 96
Detecting the Hate Code on Social Media
Magu, R., Joshi, K., & Luo, J. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14921 · Cited by 93
Cyber-aggression, Cyberbullying, and Cyber-grooming: A Survey and Research Challenges
Mladenović, M., Ošmjanski, V., & Stanković, S. V. (2022) · ACM Computing Surveys · doi:10.1145/3424246 · Cited by 79
“, Wikipedia) [17, 98, 112], Social news sites and websites of news media (The Guardian, Yahoo!Finance) [39, 75, 89, 101, 151], Forums (Incels.”
BackgroundRaman, N., Cao, M., Tsvetkov, Y., Kästner, C., & Vasilescu, B. (2020) · Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: New Ideas and Emerging Results · doi:10.1145/3377816.3381732 · Cited by 70
“For example, hate-speech detection specifically looks for strong, toxic interactions [15], trained on comments in online forums [8, 31].”
BackgroundAhmed, T., Ivan, S., Kabir, M., Mahmud, H., & Hasan, K. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00934-4 · Cited by 42
“The works of Djuric et al. (2015), Nobata et al. (2016) and Reynolds et al. (2011) focused on detecting cyberbullying in this way.”
BackgroundDesigning Toxic Content Classification for a Diversity of Perspectives
Kumar, D., Kelley, P. G., Consolvo, S., Mason, J., Bursztein, E., Durumeric, Z., Thomas, K., & Bailey, M. (2021) · arXiv · doi:10.48550/arxiv.2106.04511 · Cited by 28
Cyberbullying Detection using Recursive Neural Network through Offline Repository
Chandra, N., Khatri, S. K., & Som, S. (2018) · 2018 7th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO) · doi:10.1109/icrito.2018.8748570 · Cited by 9
Metadata
- Authors
- Djuric, N., Zhou, J., Morris, R., Grbovic, M., Radosavljevic, V., & Bhamidipati, N.
- Venue
- Proceedings of the 24th International Conference on World Wide Web
- Publisher
- ACM
- Published
- 2015-05-18
- Pages
- 29-30
- OpenAlex
- W1071251684
- Cited by
- 712
- References
- 7
- FWCI
- 25.95
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.