The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Cyber Hate Speech on Twitter: An Application of Machine Classification and Statistical Modeling for Policy and Decision Making
Open accessAbstract
The use of “Big Data” in policy and decision making is a current topic of debate. The 2013 murder of Drummer Lee Rigby in Woolwich, London, UK led to an extensive public reaction on social media, providing the opportunity to study the spread of online hate speech (cyber hate) on Twitter. Human annotated Twitter data was collected in the immediate aftermath of Rigby's murder to train and test a supervised machine learning text classifier that distinguishes between hateful and/or antagonistic responses with a focus on race, ethnicity, or religion; and more general responses. Classification features were derived from the content of each tweet, including grammatical dependencies between words to recognize “othering” phrases, incitement to respond with antagonistic action, and claims of well‐founded or justified discrimination against social groups. The results of the classifier were optimal using a combination of probabilistic, rule‐based, and spatial‐based classifiers with a voted ensemble meta‐classifier. We demonstrate how the results of the classifier can be robustly utilized in a statistical model used to forecast the likely spread of cyber hate in a sample of Twitter data. The applications to policy and decision making are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Generating Typed Dependency Parses from Phrase Structure Parses
de Marneffe, M.-C., MacCartney, B., & Manning, C. D. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3wtkaidk6ea8 · Cited by 2,258
Annotating Expressions of Opinions and Emotions in Language
Wiebe, J., Wilson, T., & Cardie, C. (2005) · Language Resources and Evaluation · doi:10.1007/s10579-005-7880-9 · Cited by 1,755
Sentiment strength detection in short informal text
Thelwall, M., Buckley, K., Paltoglou, G., Cai, D., & Kappas, A. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21416 · Cited by 1,449
Thelwall, M., Buckley, K., & Paltoglou, G. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21462 · Cited by 822
Some consequences of de-individuation in a group.
Festinger, L., Pepitone, A., & Newcomb, T. (1952) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0057906 · Cited by 718
Up and Down With Ecology: The "Issue-Attention Cycle"
(2016) · Agenda Setting · doi:10.4324/9781315538389-9 · Cited by 648
Detecting Offensive Language in Social Media to Protect Adolescent Online Safety
Chen, Y., Zhou, Y., Zhu, S., & Xu, H. (2012) · 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing · doi:10.1109/socialcom-passat.2012.55 · Cited by 637
Common Sense Reasoning for Detection, Prevention, and Mitigation of Cyberbullying
Dinakar, K., Jones, B., Havasi, C., Lieberman, H., & Picard, R. (2012) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2362394.2362400 · Cited by 432
Terrorist Events and Attitudes toward Immigrants: A Natural Experiment
Legewie, J. (2013) · American Journal of Sociology · doi:10.1086/669605 · Cited by 322
Cited by
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
A Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A., & Wiegand, M. (2017) · Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media · doi:10.18653/v1/w17-1101 · Cited by 1,395
A Survey on Automatic Detection of Hate Speech in Text
Fortuna, P., & Nunes, S. (2019) · ACM Computing Surveys · doi:10.1145/3232676 · Cited by 1,241
The Risk of Racial Bias in Hate Speech Detection
Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1163 · Cited by 817
Predicting the Type and Target of Offensive Posts in Social Media
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., & Kumar, R. (2019) · Proceedings of the 2019 Conference of the North · doi:10.18653/v1/n19-1144 · Cited by 720
SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., & Kumar, R. (2019) · Proceedings of the 13th International Workshop on Semantic Evaluation · doi:10.18653/v1/s19-2010 · Cited by 701
Detecting Hate Speech on Twitter Using a Convolution-GRU Based Deep Neural Network
Zhang, Z., Robinson, D., & Tepper, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-93417-4_48 · Cited by 638
Understanding Abuse: A Typology of Abusive Language Detection Subtasks
Waseem, Z., Davidson, T., Warmsley, D., & Weber, I. (2017) · Proceedings of the First Workshop on Abusive Language Online · doi:10.18653/v1/w17-3012 · Cited by 474
Watanabe, H., Bouazizi, M., & Ohtsuki, T. (2018) · IEEE Access · doi:10.1109/access.2018.2806394 · Cited by 467
Hate Speech Dataset from a White Supremacy Forum
de Gibert, O., Perez, N., García-Pablos, A., & Cuadros, M. (2018) · Proceedings of the 2nd Workshop on Abusive Language Online (ALW2) · doi:10.18653/v1/w18-5102 · Cited by 425
Williams, M. L., Burnap, P., & Sloan, L. (2017) · Sociology · doi:10.1177/0038038517708140 · Cited by 423
Self-citationRacial Bias in Hate Speech and Abusive Language Detection Datasets
Davidson, T., Bhattacharya, D., & Weber, I. (2019) · Proceedings of the Third Workshop on Abusive Language Online · doi:10.18653/v1/w19-3504 · Cited by 414
A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media
Mozafari, M., Farahbakhsh, R., & Crespi, N. (2020) · Studies in Computational Intelligence · doi:10.1007/978-3-030-36687-2_77 · Cited by 396
Us and them: identifying cyber hate on Twitter across multiple protected characteristics
Burnap, P., & Williams, M. L. (2016) · EPJ Data Science · doi:10.1140/epjds/s13688-016-0072-6 · Cited by 396
Self-citationHate speech review in the context of online social networks
Chetty, N., & Alathur, S. (2018) · Aggression and Violent Behavior · doi:10.1016/j.avb.2018.05.003 · Cited by 357
Deep learning for affective computing: Text-based emotion recognition in decision support
Kratzwald, B., Ilić, S., Kraus, M., Feuerriegel, S., & Prendinger, H. (2018) · Decision Support Systems · doi:10.1016/j.dss.2018.09.002 · Cited by 352
Camacho, D., Panizo-LLedot, Á., Bello-Orgaz, G., Gonzalez-Pardo, A., & Cambria, E. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.05.009 · Cited by 329
Hate speech detection: A solved problem? The challenging case of long tail on Twitter
Zhang, Z., & Luo, L. (2019) · Semantic Web · doi:10.3233/sw-180338 · Cited by 325
Rossini, P. (2022) · Communication Research · doi:10.1177/0093650220921314 · Cited by 312
Hate Lingo: A Target-Based Linguistic Analysis of Hate Speech in Social Media
ElSherief, M., Kulkarni, V., Nguyen, D., Yang Wang, W., & Belding, E. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.15041 · Cited by 294
Detecting Hate Speech in Social Media
Harvard Medical School, B., Malmasi, S., Zampieri, M., & University of Wolverhampton, W. (2017) · RANLP 2017 - Recent Advances in Natural Language Processing Meet Deep Learning · doi:10.26615/978-954-452-049-6_062 · Cited by 285
A survey of Twitter research: Data model, graph structure, sentiment analysis and attacks
Antonakaki, D., Fragopoulou, P., & Ioannidis, S. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114006 · Cited by 265
Challenges in discriminating profanity from hate speech
Malmasi, S., & Zampieri, M. (2018) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2017.1409284 · Cited by 264
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M. W., & Weber, I. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04009 · Cited by 263
Hate speech detection and racial bias mitigation in social media based on BERT model
Mozafari, M., Farahbakhsh, R., & Crespi, N. (2020) · PLOS ONE · doi:10.1371/journal.pone.0237861 · Cited by 260
Bias in Large Language Models: Origin, Evaluation, and Mitigation
Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., & Liu, S. S. (2024) · arXiv · doi:10.48550/arxiv.2411.10915 · Cited by 208
Gongane, V. U., Munot, M. V., & Anuse, A. D. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00951-3 · Cited by 153
BiCHAT: BiLSTM with deep CNN and hierarchical attention for hate speech detection
Khan, S., Fazil, M., Sejwal, V. K., Alshara, M. A., Alotaibi, R. M., Kamal, A., & Baig, A. R. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2022.05.006 · Cited by 115
“Burnap and Williams (2015) used various n-gram (1–5) features and trained three machine learning models to classify hate speech from Twitter and concluded that voted ensemble classifier shows the best performance of 0 : 89 ; 0 : 69, and 0 : 77 considering accuracy, recall, and f-score, respectively.”
Lingiardi, V., Carone, N., Semeraro, G., Musto, C., D’Amico, M., & Brena, S. (2020) · Behaviour & Information Technology · doi:10.1080/0144929x.2019.1607903 · Cited by 113
Understanding the Effect of Deplatforming on Social Networks
Ali, S., Saeed, M. H., Aldreabi, E., Blackburn, J., De Cristofaro, E., Zannettou, S., & Stringhini, G. (2021) · 13th ACM Web Science Conference 2021 · doi:10.1145/3447535.3462637 · Cited by 109
Detecting the Hate Code on Social Media
Magu, R., Joshi, K., & Luo, J. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14921 · Cited by 93
Akhter, M. P., Jiangbin, Z., Naqvi, I. R., AbdelMajeed, M., & Zia, T. (2022) · Multimedia Systems · doi:10.1007/s00530-021-00784-8 · Cited by 84
Towards Cyberbullying-free social media in smart cities: a unified multi-modal approach
Kumari, K., Singh, J. P., Dwivedi, Y. K., & Rana, N. P. (2020) · Soft Computing · doi:10.1007/s00500-019-04550-x · Cited by 81
Cyber-aggression, Cyberbullying, and Cyber-grooming: A Survey and Research Challenges
Mladenović, M., Ošmjanski, V., & Stanković, S. V. (2022) · ACM Computing Surveys · doi:10.1145/3424246 · Cited by 79
Calderón, C. A., de la Vega, G., & Herrero, D. B. (2020) · Social Sciences · doi:10.3390/socsci9110188 · Cited by 73
Hate speech detection: A comprehensive review of recent works
Gandhi, A., Ahir, P., Adhvaryu, K., Shah, P., Lohiya, R., Cambria, E., Poria, S., & Hussain, A. (2024) · Expert Systems · doi:10.1111/exsy.13562 · Cited by 66
Digital Wildfires: Propagation, Verification, Regulation, and Responsible Innovation
Webb, H., Burnap, P., Procter, R., Rana, O., Stahl, B. C., Williams, M., Housley, W., Edwards, A., & Jirotka, M. (2016) · ACM Transactions on Information Systems · doi:10.1145/2893478 · Cited by 64
Self-citationOzalp, S., Williams, M. L., Burnap, P., Liu, H., & Mostafa, M. (2020) · Social Media + Society · doi:10.1177/2056305120916850 · Cited by 57
Self-citationEarly detection of deception and aggressiveness using profile-based representations
Escalante, H. J., Villatoro-Tello, E., Garza, S. E., López-Monroy, A. P., Montes-y-Gómez, M., & Villaseñor-Pineda, L. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.07.040 · Cited by 44
Ahmed, T., Ivan, S., Kabir, M., Mahmud, H., & Hasan, K. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00934-4 · Cited by 42
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey
Kumar, S., Balachandran, V., Njoo, L., Anastasopoulos, A., & Tsvetkov, Y. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.241 · Cited by 38
Blanco-Alfonso, I., Rodríguez-Fernández, L., & Arce-García, S. (2022) · Revista de Comunicación · doi:10.26441/rc21.2-2022-a2 · Cited by 38
Accountability Issues, Online Covert Hate Speech, and the Efficacy of Counter‐Speech
Baider, F. (2023) · Politics and Governance · doi:10.17645/pag.v11i2.6465 · Cited by 33
Liu, H. K., Tang, M., & Collard, A. S. J. (2025) · Government Information Quarterly · doi:10.1016/j.giq.2024.102006 · Cited by 17
Swedish journalists' perceptions of legal protection against unlawful online harassment
Björkenfeldt, O. (2023) · Frontiers in Sociology · doi:10.3389/fsoc.2023.1154495 · Cited by 6
Metadata
- Authors
- Burnap, P., & Williams, M. L.
- Venue
- Policy & Internet
- Publisher
- Wiley
- Published
- 2015-04-22
- Volume
- 7
- Issue
- 2
- Pages
- 223-242
- ISSN
- 1944-2866 (ISSN-L)
- OpenAlex
- W1871142974
- Cited by
- 616
- References
- 29
- FWCI
- 33.16
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.