The DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index.
Resources and benchmark corpora for hate speech detection: a systematic review
Open accessAbstract
Hate Speech in social media is a complex phenomenon, whose detection has recently gained significant traction in the Natural Language Processing community, as attested by several recent review works. Annotated corpora and benchmarks are key resources, considering the vast number of supervised approaches that have been proposed. Lexica play an important role as well for the development of hate speech detection systems. In this review, we systematically analyze the resources made available by the community at large, including their development methodology, topical focus, language coverage, and other factors. The results of our analysis highlight a heterogeneous, growing landscape, marked by several issues and venues for improvement.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Racial microaggressions in everyday life: Implications for clinical practice.
Sue, D. W., Capodilupo, C. M., Torino, G. C., Bucceri, J. M., Holder, A. M. B., Nadal, K. L., & Esquilin, M. (2007) · American Psychologist · doi:10.1037/0003-066x.62.4.271 · Cited by 5,569
(2004) · Cited by 5,198
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
Emotion: A Psychoevolutionary Synthesis
Camras, L., & Plutchik, R. (1980) · The American Journal of Psychology · doi:10.2307/1422394 · Cited by 2,487
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Waseem, Z., & Hovy, D. (2016) · Proceedings of the NAACL Student Research Workshop · doi:10.18653/v1/n16-2013 · Cited by 1,655
A Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A., & Wiegand, M. (2017) · Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media · doi:10.18653/v1/w17-1101 · Cited by 1,395
A Survey on Automatic Detection of Hate Speech in Text
Fortuna, P., & Nunes, S. (2019) · ACM Computing Surveys · doi:10.1145/3232676 · Cited by 1,241
The WaCky wide web: a collection of very large linguistically processed web-crawled corpora
Baroni, M., Bernardini, S., Ferraresi, A., & Zanchetta, E. (2009) · Language Resources and Evaluation · doi:10.1007/s10579-009-9081-4 · Cited by 1,200
Abusive Language Detection in Online User Content
Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., & Chang, Y. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883062 · Cited by 1,122
SemEval-2019 Task 5: Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter
Basile, V., Bosco, C., Fersini, E., Nozza, D., Patti, V., Rangel Pardo, F. M., Rosso, P., & Sanguinetti, M. (2019) · Proceedings of the 13th International Workshop on Semantic Evaluation · doi:10.18653/v1/s19-2007 · Cited by 864
Self-citation
Cited by
HateBERT: Retraining BERT for Abusive Language Detection in English
Caselli, T., Basile, V., Mitrović, J., & Granitzer, M. (2021) · Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021) · doi:10.18653/v1/2021.woah-1.3 · Cited by 307
Self-citationA systematic review of hate speech automatic detection using natural language processing
Jahan, M. S., & Oussalah, M. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126232 · Cited by 279
Dynabench: Rethinking Benchmarking in NLP
Kiela, D., Bartolo, M., Nie, Y., Kaushik, D., Geiger, A., Wu, Z., Vidgen, B., Prasad, G., Singh, A., Ringshia, P., et al. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.324 · Cited by 254
Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection
Vidgen, B., Thrush, T., Waseem, Z., & Kiela, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.132 · Cited by 174
Offensive, aggressive, and hate speech analysis: From data-centric to human-centered approach
Kocoń, J., Figas, A., Gruza, M., Puchalska, D., Kajdanowicz, T., & Kazienko, P. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102643 · Cited by 103
Metadata
- Authors
- Poletto, F., Basile, V., Sanguinetti, M., Bosco, C., & Patti, V.
- Venue
- Language Resources and Evaluation
- Published
- 2020-09-30
- Volume
- 55
- Issue
- 2
- Pages
- 477-523
- ISSN
- 1574-020X (ISSN-L)
- 1574-0218
- OpenAlex
- W3091315987
- Cited by
- 385
- References
- 94
- FWCI
- 7.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Resources and benchmark corpora for hate speech detection: a systematic review. https://app.cochat.ai/papers/10.1007/s10579-020-09502-8
- The paper
- Poletto, F., Basile, V., Sanguinetti, M., Bosco, C., & Patti, V. (2021). Resources and benchmark corpora for hate speech detection: a systematic review. Language Resources and Evaluation. https://doi.org/10.1007/s10579-020-09502-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.