The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
The Risk of Racial Bias in Hate Speech Detection
Open accessAbstract
We investigate how annotators' insensitivity to differences in dialect can lead to racial bias in automatic hate speech detection models, potentially amplifying harm against minority populations.We first uncover unexpected correlations between surface markers of African American English (AAE) and ratings of toxicity in several widely-used hate speech datasets.Then, we show that models trained on these corpora acquire and propagate these biases, such that AAE tweets and tweets by self-identified African Americans are up to two times more likely to be labelled as offensive compared to others.Finally, we propose dialect and race priming as ways to reduce the racial bias in annotation, showing that when annotators are made explicitly aware of an AAE tweet's dialect they are significantly less likely to label the tweet as offensive.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Toxicity detection models misclassify African-American English tweets as negative more often than those written in Standard American English (Mozafari, Farahbakhsh, and Crespi 2020; Sap et al. 2019).”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Sap et al. 2019
The cited source says what the claim attributes to it.
In the citing paper
“For example, toxicity classifiers may disproportionately flag African-American English (Mozafari, Farahbakhsh, and Crespi 2020; Sap et al. 2019)”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Sap et al. 2019
In the citing paper
“models are trained with historical data and can present issues such as gender bias and racial bias in AI models [68,73], potentially exacerbating structural inequalities [10]”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [73]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A. J., & Hovy, E. H. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1174 · Cited by 5,022
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Attention-based LSTM for Aspect-level Sentiment Classification
Wang, Y., Huang, M., zhu, X., & Zhao, L. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1058 · Cited by 2,450
The Impact of Social Media on Children, Adolescents, and Families
O'Keeffe, G. S., Clarke-Pearson, K., & Council on Communications and Media (2011) · Pediatrics · doi:10.1542/peds.2011-0054 · Cited by 1,987 · cited 1×
“Toxic language (e.g., hate speech, abusive speech, or other offensive speech) primarily targets members of minority groups and can catalyze real-life violence towards them (O’Keeffe et al., 2011; Cleland, 2014; Mozur, 2018).”
BackgroundHateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Talat, Z., & Hovy, D. (2016) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n16-2013 · Cited by 1,919
Internal and external motivation to respond without prejudice.
Plant, E. A., & Devine, P. G. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.75.3.811 · Cited by 1,689
A Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A., & Wiegand, M. (2017) · SocialNLP@EACL · doi:10.18653/v1/w17-1101 · Cited by 1,478
Detecting Hate Speech on Twitter Using a Convolution-GRU Based Deep Neural Network
Zhang, Z., Robinson, D., & Tepper, J. A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93417-4_48 · Cited by 676
Research works printed but not linked: 2
- E. Ashby Plant and Patricia G. Devine. 1998. Inter- nal and external motivation to respond without prej- udice. J. Pers. Soc. Psychol., 75(3):811-832.
- Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alexander J. Smola, and Eduard H. Hovy. 2016. Hi- erarchical attention networks for document classifi- cation. In NAACL.
Cited by
Sorted by citations
Items 301–400 of 809
de Vries, I., & Radford, J. P. (2021) · The British Journal of Criminology · doi:10.1093/bjc/azab077 · Cited by 20
Han, X., & Tsvetkov, Y. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.374 · Cited by 25
Namespotting: Username toxicity and actual toxic behavior on Reddit
Urbaniak, R., Tempska, P., Dowgiałło, M., Ptaszynski, M., Fortuna, M., Marcinczuk, M., Piesiewicz, J., Leliwa, G., Soliwoda, K., Dziublewska, I., et al. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107371 · Cited by 14
Addressing religious hate online: from taxonomy creation to automated detection
Ramponi, A., Testa, B., Tonelli, S., & Ježek, E. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.1128 · Cited by 21
GraphBERT: Bridging Graph and Text for Malicious Behavior Detection on Social Media
Wu, J., Zhang, C., Liu, Z., Zhang, E., Wilson, S. L., & Zhang, C. (2022) · Proceedings · doi:10.1109/icdm54844.2022.00065 · Cited by 18
Das, D., Guha, S., & Semaan, B. (2023) · Workshop on Cross-Cultural Considerations in NLP (C3NLP) · doi:10.18653/v1/2023.c3nlp-1.8 · Cited by 16
From Toxicity in Online Comments to Incivility in American News: Proceed with Caution
Hede, A., Agarwal, O., Lu, L., Mutz, D. C., & Nenkova, A. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.225 · Cited by 16
Tekiroğlu, S. S., Bonaldi, H., Fanton, M., & Guerini, M. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.245 · Cited by 25
Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design
Bergman, A. S., Abercrombie, G., Spruit, S. L., Hovy, D., Dinan, E., Boureau, Y.-L., & Rieser, V. (2022) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2022.sigdial-1.4 · Cited by 16
Empirical Analysis of Multi-Task Learning for Reducing Model Bias in Toxic Comment DetectionPreprint
Vaidya, A., Mai, F., & Ning, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09758 · Cited by 16
Mitigating harm in language models with conditional-likelihood filtrationPreprint
Ngo, H., Raterink, C., Araújo, J. G. M., Zhang, I., Chen, C., Morisot, A., & Frosst, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.07790 · Cited by 60
Hartmann, D., Oueslati, A., Staufer, D., Pohlmann, L., Munzert, S., & Heuer, H. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713998 · Cited by 29
Bias and comparison framework for abusive language datasets
Wich, M., Eder, T., Kuwatly, H. A., & Groh, G. (2021) · AI and Ethics · doi:10.1007/s43681-021-00081-0 · Cited by 16
Hate Speech Criteria: A Modular Approach to Task-Specific Hate Speech Definitions
Khurana, U., Vermeulen, I., Nalisnick, E., van Noorloos, M., & Fokkens, A. S. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.17 · Cited by 15
A comparative study on the impact of model compression techniques on fairness in language models
Ramesh, K., Chavan, A., Pandit, S., & Sitaram, S. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.878 · Cited by 15
Detecting Unintended Social Bias in Toxic Language Datasets
Sahoo, N., Gupta, H., & Bhattacharyya, P. (2022) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2022.conll-1.10 · Cited by 25
Quality aspects of annotated data
Beck, J. (2023) · AStA Wirtschafts- und Sozialstatistisches Archiv · doi:10.1007/s11943-023-00332-y · Cited by 20
Das, D., Guha, S., Brubaker, J. R., & Semaan, B. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642669 · Cited by 27
A comprehensive review on detection of hate speech for multi-lingual data
Narula, R., & Chaudhary, P. (2025) · Social Network Analysis and Mining · doi:10.1007/s13278-024-01401-y · Cited by 23
Tracking Hate in Social Media: Evaluation, Challenges and Approaches
Modha, S., Mandl, T., Majumder, P., & Patel, D. D. (2020) · SN Computer Science · doi:10.1007/s42979-020-0082-0 · Cited by 18
SalamNET at SemEval-2020 Task 12: Deep Learning Approach for Arabic Offensive Language Detection
Husain, F., Lee, J., Henry, S., & Uzuner, Ö. (2020) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2020.semeval-1.283 · Cited by 17
Surfacing Racial Stereotypes through Identity Portrayal
Kambhatla, G., Stewart, I., & Mihalcea, R. F. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533217 · Cited by 15
Phutane, M., Seelam, A., & Vashistha, A. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732128 · Cited by 14
Choenni, R., Shutova, E., & van Rooij, R. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.111 · Cited by 20
Ethical Implications of Generative AI in Art and the Media
Vyas, B. (2022) · International Journal For Multidisciplinary Research · doi:10.36948/ijfmr.2022.v04i04.9392 · Cited by 16
LOGAN: Local Group Bias Detection by Clustering
Zhao, J., & Chang, K.-W. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.155 · Cited by 15
Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hate
Botelho, A., Hale, S. A., & Vidgen, B. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.166 · Cited by 14
A LEGAL FRAMEWORK FOR ARTIFICIAL INTELLIGENCE FAIRNESS REPORTING
Yap, J. Q., & Lim, E. W. K. (2022) · The Cambridge Law Journal · doi:10.1017/s0008197322000460 · Cited by 20
Kurpicz-Briki, M. (2020) · ARBOR - Bern University of Applied Sciences Repository · doi:10.24451/arbor.11922 · Cited by 14
Hardage, D., & Najafirad, P. (2020) · Workshop on NLP for COVID-19 (NLP-COVID19) · doi:10.18653/v1/2020.nlpcovid19-2.36 · Cited by 14
FairPrism: Evaluating Fairness-Related Harms in Text Generation
Fleisig, E., Amstutz, A., Atalla, C., Blodgett, S. L., Daumé III, H., Olteanu, A., Sheng, E., Vann, D., & Wallach, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.343 · Cited by 14
“…were trained on text written by humans and have issues identifying some types of fairness-related harms, such as ignoring implicit hate speech (ElSherief et al., 2021) and mislabeling African-American English as hate speech (Mozafari et al., 2020; Sap et al., 2019; Davidson et al., 2019).”
BackgroundWich, M., Gorniak, A., Eder, T., Bartmann, D., Çakici, B. E., & Groh, G. (2022) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v16i1.19364 · Cited by 19
Detection of Homophobia & Transphobia in Malayalam and Tamil: Exploring Deep Learning Methods
Sharma, D., Gupta, V., & Singh, V. K. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-28183-9_15 · Cited by 13
Ding, X., Horning, M., & Rho, E. H. R. (2023) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v17i1.22135 · Cited by 16
Spinde, T., Richter, E., Wessel, M. P., Kulshrestha, J., & Donnay, K. (2023) · Online Social Networks and Media · doi:10.1016/j.osnem.2023.100264 · Cited by 18
Navigating the Metaverse: A New Virtual Tool with Promising Real Benefits for Breast Cancer Patients
Żydowicz, W. M., Skokowski, J., Marano, L., & Połom, K. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13154337 · Cited by 14
Smith, C. E., Klassen, S., Prinster, G. H., Tan, C., & Keegan, B. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686920 · Cited by 16
Exploiting GPT for synthetic data generation: An empirical study
Busker, T., Choenni, S., & Bargh, M. S. (2024) · Government Information Quarterly · doi:10.1016/j.giq.2024.101988 · Cited by 14
Arthurs, N., & Alvero, A. (2020) · Educational Data Mining · Cited by 13
Disentangling Representations of Text by Masking Transformers
Zhang, X., van de Meent, J.-W., & Wallace, B. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.60 · Cited by 20
AAA: Fair Evaluation for Abuse Detection Systems Wanted
Calabrese, A., Bevilacqua, M., Roß, B., Tripodi, R., & Roberto Navigli, A. (2021) · ACM Web Science Conference (WebSci) · doi:10.1145/3447535.3462484 · Cited by 13
Gender Bias in Machine Translation Systems
Ullmann, S. (2022) · Artificial Intelligence and Its Discontents · doi:10.1007/978-3-030-88615-8_7 · Cited by 7
Representational ethical model calibration
Carruthers, R., Straw, I., Ruffle, J. K., Herron, D., Nelson, A., Bzdok, D., Fernández-Reyes, D., Rees, G. E., & Nachev, P. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00716-4 · Cited by 15
COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements
Zhou, X., Zhu, H., Yerukola, A., Davidson, T., Hwang, J. D., Swayamdipta, S., & Sap, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.392 · Cited by 21
Self-citationParameter-efficient Modularised Bias Mitigation via AdapterFusion
Kumar, D., Lesota, O., Zerveas, G., Cohen, D., Eickhoff, C., Schedl, M., & Rekabsaz, N. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.201 · Cited by 13
Joint Modelling of Emotion and Abusive Language Detection
Rajamanickam, S., Mishra, P., Yannakoudakis, H., & Shutova, E. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.394 · Cited by 26
DeToxy: A Large-Scale Multimodal Dataset for Toxicity Classification in Spoken Utterances
Ghosh, S., Lepcha, S., Sakshi, S., Shah, R. R., & Umesh, S. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10752 · Cited by 12
AI Within Online Discussions: Rational, Civil, Privileged?
Carstens, J. A., & Frieß, D. (2024) · Minds and Machines · doi:10.1007/s11023-024-09658-0 · Cited by 22
Yi, F., Singh, A., & Tao, Z. (2024) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000101 · Cited by 13
Paul, J., Mallick, S., Mitra, A., Roy, A., & Sil, J. (2025) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3713077 · Cited by 13
Ngo, A., & Kocoń, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102966 · Cited by 14
Analyzing Social Media Research: A Data Quality and Research Reproducibility Perspective
Srivastava, A. K., & Mishra, R. (2021) · IIM Kozhikode Society & Management Review · doi:10.1177/22779752211011810 · Cited by 13
Anti-Islamic Arabic Text Categorization using Text Mining and Sentiment Analysis Techniques
Alraddadi, R. A., & Ghembaza, M. I. E. (2021) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2021.0120889 · Cited by 12
Could a Conversational AI Identify Offensive Language?
da Silva, D. A., Louro, H. D. B., Gonçalves, G. S., Marques, J., Dias, L. A. V., da Cunha, A. M., & Tasinaffo, P. M. (2021) · Information · doi:10.3390/info12100418 · Cited by 12
Merging public health and automated approaches to address online hate speech
Nguyen, T. (2023) · AI and Ethics · doi:10.1007/s43681-023-00281-w · Cited by 12
On the independence of association bias and empirical fairness in language models
Cabello, L., Jørgensen, A. K., & Søgaard, A. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594004 · Cited by 14
Incorporating Stock Market Signals for Twitter Stance Detection
Conforti, C., Berndt, J., Pilehvar, M. T., Giannitsarou, C., Toxvaerd, F., & Collier, N. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.281 · Cited by 13
Efe, A. (2022) · Yeni Medya Dergisi · doi:10.55609/yenimedya.1146586 · Cited by 12
Examining Racial Bias in an Online Abuse Corpus with Structural Topic ModelingPreprint
Davidson, T., & Bhattacharya, D. (2020) · arXiv.org · doi:10.48550/arxiv.2005.13041 · Cited by 17
Customizing Triggers with Concealed Data Poisoning.Preprint
Wallace, E., Zhao, T. Z., Feng, S., & Singh, S. (2020) · arXiv (Cornell University) · Cited by 12
Leveraging Digital Intelligence for Community Well-Being
Narayan, R. (2020) · International Journal of Community Well-Being · doi:10.1007/s42413-020-00085-4 · Cited by 14
Chilling Tales: Understanding the Impact of Copyright Takedowns on Transformative Content Creators
Fiesler, C. L., Paup, J., & Zacher, C. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610095 · Cited by 13
Comparing Biases and the Impact of Multilingual Training across Multiple Languages
Levy, S., John, N. A., Liu, L., Vyas, Y., Ma, J., Fujinuma, Y., Ballesteros, M., Castelli, V., & Roth, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.634 · Cited by 18
Who's in and who's out? A case study of multimodal CLIP-filtering in DataComp
Hong, R., Agnew, W., Kohno, T., & Morgenstern, J. (2024) · Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3689904.3694702 · Cited by 11
Addressing Ethical Considerations and Responsible AI Practices
Luthra, A., Dixit, S., Garg, S., Singh, A., & Anchal, M. (2025) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-83549-852-120251007 · Cited by 14
Artificial intelligence and the rights to assembly and association
Ashraf, C. (2020) · Journal of Cyber Policy · doi:10.1080/23738871.2020.1778760 · Cited by 12
Abusive Language Detection using Syntactic Dependency Graphs
Narang, K., & Brew, C. (2020) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2020.alw-1.6 · Cited by 11
Mitigation of Diachronic Bias in Fake News Detection Dataset
Murayama, T., Wakamiya, S., & Aramaki, E. (2021) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/2021.wnut-1.21 · Cited by 15
Syntax and prejudice: ethically-charged biases of a syntax-based hate speech recognizer unveiled
Mastromattei, M., Ranaldi, L., Fallucchi, F., & Zanzotto, F. M. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.859 · Cited by 11
Accounting for Offensive Speech as a Practice of Resistance
Díaz, M., Amironesei, R., Weidinger, L., & Gabriel, I. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.18 · Cited by 16
Fine-Grained Fairness Analysis of Abusive Language Detection Systems with CheckList
Manerba, M. M., & Tonelli, S. (2021) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2021.woah-1.9 · Cited by 12
Insights Into Incitement: A Computational Perspective on Dangerous Speech on Twitter in India
Dash, S., Grover, R., Shekhawat, G., Kaur, S., Mishra, D., & Pal, J. (2022) · ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies (COMPASS) · doi:10.1145/3530190.3534800 · Cited by 11
Detoxifying Language Models Risks Marginalizing Minority VoicesPreprint
Xu, A., Pathak, E., Wallace, E., Gururangan, S., Sap, M., & Klein, D. (2021) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2104.06390 · Cited by 155
Self-citationWei, K., Lin, Y.-R., & Yan, M. (2020) · Proceedings of The Web Conference 2020 · doi:10.1145/3366423.3380307 · Cited by 11
Who are the haters? A corpus-based demographic analysis of authors of hate speech
Hilte, L., Markov, I., Ljubešić, N., Fišer, D., & Daelemans, W. M. P. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.986890 · Cited by 10
Moderation, Networks, and Anti-Social Behavior Online
Haythornthwaite, C. (2023) · Social Media + Society · doi:10.1177/20563051231196874 · Cited by 18
Contrastive adversarial gender debiasing
Torres, N. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100092 · Cited by 11
Cross-lingual hate speech detection using domain-specific word embeddings
Arango, A., Rojas, J. P., & Labra, B. P. (2024) · PLoS ONE · doi:10.1371/journal.pone.0306521 · Cited by 30
Towards an Epistemic Compass for Online Content Moderation
Tobi, A. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00791-3 · Cited by 15
Evaluating Synthetic Data Generation from User Generated Text
Chim, J., Ive, J., & Liakata, M. (2024) · Computational Linguistics · doi:10.1162/coli_a_00540 · Cited by 12
A comprehensive analysis of gender, racial, and prompt-induced biases in large language models
Torres, N., Ulloa, C., Araya, I., Ayala, M., & Jara, S. (2024) · International Journal of Data Science and Analytics · doi:10.1007/s41060-024-00696-6 · Cited by 14
Gender bias detection on hate speech classification: an analysis at feature-level
Nascimento, F. R. S., Cavalcanti, G. D. C., & Costa‐Abreu, M. D. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10841-8 · Cited by 13
"Ignorance is not Bliss": Designing Personalized Moderation to Address Ableist Hate on Social Media
Heung, S., Jiang, L., Azenkot, S., & Vashistha, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713997 · Cited by 10
The Value of Disagreement in AI Design, Evaluation, and Alignment
Fazelpour, S., & Fleisher, W. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732146 · Cited by 10
Understanding User Sensemaking in Machine Learning Fairness Assessment Systems
Gu, Z., Yan, J. N., & Rzeszotarski, J. M. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450092 · Cited by 15
Towards Equal Gender Representation in the Annotations of Toxic Language Detection
Excell, E., & Moubayed, N. A. (2021) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2021.gebnlp-1.7 · Cited by 13
How Colonial Continuities Underlie Algorithmic Injustices in Education
Karumbaiah, S., & Brooks, J. (2021) · Research on Equity and Sustained Participation in Engineering, Computing, and Technology (RESPECT) · doi:10.1109/respect51740.2021.9620605 · Cited by 10
DataLab: A Platform for Data Analysis and Intervention
Yang, X., Fu, J., Yuan, W., Viswanathan, V., Liu, Z., Liu, Y., Neubig, G., & Liu, P. (2022) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2022.acl-demo.18 · Cited by 10
Listening to Affected Communities to Define Extreme Speech: Dataset and Experiments
Maronikolakis, A., Wisiorek, A., Nann, L., Jabbar, H., Udupa, S., & Schuetze, H. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.87 · Cited by 10
When More Data Lead Us Astray: Active Data Acquisition in the Presence of Label Bias
Li, Y., De‐Arteaga, M., & Saar‐Tsechansky, M. (2022) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v10i1.21994 · Cited by 12
Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworks
Hauzenberger, L., Masoudian, S., Kumar, D., Schedl, M., & Rekabsaz, N. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.386 · Cited by 16
Human-in-the-loop Evaluation for Early Misinformation Detection: A Case Study of COVID-19 Treatments
Mendes, E., Chen, Y., Xu, W. H., & Ritter, A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.881 · Cited by 17
Online harassment of Japanese celebrities and influencers
Takano, M., Taka, F., Ogiue, C., & Nagata, N. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1386146 · Cited by 10
Exposing the many biases in machine learning
Richardson, S. (2022) · Business Information Review · doi:10.1177/02663821221121024 · Cited by 10
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-ExpertsPreprint
Liu, A., Sap, M., Lu, X., Swayamdipta, S., Bhagavatula, C., Smith, N. A., & Choi, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.03023 · Cited by 10
Self-citationCountering Online Hate Speech: An NLP PerspectivePreprint
Chaudhary, M., Saxena, C., & Meng, H. M. L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.02941 · Cited by 31
HarmonyNet: Navigating hate speech detection
Raza, S., & Chatrath, V. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100098 · Cited by 11
Exploring hate speech detection: challenges, resources, current research and future directions
Geetanjali, G., & Kumar, M. (2025) · Multimedia Tools and Applications · doi:10.1007/s11042-025-20716-2 · Cited by 12
Chown, E., & Nascimento, F. (2023) · Lever Press eBooks · doi:10.3998/mpub.12668201 · Cited by 9
Metadata
- Authors
- Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A.
- Affiliations
- University of Washington (US)
- Carnegie Mellon University (US)
- Allen Institute for Artificial Intelligence (US)
- Corresponding author
- Sap, Maarten
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 1668-1678
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2949678053
- Cited by
- 817
- References
- 40
- Citations by year
- 2026: 59 · 2025: 96 · 2024: 86 · 2023: 139 · 2022: 142 · 2021: 178 · 2020: 105 · 2019: 12
- FWCI
- 40.99
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- National Science Foundation (1714566)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- hate speech detection, racial bias, African American English, dialects, annotation bias, hate speech datasets, priming, minority groups
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
