The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
The Risk of Racial Bias in Hate Speech Detection
Open accessAbstract
We investigate how annotators' insensitivity to differences in dialect can lead to racial bias in automatic hate speech detection models, potentially amplifying harm against minority populations.We first uncover unexpected correlations between surface markers of African American English (AAE) and ratings of toxicity in several widely-used hate speech datasets.Then, we show that models trained on these corpora acquire and propagate these biases, such that AAE tweets and tweets by self-identified African Americans are up to two times more likely to be labelled as offensive compared to others.Finally, we propose dialect and race priming as ways to reduce the racial bias in annotation, showing that when annotators are made explicitly aware of an AAE tweet's dialect they are significantly less likely to label the tweet as offensive.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Toxicity detection models misclassify African-American English tweets as negative more often than those written in Standard American English (Mozafari, Farahbakhsh, and Crespi 2020; Sap et al. 2019).”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Sap et al. 2019
The cited source says what the claim attributes to it.
In the citing paper
“For example, toxicity classifiers may disproportionately flag African-American English (Mozafari, Farahbakhsh, and Crespi 2020; Sap et al. 2019)”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Sap et al. 2019
In the citing paper
“models are trained with historical data and can present issues such as gender bias and racial bias in AI models [68,73], potentially exacerbating structural inequalities [10]”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [73]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A. J., & Hovy, E. H. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1174 · Cited by 5,022
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Attention-based LSTM for Aspect-level Sentiment Classification
Wang, Y., Huang, M., zhu, X., & Zhao, L. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1058 · Cited by 2,450
The Impact of Social Media on Children, Adolescents, and Families
O'Keeffe, G. S., Clarke-Pearson, K., & Council on Communications and Media (2011) · Pediatrics · doi:10.1542/peds.2011-0054 · Cited by 1,987 · cited 1×
“Toxic language (e.g., hate speech, abusive speech, or other offensive speech) primarily targets members of minority groups and can catalyze real-life violence towards them (O’Keeffe et al., 2011; Cleland, 2014; Mozur, 2018).”
BackgroundHateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Talat, Z., & Hovy, D. (2016) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n16-2013 · Cited by 1,919
Internal and external motivation to respond without prejudice.
Plant, E. A., & Devine, P. G. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.75.3.811 · Cited by 1,689
A Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A., & Wiegand, M. (2017) · SocialNLP@EACL · doi:10.18653/v1/w17-1101 · Cited by 1,478
Detecting Hate Speech on Twitter Using a Convolution-GRU Based Deep Neural Network
Zhang, Z., Robinson, D., & Tepper, J. A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93417-4_48 · Cited by 676
Research works printed but not linked: 2
- E. Ashby Plant and Patricia G. Devine. 1998. Inter- nal and external motivation to respond without prej- udice. J. Pers. Soc. Psychol., 75(3):811-832.
- Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alexander J. Smola, and Eduard H. Hovy. 2016. Hi- erarchical attention networks for document classifi- cation. In NAACL.
Cited by
Sorted by citations
Items 401–500 of 808
Dialect Diversity in Text Summarization on Twitter
Keswani, V., & Celis, L. E. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450108 · Cited by 13
Use of Formal Ethical Reviews in NLP Literature: Historical Trends and Current Practices
Santy, S., Rani, A., & Choudhury, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.414 · Cited by 9
Toward Annotator Group Bias in Crowdsourcing
Liu, H., Thekinen, J., Mollaoglu, S., Tang, D. Q., Yang, J. S., Cheng, Y., Liu, H., & Tang, J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.126 · Cited by 16
FairShades: Fairness Auditing via Explainability in Abusive Language Detection Systems
Manerba, M. M., & Guidotti, R. (2021) · IEEE International Conference on Cognitive Machine Intelligence (CogMI) · doi:10.1109/cogmi52975.2021.00014 · Cited by 11
Flexible text generation for counterfactual fairness probing
Fryer, Z., Axelrod, V., Packer, B., Beutel, A., Chen, J., & Webster, K. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.20 · Cited by 10
A study of bias mitigation strategies for speaker recognition
Peri, R., Somandepalli, K., & Narayanan, S. S. (2022) · Computer Speech & Language · doi:10.1016/j.csl.2022.101481 · Cited by 17
The Sins of the Parents Are to Be Laid Upon the Children: Biased Humans, Biased Data, Biased Models
Osborne, M. R., Omrani, A., & Dehghani, M. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231180099 · Cited by 9
Being Right for Whose Right Reasons?
Jakobsen, T. S. T., Cabello, L., & Søgaard, A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.59 · Cited by 10
Crowdsourced Detection of Emotionally Manipulative Language
Huffaker, J. S., Kummerfeld, J. K., Lasecki, W. S., & Ackerman, M. S. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376375 · Cited by 13
Analyzing Hate Speech Data along Racial, Gender and Intersectional Axes
Maronikolakis, A., Baader, P., & Schütze, H. (2022) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.1 · Cited by 9
Rye, E. C., Blackburn, J., & Beverly, R. (2020) · ACM Internet Measurement Conference (IMC) · doi:10.1145/3419394.3423615 · Cited by 9
Towards Ethics by Design in Online Abusive Content DetectionPreprint
Kiritchenko, S., & Nejadgholi, I. (2020) · arXiv (Cornell University) · doi:10.4224/40002027 · Cited by 15
GeDi: Generative Discriminator Guided Sequence GenerationPreprint
Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., & Rajani, N. F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.06367 · Cited by 164
The Ecological Fallacy in Annotation: Modeling Human Label Variation goes beyond Sociodemographics
Orlikowski, M., Röttger, P., Cimiano, P., & Hovy, D. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2023.acl-short.88 · Cited by 9
RL with KL penalties is better viewed as Bayesian inference
Korbak, T., Perez, E., & Buckley, C. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.77 · Cited by 8
Lopez, H., & Kübler, S. (2025) · Discourse Context & Media · doi:10.1016/j.dcm.2024.100848 · Cited by 13
TARGE: large language model-powered explainable hate speech detection
Hashir, M. H., Memoona, & Kim, S. W. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.2911 · Cited by 13
How Managers Perceive AI-Assisted Conversational Training for Workplace Communication
Wilhelm, L. A., Ding, X., Knutsen, K. M., Carik, B., & Rho, E. H. R. (2025) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3719160.3736639 · Cited by 13
A systematic survey of natural language processing for the Greek language
Bakagianni, J., Pouli, K., Gavriilidou, M., & Pavlopoulos, J. (2025) · Patterns · doi:10.1016/j.patter.2025.101313 · Cited by 20
Video sentiment analysis on social media using an advanced VADER technique
Dixit, M., Bali, M. S., Kour, K., Ioannou, I. I., Ghantasala, G. S. P., & Vassiliou, V. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00849-1 · Cited by 9
Improving Counterfactual Generation for Fair Hate Speech Detection
Davani, A. M., Omrani, A., Kennedy, B., Atari, M., Ren, X., & Dehghani, M. (2021) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2021.woah-1.10 · Cited by 9
Mind the data gap(s): Investigating power in speech and language datasets
Markl, N. (2022) · Workshop on Language Technology for Equality, Diversity, Inclusion (LT-EDI) · doi:10.18653/v1/2022.ltedi-1.1 · Cited by 11
Kaplan, D. M., Tidwell, C. A., Chung, J. M., Alisic, E., Demiray, B., Bruni, M. R., Evora, S., Gajewski‐Nemes, J. A., Macbeth, A., Mangelsdorf, S. N., et al. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02293-0 · Cited by 9
Modeling Empathic Similarity in Personal Narratives
Shen, J. J., Sap, M., Colon-Hernandez, P., Park, H. W., & Breazeal, C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.383 · Cited by 15
Self-citationDADA: Dialect Adaptation via Dynamic Aggregation of Linguistic Rules
Liu, Y., Held, W., & Yang, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.850 · Cited by 10
Putting algorithmic bias on top of the agenda in the discussions on autonomous weapons systems
Bhila, I. (2024) · Digital War · doi:10.1057/s42984-024-00094-z · Cited by 8
A Dataset and Baselines for Multilingual Reply Suggestion
Zhang, M., Wang, W., Deb, B., Zheng, G., Shokouhi, M., & Awadallah, A. H. (2021) · doi:10.18653/v1/2021.acl-long.97 · Cited by 8
Giorgi, S., Guntuku, S. C., Eichstaedt, J. C., Pajot, C., Schwartz, H. A., & Ungar, L. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18132 · Cited by 8
Adversarial Scrubbing of Demographic Information for Text Classification
Chowdhury, S. B. R., Ghosh, S., Li, Y., Oliva, J. B., Srivastava, S., & Chaturvedi, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.43 · Cited by 13
Havens, L., Alex, B., Bach, B., & Terras, M. (2022) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.4 · Cited by 9
Separating Hate Speech and Offensive Language Classes via Adversarial Debiasing
Yuan, S., Maronikolakis, A., & Schütze, H. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.1 · Cited by 9
Anti-Asian Hate Speech Detection via Data Augmented Semantic Relation Inference
Li, J., & Ning, Y. (2022) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v16i1.19319 · Cited by 11
Beyond Denouncing Hate: Strategies for Countering Implied Biases and Stereotypes in Language
Mun, J., Allaway, E., Yerukola, A., Vianna, L., Leslie, S., & Sap, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.653 · Cited by 8
Self-citationWhat can be done? : digital media policy options for strengthening European democracy
Nielsen, R. K., Gorwa, R., & Buning, M. D. C. (2019) · Cited by 8
Modeling the Second Player in Distributionally Robust OptimizationPreprint
Michel, P., Hashimoto, T., & Neubig, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.10282 · Cited by 16
Mitigating Biases in Hate Speech Detection from A Causal Perspective
Zhang, Z., Chen, J., & Yang, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.440 · Cited by 10
DBNLP: detecting bias in natural language processing system for India-centric languages
G, K. K. T., Mendke, S., Parihar, R., Mayya, S., Venkatesh, S., & Koolagudi, S. G. (2025) · International Journal of Information Technology · doi:10.1007/s41870-025-02437-9 · Cited by 7
Confront hate with AI: how AI-generated counter speech helps against hate speech on social Media?
Wu, C., Wang, Y., Zhang, Y., Wang, H., & Pang, Y. (2025) · Telematics and Informatics · doi:10.1016/j.tele.2025.102304 · Cited by 9
Exploring the Intersection of AI, Language, and Law: A Bibliometric Analysis
Lan, Z., & Zhang, X. (2026) · SAGE Open · doi:10.1177/21582440261431000 · Cited by 8
Using Social Media Categorical Reactions as a Gateway to Identify Hate Speech in COVID-19 News
de Oliveira, L. G., & Azevedo, J. (2022) · SN Computer Science · doi:10.1007/s42979-022-01421-5 · Cited by 7
Rewriting Algorithms for Just Recognition: From Digital Aural Redlining to Accent Activism
Eidsheim, N. S. (2023) · University of California Press eBooks · doi:10.1525/luminos.148.h · Cited by 7
Gupta, S. S., Lee, S., De‐Arteaga, M., & Lease, M. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583290 · Cited by 10
Socially Responsible Hate Speech Detection: Can Classifiers Reflect Social Stereotypes?
Vargas, F., Carvalho, I., Hürriyetoğlu, A., KNAW Humanities Cluster, D., Pardo, T. A. S., & Benevenuto, F. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_126 · Cited by 8
Synthetic Dataset Generation for Fairer Unfairness Research
Jiang, L., Belitz, C., & Bosch, N. (2024) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3636555.3636868 · Cited by 7
Bias-aware ranking from pairwise comparisons
Ferrara, A., Bonchi, F., Fabbri, F., Karimi, F., & Wagner, C. (2024) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-024-01024-z · Cited by 7
Not Judging a User by Their Cover
Jiang, J., & Vosoughi, S. (2020) · doi:10.1145/3422841.3423534 · Cited by 9
Ghosh, S., & Kumar, S. (2021) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2021.semeval-1.29 · Cited by 7
Hate versus politics: detection of hate against policy makers in Italian tweets
Duzha, A., Casadei, C., Tosi, M., & Celli, F. (2021) · SN Social Sciences · doi:10.1007/s43545-021-00234-2 · Cited by 8
Detecting Inspiring Content on Social Media
Ignat, O., Boureau, Y.-L., Yu, J. A., & Halevy, A. Y. (2021) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii52823.2021.9597431 · Cited by 8
Text Analysis in Python for Social Scientists
Hovy, D. (2022) · Cambridge University Press eBooks · doi:10.1017/9781108960885 · Cited by 8
Jurgens, D., Seth, A., Sargent, J., Aghighi, A., & Geraci, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.616 · Cited by 11
The Risk to Population Health Equity Posed by Automated Decision Systems: A Narrative Review
Burger, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06615 · Cited by 13
An Information Retrieval Approach to Building Datasets for Hate Speech DetectionPreprint
Rahman, M. M., Balakrishnan, D., Murthy, D., Kutlu, M., & Lease, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09775 · Cited by 16
Basoah, J., Chechelnitsky, D., Long, T., Reinecke, K., Zerva, C., Zhou, K., Díaz, M., & Sap, M. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732045 · Cited by 16
Self-citationDavidson, T. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02360-w · Cited by 7
Attribute-specific Cyberbullying Detection Using Artificial Intelligence
Orelaja, A., Ejiofor, C., Sarpong, S., Imakuh, S., Bassey, C., Opara, I., Tettey, J. N. A., & Akinola, O. A. (2024) · Journal of Electronic & Information Systems · doi:10.30564/jeis.v6i1.6206 · Cited by 6
Towards a Critical Social Epistemology of Social Media
Habgood‐Coote, J. (2025) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780190949945.013.21 · Cited by 6
Markl, N., & Lai, C. (2021) · Cited by 6
An Overview of Fairness in Data – Illuminating the Bias in Data Pipeline
Kumar, B. S., Chandrabose, A., & Chakravarthi, B. R. (2021) · Cited by 6
Stanceosaurus: Classifying Stance Towards Multicultural Misinformation
Zheng, J., Baheti, A., Naous, T., Xu, W. H., & Ritter, A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.138 · Cited by 12
Bender, E. M., & Grissom, A. (2024) · Oxford University Press eBooks · doi:10.1093/oso/9780197755303.003.0010 · Cited by 7
Comparative Analysis of GPT Models for Detecting Cyberbullying in Social Media Platforms Threads
Islam, M. S., & Rafiq, R. I. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-63616-5_25 · Cited by 7
Cross-lingual hate speech detection based on multilingual domain-specific word embeddingsPreprint
Arango, A., Pérez, J. E. P., & Poblete, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.14728 · Cited by 6
Information-Theoretic Measures of Dataset Difficulty.Preprint
Ethayarajh, K., Choi, Y., & Swayamdipta, S. (2021) · arXiv (Cornell University) · Cited by 6
Self-citationReconsidering Annotator Disagreement about Racist Language: Noise or Signal?Preprint
Larimore, S., Kennedy, I., Haskett, B., & Arseniev‐Koehler, A. (2021) · doi:10.31235/osf.io/4wu7r · Cited by 6
Lameiro, F., Dunagan, L., Card, D., Gilbert, E., & Haimson, O. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732095 · Cited by 6
Self-citationPonnusamy, R., Sivagnanam, B., Kizhakkeparambil, A., Du, P., Saranya, R., Paul Buitelaar, C., & Chakravarthi, B. R. (2026) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3776574.3832465 · Cited by 5
Task-Agnostic Low-Rank Adapters for Unseen English Dialects
Xiao, Z., Held, W., Liu, Y., & Yang, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.487 · Cited by 12
StereoMap: Quantifying the Awareness of Human-like Stereotypes in Large Language Models
Jeoung, S., Ge, Y., & Diesner, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.752 · Cited by 12
Analyzing Norm Violations in Live-Stream Chat
Moon, J., Lee, D., Cho, H., Jin, W., Park, C., Kim, M., May, J., Pujara, J., & Park, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.55 · Cited by 6
Giorgi, S., Bellew, D., Habib, D. R. S., Sedoc, J., Smitterberg, C. W., Devoto, A., Himelein-Wachowiak, M., & Curtis, B. L. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31328 · Cited by 15
Dutta, A., BANDUCCI, S. A., & Camargo, C. Q. (2025) · Scientometrics · doi:10.1007/s11192-025-05410-2 · Cited by 12
Should AI Mimic People? Understanding AI-Supported Writing Technology Among Black Users
Basoah, J., Cunningham, J. L., Adams, E. E., Bose, A., Jain, A., Yadav, K., Yang, Z., Reinecke, K., & Rosner, D. K. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757423 · Cited by 10
Who Watches the Birdwatchers? Sociotechnical Vulnerabilities in Twitter’s Content Contextualisation
Benjamin, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-10183-0_1 · Cited by 5
Investigating Annotator Bias in Abusive Language Datasets
Wich, M., Widmer, C., Hagerer, G., & Groh, G. (2021) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-072-4_170 · Cited by 12
Towards a Deep Multi-layered Dialectal Language Analysis: A Case Study of African-American English
Dacon, J. (2022) · Workshop on Bridging Human-Computer Interaction and Natural Language Processing (HCI+NLP) · doi:10.18653/v1/2022.hcinlp-1.8 · Cited by 7
Siri, Stereotypes, and the Mechanics of Sexism
Elder, A. M. (2022) · Feminist Philosophy Quarterly · doi:10.5206/fpq/2022.3/4.14294 · Cited by 6
Cross-lingual Transfer Can Worsen Bias in Sentiment Analysis
Goldfarb-Tarrant, S., Roß, B., & Lopez, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.346 · Cited by 13
K-HATERS: A Hate Speech Detection Corpus in Korean with Target-Specific Ratings
Park, C., Kim, S., Park, K., & Park, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.952 · Cited by 7
Zsisku, E., Zubiaga, A., & Dubossarsky, H. (2024) · ACM Web Science Conference (WebSci) · doi:10.1145/3614419.3644025 · Cited by 5
Hate Speech Detection for Transliterated English and Sinhala Code-Mixed Data
Muthuthanthri, M., & Smith, R. (2024) · International Conference on Advanced Research in Computing (ICARC) · doi:10.1109/icarc61713.2024.10499768 · Cited by 5
Empowering Safe Online Spaces: AI in Gender Violence Detection and Prevention
Manche, R., Samaah, F., Bollikonda, T., & Myakala, P. K. (2025) · Journal of Science and Technology · doi:10.46243/jst.2025.v10.i02.pp39-50 · Cited by 9
Policycorpus XL: An Italian Corpus for the detection of Hate Speech Against Politics
Celli, F., Lai, M., Duzha, A., Bosco, C., & Patti, V. (2022) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.10505 · Cited by 5
Adding Instructions during Pretraining: Effective way of Controlling Toxicity in Language Models
Prabhumoye, S., Patwary, M. A., Shoeybi, M., & Catanzaro, B. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.193 · Cited by 17
RECAST: Interactive Auditing of Automatic Toxicity Detection Models
Wright, A. P., Shaikh, O., Park, H., Epperson, W., Ahmed, M., Pinel, S., Yang, D., & Chau, D. H. (2020) · doi:10.1145/3403676.3403691 · Cited by 5
Trouble on the Horizon: Forecasting the Derailment of Online Conversations as they DevelopPreprint
Chang, J. P., & Danescu-Niculescu-Mizil, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01362 · Cited by 14
What do Bias Measures MeasurePreprint
Dev, S., Sheng, E., Zhao, J., Sun, J., Hou, Y., Sanseverino, M., Kim, J., Peng, N., & Chang, K.-W. (2021) · arXiv (Cornell University) · Cited by 6
NLPositionality: Characterizing Design Biases of Datasets and Models
Santy, S., Liang, J. T., Bras, R. L., Reinecke, K., & Sap, M. (2023) · arXiv · doi:10.48550/arxiv.2306.01943 · Cited by 31
“…has also reported various biases against these populations in datasets and models: people who are non-binary (e.g., Dev et al., 2021), Black (e.g., Sap et al., 2019; Davidson et al., 2019), Latinx (e.g., Dodge et al., 2021), Native American (e.g., Mager et al., 2018); and people who are not…”
BackgroundSelf-citationDigital accents, homogeneity-by-design, and the evolving social science of written language
Alvero, A., Sedlacek, Q. C., León, M., & Peña, C. (2025) · Annual Review of Applied Linguistics · doi:10.1017/s0267190525000042 · Cited by 6
Pandey, S. S., Garcia‐Robledo, A., & Zangiabady, M. (2024) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc59802.2024.00059 · Cited by 4
Large Language Models and Knowledge Graphs: A State-of-the-Art Exploration
Gallardo-Delgado, A. A., Nolazco‐Flores, J. A., Sánchez, B. P., & Buscaldi, D. (2025) · SN Computer Science · doi:10.1007/s42979-025-04277-7 · Cited by 4
Wei, D., Chau, M., & Li, Z. (2025) · MIS Quarterly · doi:10.25300/misq/2025/18416 · Cited by 4
“Watch me pretend to punch my girlfriend”: exploring youth responses to viral dating violence
Maddocks, S., & Parfaite, F. (2023) · Feminist Media Studies · doi:10.1080/14680777.2023.2185564 · Cited by 5
Sahoo, N., Mallela, N., & Bhattacharyya, P. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.842 · Cited by 4
A Weakly Supervised Classifier and Dataset of White Supremacist Language
Yoder, M. M., Diab, A., Brown, D., & Carley, K. M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.17 · Cited by 5
The State of Profanity Obfuscation in Natural Language Processing Scientific Publications
Nozza, D., & Hovy, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.240 · Cited by 5
Metadata
- Authors
- Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A.
- Affiliations
- University of Washington (US)
- Carnegie Mellon University (US)
- Allen Institute for Artificial Intelligence (US)
- Corresponding author
- Sap, Maarten
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 1668-1678
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2949678053
- Cited by
- 817
- References
- 40
- Citations by year
- 2026: 59 · 2025: 96 · 2024: 86 · 2023: 139 · 2022: 142 · 2021: 178 · 2020: 105 · 2019: 12
- FWCI
- 40.99
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- National Science Foundation (1714566)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- hate speech detection, racial bias, African American English, dialects, annotation bias, hate speech datasets, priming, minority groups
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
