The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Reducing Gender Bias in Abusive Language Detection
Open accessAbstract
Abusive language detection models tend to have a problem of being biased toward identity words of a certain group of people because of imbalanced training datasets.For example, "You are a good woman" was considered "sexist" when trained on an existing dataset.Such model bias is an obstacle for models to be robust enough for practical use.In this work, we measure gender biases on models trained with different abusive language datasets, while analyzing the effect of different pre-trained word embeddings and model architectures.We also experiment with three bias mitigation methods: (1) debiased word embeddings, (2) gender swap data augmentation, and (3) fine-tuning with a larger corpus.These methods can effectively reduce gender bias by 90-98% and can be extended to correct model bias in other scenarios.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Prior work has shown that toxicity classifiers can pick up on biases in the training data and spuriously associate toxicity with the mention of certain demographics (Park et al., 2018; Dixon et al., 2018).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Park et al., 2018
In the citing paper
“models are trained with historical data and can present issues such as gender bias and racial bias in AI models [68,73], potentially exacerbating structural inequalities [10]”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [68]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Talat, Z., & Hovy, D. (2016) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n16-2013 · Cited by 1,919 · cited 2×
“So far, many efforts were put into defining and constructing abusive language datasets from different sources and labeling them through crowd-sourcing or user moderation (Waseem and Hovy, 2016; Waseem, 2016; Founta et al., 2018; Wulczyn et al., 2017).”
BackgroundDeep Learning for Hate Speech Detection in Tweets
Badjatiya, P., Gupta, S., Gupta, M., & Varma, V. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3041021.3054223 · Cited by 1,253
Felbo, B., Mislove, A. E., Søgaard, A., Rahwan, I., & Lehmann, S. (2017) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 712
Gender bias in coreference resolution: Evaluation and debiasing methods
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2003 · Cited by 684 · cited 3×
“This simple method removes correlation between gender and classification decision and has proven to be effective for correcting gender biases in co-reference resolution task (Zhao et al., 2018).”
BackgroundMethodInfluentialMeasuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674 · cited 5×
“Therefore, we generate a separate unbiased test set for each gender, male and female, using the identity term template method proposed in Dixon et al. (2017).”
BackgroundMethodInfluentialEx Machina: Personal Attacks Seen at Scale
Wulczyn, E., Thain, N., & Dixon, L. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052591 · Cited by 636 · cited 1×
“So far, many efforts were put into defining and constructing abusive language datasets from different sources and labeling them through crowd-sourcing or user moderation (Waseem and Hovy, 2016; Waseem, 2016; Founta et al., 2018; Wulczyn et al., 2017).”
BackgroundLarge Scale Crowdsourcing and Characterization of Twitter Abusive Behavior
Founta, A.-M., Djouvas, C., Chatzakou, D., Ilias Leontiadis, A., Blackburn, J., Stringhini, G., Vakali, A. I., Sirivianos, M., & Kourtellis, N. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.14991 · Cited by 605
Research works printed but not linked: 6
- Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Ad- vances in Neural Information Processing Systems, pages 4349-4357.
- Svetlana Kiritchenko and Saif M Mohammad. 2018. Examining gender and race bias in two hundred sen- timent analysis systems. Proceedings of the 7th Joint Conference on Lexical and Computational Se- mantics(*SEM), New Orleans, USA.
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013. Distributed representa- tions of words and phrases and their compositional- ity. In NIPS, pages 3111-3119.
- Ji Ho Park and Pascale Fung. 2017. One-step and two- step classification for abusive language detection on twitter. ALW1: 1st Workshop on Abusive Language Online to be held at the annual meeting of the Asso- ciation of Computational Linguistics (ACL) 2017.
- Zeerak Waseem. 2016. Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter. In Proceedings of the first workshop on NLP and computational social science, pages 138- 142.
- Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with ad- versarial learning. Proceedings of AAAI/ACM Con- ference on Ethics and Society(AIES) 2018.
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-citationLanguage (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050
“…(2017); Schnoebelen (2017); Blodgett et al. (2018); Curry and Rieser (2018); Díaz et al. (2018); Dixon et al. (2018); Kiritchenko and Mohammad (2018); Park et al. (2018); Shen et al. (2018); Thelwall (2018); Zhao et al. (2018b); Badjatiya et al. (2019); Bagdasaryan et al. (2019); Bamman et al.…”
BackgroundInfluentialThe Risk of Racial Bias in Hate Speech Detection
Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1163 · Cited by 817
“Since previous research has exposed the potential for other identity-based biases in offensive language detection (e.g., gender bias; Park et al., 2018), here we investigate racial bias against speech by African Americans, focusing on Twitter as it is a particularly important space for Black activism (Williams and Dom…”
BackgroundMethodFairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646
“Unintended bias can be introduced to the data by personal biases during manual data labelling [84] as well as biases that occur in language use for example through non-native speaker errors [293, 258, 35] or how the text data is prepared as well as the general model architecture [215].”
Mitigating Gender Bias in Natural Language Processing: Literature Review
Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W., & Wang, W. Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1159 · Cited by 492
“Park et al. (2018) find that gender-swapping was more effective at both removing bias and retaining performance than bias fine-tuning.”
BackgroundMethodInfluential
Metadata
- Authors
- Park, J. H., Shin, J., & Fung, P.
- Affiliations
- Hong Kong University of Science and Technology (HK)
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 2799-2804
- OpenAlex
- W2888167352
- Cited by
- 323
- References
- 13
- Citations by year
- 2026: 1 · 2025: 23 · 2024: 26 · 2023: 56 · 2022: 53 · 2021: 75 · 2020: 53 · 2019: 34 · 2018: 2
- FWCI
- 16.75
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Hong Kong University of Science and Technology
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- offensive language detection, gender bias, model bias, bias mitigation, fine-tuning, imbalanced training data, word embeddings
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
