The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Reducing Gender Bias in Abusive Language Detection
Open accessAbstract
Abusive language detection models tend to have a problem of being biased toward identity words of a certain group of people because of imbalanced training datasets.For example, "You are a good woman" was considered "sexist" when trained on an existing dataset.Such model bias is an obstacle for models to be robust enough for practical use.In this work, we measure gender biases on models trained with different abusive language datasets, while analyzing the effect of different pre-trained word embeddings and model architectures.We also experiment with three bias mitigation methods: (1) debiased word embeddings, (2) gender swap data augmentation, and (3) fine-tuning with a larger corpus.These methods can effectively reduce gender bias by 90-98% and can be extended to correct model bias in other scenarios.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Prior work has shown that toxicity classifiers can pick up on biases in the training data and spuriously associate toxicity with the mention of certain demographics (Park et al., 2018; Dixon et al., 2018).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Park et al., 2018
In the citing paper
“models are trained with historical data and can present issues such as gender bias and racial bias in AI models [68,73], potentially exacerbating structural inequalities [10]”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [68]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Talat, Z., & Hovy, D. (2016) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n16-2013 · Cited by 1,919 · cited 2×
“So far, many efforts were put into defining and constructing abusive language datasets from different sources and labeling them through crowd-sourcing or user moderation (Waseem and Hovy, 2016; Waseem, 2016; Founta et al., 2018; Wulczyn et al., 2017).”
BackgroundDeep Learning for Hate Speech Detection in Tweets
Badjatiya, P., Gupta, S., Gupta, M., & Varma, V. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3041021.3054223 · Cited by 1,253
Felbo, B., Mislove, A. E., Søgaard, A., Rahwan, I., & Lehmann, S. (2017) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 712
Gender bias in coreference resolution: Evaluation and debiasing methods
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2003 · Cited by 684 · cited 3×
“This simple method removes correlation between gender and classification decision and has proven to be effective for correcting gender biases in co-reference resolution task (Zhao et al., 2018).”
BackgroundMethodInfluentialMeasuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674 · cited 5×
“Therefore, we generate a separate unbiased test set for each gender, male and female, using the identity term template method proposed in Dixon et al. (2017).”
BackgroundMethodInfluentialEx Machina: Personal Attacks Seen at Scale
Wulczyn, E., Thain, N., & Dixon, L. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052591 · Cited by 636 · cited 1×
“So far, many efforts were put into defining and constructing abusive language datasets from different sources and labeling them through crowd-sourcing or user moderation (Waseem and Hovy, 2016; Waseem, 2016; Founta et al., 2018; Wulczyn et al., 2017).”
BackgroundLarge Scale Crowdsourcing and Characterization of Twitter Abusive Behavior
Founta, A.-M., Djouvas, C., Chatzakou, D., Ilias Leontiadis, A., Blackburn, J., Stringhini, G., Vakali, A. I., Sirivianos, M., & Kourtellis, N. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.14991 · Cited by 605
Flagged first, then by citations
Research works printed but not linked: 6
- Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Ad- vances in Neural Information Processing Systems, pages 4349-4357.
- Svetlana Kiritchenko and Saif M Mohammad. 2018. Examining gender and race bias in two hundred sen- timent analysis systems. Proceedings of the 7th Joint Conference on Lexical and Computational Se- mantics(*SEM), New Orleans, USA.
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013. Distributed representa- tions of words and phrases and their compositional- ity. In NIPS, pages 3111-3119.
- Ji Ho Park and Pascale Fung. 2017. One-step and two- step classification for abusive language detection on twitter. ALW1: 1st Workshop on Abusive Language Online to be held at the annual meeting of the Asso- ciation of Computational Linguistics (ACL) 2017.
- Zeerak Waseem. 2016. Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter. In Proceedings of the first workshop on NLP and computational social science, pages 138- 142.
- Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with ad- versarial learning. Proceedings of AAAI/ACM Con- ference on Ethics and Society(AIES) 2018.
Cited by
Sorted by citations
Items 201–300 of 322
Supporting Complaints Investigation for Nursing and Midwifery Regulatory Agencies
Lertvittayakumjorn, P., Petej, I., Gao, Y., Krishnamurthy, Y., van der Gaag, A., Jago, R., & Stathis, K. (2021) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2021.acl-demo.10 · Cited by 7
“Popular gender debias-ing methods include gender swapping (Zhao et al., 2018), gender-debiased word embeddings (Boluk-basi et al., 2016; Manzini et al., 2019), adversarial training (Zhang et al., 2018), and fine-tuning (Park et al., 2018).”
MethodSex and gender bias in natural language processing
Cirillo, D., Gonen, H., Santus, E., Valencia, A., Costa‐jussà, M. R., & Villegas, M. (2022) · Elsevier eBooks · doi:10.1016/b978-0-12-821392-6.00009-1 · Cited by 8
Mitigation of Unintended Biases against Non-Native English Texts in Sentiment Analysis.
Zhiltsova, A., Caton, S. J., & Mulway, C. (2019) · IASC · Cited by 7
Youth language and emerging slurs: tackling bias in BERT-based hate speech detection
Fillies, J., & Paschke, A. (2025) · AI and Ethics · doi:10.1007/s43681-025-00701-z · Cited by 6
NLPGuard: A Framework for Mitigating the Use of Protected Attributes by NLP Classifiers
Greco, S., Zhou, K., Capra, L., Cerquitelli, T., & Quercia, D. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686924 · Cited by 6
“AI regulations will continue to evolve in the coming years [34, 49], driven by a common goal: to minimize discriminatory outputs based on protected characteristics [10, 21, 26].”
Gender Stereotypes Differ between Male and Female Writings
Qian, Y. (2019) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/p19-2007 · Cited by 5
“Park et al. (2018) measured gender stereotypes on various abusive language models, while analyzing the effect of different pre-trained word embeddings and model architectures.”
BackgroundHateful Messages: A Conversational Data Set of Hate Speech Produced by Adolescents on Discord
Fillies, J., Peikert, S., & Paschke, A. (2024) · Springer eBooks · doi:10.1007/978-3-031-42171-6_5 · Cited by 11
Does ChatGPT show gender bias in behavior detection?
Wu, J., Song, Y., & Wu, D. C. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-04219-3 · Cited by 6
“For instance, studies like (Dixon et al., 2018; Park et al., 2018; Zhang et al., 2020) delve into the discriminatory behavior exhibited by text classi fi cation models when encountering texts containing demographic terms such as “ gay ” or “ Muslim.”
Meme and cyber sexism: Habitus and symbolic violence of patriarchy on the Internet
Maulana, M. F. (2021) · SIMULACRA JURNAL SOSIOLOGI · doi:10.21107/sml.v4i2.11899 · Cited by 8
“Laurie Penny (2013) explained that the Internet must be a neutral place not insulated by social, cultural, gender, economic, and political structures.”
BackgroundTrouble on the Horizon: Forecasting the Derailment of Online Conversations as they DevelopPreprint
Chang, J. P., & Danescu-Niculescu-Mizil, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01362 · Cited by 14
“tion that any automated systems might encode or even amplify the biases existing in the training data (Park et al., 2018; Sap et al., 2019; Wiegand et al., 2019), so a public-facing implementation would need to be exhaustively scrutinized for such biases (Feldman et al.”
BackgroundWhat do Bias Measures MeasurePreprint
Dev, S., Sheng, E., Zhao, J., Sun, J., Hou, Y., Sanseverino, M., Kim, J., Peng, N., & Chang, K.-W. (2021) · arXiv (Cornell University) · Cited by 6
Fair Text Classification with Wasserstein Independence
Leteno, T., Gourru, A., Laclau, C., Emonet, R., & Gravier, C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.978 · Cited by 7
Explainable AI Discloses Gender Bias in Sexism Detection Algorithm
Muntasir, F., & Noor, J. (2024) · International Conference on Networking, Systems and Security (NSysS) · doi:10.1145/3704522.3704524 · Cited by 5
“Gender swap data augmentation was discussed in [20, 32].”
Reward Model Interpretability via Optimal and Pessimal Tokens
Christian, B., Kirk, H. R., Thompson, J. A. F., Summerfield, C., & Dumbalska, T. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732068 · Cited by 10
“…documented phenomena in hate-speech detection, where models develop oversensitive false positive rates for identity terms (e.g., “Muslim,” “mosque”) or reclaimed slurs due to their overrepresentation in negative training contexts and underrepresentation in neutral or positive ones [8, 38, 41].”
GenderQuant: Quantifying Mention-Level Genderedness
Ananya, Parthasarthi, N., & Singh, S. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1303 · Cited by 4
“…object classification and visual semantic role labeling are gender-biased and that models trained on these datasets amplify this bias, while Rudinger et al. (2017) find racial, religious and gender stereotypes in the SNLI corpus and Park et al. (2018) analyze gender bias in abusive language datasets.”
BackgroundYee, K., Sebag, A. S., Redfield, O., Eck, M., Sheng, E., & Belli, L. S. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.10 · Cited by 5
CPL-NoViD: Context-Aware Prompt-Based Learning for Norm Violation Detection in Online Communities
He, Z., May, J., & Lerman, K. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31335 · Cited by 5
Efficiently Mitigating Classification Bias via Transfer Learning.Preprint
Jin, X., Barbieri, F., Davani, A. M., Kennedy, B., Neves, L., & Ren, X. (2020) · arXiv (Cornell University) · Cited by 4
Detecting Inappropriate Messages on Sensitive Topics that Could Harm a Company's ReputationPreprint
Babakov, N., Logacheva, V., Kozlova, O., Semenov, N., & Panchenko, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.05345 · Cited by 11
Hate Speech Classifiers Learn Human-Like Social StereotypesPreprint
Davani, A. M., Atari, M., Kennedy, B., & Dehghani, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14839 · Cited by 4
“These statistics are consistent with observations of previous findings (Davidson et al., 2017; Kwok & Wang, 2013; Dixon et al., 2018; Park et al., 2018), which identify false-positive errors as the more critical issue with hate speech classifiers.”
ResultProceedings of The ACM Collective Intelligence Conference
(2023) · doi:10.1145/3582269 · Cited by 4
“Since Lawyers are Males..”: Examining Implicit Gender Bias in Hindi Language Generation by LLMs
Gupta, I., Joshi, I., Dey, A., & Parikh, T. S. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732208 · Cited by 10
An Empirical Study of the Downstream Reliability of Pre-Trained Word Embeddings
Rios, A., & Lwowski, B. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.299 · Cited by 3
“It was then used to evaluate the fairness of offensive language classifiers in Park et al. (2018).”
BackgroundMethodInfluentialIncorporating subjectivity into gendered ambiguous pronoun (GAP) resolution using style transfer
Pant, K., & Dadu, T. (2022) · Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.28 · Cited by 3
“Many recent studies aim to detect, analyze, and mitigate gender bias in different NLP tools and applications (Bolukbasi et al., 2016; Rudinger et al., 2018; Park et al., 2018).”
BackgroundIdentity Term Sampling for Measuring Gender Bias in Training Data
Sobhani, N. C., & Delany, S. J. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-26438-2_18 · Cited by 3
Targeting the Source: Selective Data Curation for Debiasing NLP Models
Gaci, Y., Benatallah, B., Casati, F., & Benabdeslem, K. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43415-0_17 · Cited by 3
Uncertainty Estimation for Debiased Models: Does Fairness Hurt Reliability?
Kuzmin, G., Vazhentsev, A., Shelmanov, A., Han, X., Šuster, S., Panov, M., Panchenko, A., & Baldwin, T. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-main.48 · Cited by 3
“ML models often exhibit biases due to artifacts in training data or pre-trained language models that can cause, for example, unfair decisions correlated with race, gender, and other demographic and socio-economic factors (Díaz et al., 2019; Park et al., 2018; Badjatiya et al., 2019).”
BackgroundA Multi-task Learning Model for Fine-Grain Dialogue Social Bias Measurement
Mai, H., Zhou, X., & Wang, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17189-5_27 · Cited by 3
Femininity and Masculinity in Arabic Words: Gender Marking in Muslim Cosmology
Muassomah, M., Wargadinata, W., Rohmah, G. N., Indah, R. N., Masitoh, S., Istiadah, I., & Abdullah, I. (2021) · Langkawi Journal of The Association for Arabic and English · doi:10.31332/lkw.v7i2.3130 · Cited by 3
“Studies of the link between gender and language have found that language embodies particular inequalities and biases and may even be sexist and abusive (Menegatti & Rubini, 2017; Park et al., 2018; Weatherall, 2016).”
BackgroundPELECEHAN TERHADAP PEREMPUAN DALAM MEME
Hidayat, H. N., & Immerry, T. (2020) · Kafa`ah Journal of Gender Studies · doi:10.15548/jk.v10i2.330 · Cited by 3
Determining Bias in Machine Translation with Deep Learning Techniques
Font, J. E. (2019) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · Cited by 3
Dialect Diversity in Text Summarization on TwitterPreprint
Celis, L. E., & Keswani, V. (2020) · arXiv (Cornell University) · Cited by 2
Fair Hate Speech Detection through Evaluation of Social Group CounterfactualsPreprint
Davani, A. M., Omrani, A., Kennedy, B., Atari, M., Ren, X., & Dehghani, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12779 · Cited by 6
“Satisfying these metrics has motivated data augmentation approaches to balance the data distribution (Dixon et al. 2018; Zhao et al. 2018; Park, Shin, and Fung 2018) or fair input representations to equalize model performance with respect to protected groups (Madras et al. 2018; Zhang, Lemoine, and…”
BackgroundMethodEverything Has a Cause: Leveraging Causal Inference in Legal Text AnalysisPreprint
Liu, X., Yin, D., Feng, Y., Wu, Y., & Zhao, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.09420 · Cited by 3
Investigating Bias In Automatic Toxic Comment Detection: An Empirical StudyPreprint
Kumar, A., & Kumar, P. M. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.06487 · Cited by 5
“Park et al. (2018) [5] used the combination of debiased word2vec and gender swap data augmentation to reduce the gender bias.”
Zhao, Z., Zhang, Z., & Hopfgartner, F. (2021) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.48550/arxiv.2109.02691 · Cited by 3
Feature Attribution-Guided Contrastive Learning: Mitigating Lexical Bias in Toxic Speech Detection
Peng, Z., Ren, Y., Wang, D., Zhang, L., Yuan, L., & Ji, Y. (2023) · International Conference on New Trends in Computational Intelligence (NTCI) · doi:10.1109/ntci60157.2023.10403665 · Cited by 2
“Park et al. [14] replaced sentences with male entities in the dataset with female entities to address the model’s bias against females.”
BackgroundBias mitigation in text classification through cGAN and LLMs
Kumar, G., & Singh, J. P. (2024) · Proceedings of the Indian National Science Academy · doi:10.1007/s43538-024-00371-1 · Cited by 2
“The pre-processing of the dataset to reduce the bias was done by researchers such as Calmon et al. (2017), Kamiran and Calders (2012), and Park et al. (2018).”
Rizvi, N., Flores, A. M., Rizvi, M., Ousidhoum, N. D., & Munyaka, I. N. S. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757478 · Cited by 3
“Similarly, researchers have shown that toxicity models often label sentences negatively for the mere presence of gender identity words [53] or disability terms [48].”
Mining the online infosphere: A survey
Adak, S., Chakraborty, S., Das, P., Das, M., Dash, A., Hazra, R., Mathew, B., Saha, P., Sarkar, S., & Mukherjee, A. (2022) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1453 · Cited by 3
Suum Cuique: Studying Bias in Taboo Detection with a Community Perspective
Khalid, O., Rusert, J., & Srinivasan, P. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.227 · Cited by 2
Analysis of Bias in NLP Models With Regression and Effect Sizes
Dayanık, E., Vu, N. T., & Padó, S. (2022) · Northern European Journal of Language Technology · doi:10.3384/nejlt.2000-1533.2022.3505 · Cited by 3
“Approaches from the first family create a modified version of the original data set that is biased in the opposite direction, training models on the union (Park et al., 2018; Zhao et al., 2019; Stanovsky et al., 2019).”
BackgroundUnmasking the Stereotypes: Evaluating Social Biases in Chinese BERT
Zhang, S., & Li, P. (2022) · 2022 4th International Conference on Natural Language Processing (ICNLP) · doi:10.1109/icnlp55136.2022.00059 · Cited by 2
“In addition, bias has been found in machine translation [10][11][12], emotion analysis [13], speech recognition [14], recommendation system [15], language generation [16] and other fields.”
BackgroundAdversarial Removal of Population Bias in Genomics Phenotype Prediction
Zhao, H., & Wang, W. (2022) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw58026.2022.00052 · Cited by 2
“In deep text classification models, adding additional data [23] and template-based term switching [24] are used to mitigate explicit bias caused by the imbalance of training data.”
MethodPEFTDebias : Capturing debiasing information using PEFTs
Agarwal, S. D., Veerubhotla, A. S., & Bansal, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.122 · Cited by 3
Oppong, A. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3583942 · Cited by 2
Pablo, D. O., Badri, S., Norén, E., & Nötzli, C. (2023) · Journal of WSCG · doi:10.24132/jwscg.2023.6 · Cited by 3
Pu, C., & Zhou, X. (2023) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2023.semeval-1.59 · Cited by 2
“However, there are still some deficiencies in current sexism detection, which need to be improved, such as the abuse of language detection models, and the misestimation of sexism (Park et al., 2018).”
BackgroundSobhani, N., Sengupta, K., Microsoft, D., & Delany, S. J. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_119 · Cited by 2
“This replacement technique aimed to avoid mitigating bias through gender swapping which is a technique to mitigate bias by augmenting the data with additional examples that are gender-swapped (Park et al., 2018).”
BackgroundMethodInfluentialMcMilin, E. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29842 · Cited by 2
“Successes seen in rebal-ancing data corpora (Webster et al. 2018) and retraining or fine-tuning models (Zhao et al. 2018; Park, Shin, and Fung 2018) have become less practical at the current scale of LLMs.”
BackgroundMeasure Utility, Gain Trust: Practical Advice for XAI ResearcherPreprint
Davis, B., Glenski, M., Sealy, W. I. N., & Arendt, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.12924 · Cited by 2
“These domains include predictive policing [17], recidivism prediction [12,15], and hate speech and abusive language identification online [13,36,40].”
BackgroundAlgorithmic Censorship by Social Platforms: Power and ResistancePreprint
Cobbe, J. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3437304 · Cited by 2
Detecting Cross-Geographic Biases in Toxicity Modeling on Social MediaPreprint
Ghosh, S., Baker, D., Jurgens, D., & Prabhakaran, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.06999 · Cited by 2
“…recent research has shown that these models often encode various societal biases against marginalized groups (Waseem, 2016; Caliskan et al., 2017; Park et al., 2018; van Aken et al., 2018; Dixon et al., 2018; Sap et al., 2019; Davidson et al., 2019; Hutchinson et al., 2020; Zhou et al., 2021)…”
BackgroundHate speech and offensive language detection using transfer learning approaches
Mozafari, M. (2021) · theses.fr (ABES) · doi:10.70675/9fc685efz2a9cz4fa0z9041z5001b09e8195 · Cited by 2
Lahjoita puhetta: a large-scale corpus of spoken Finnish with some benchmarks
Moisio, A., Porjazovski, D., Rouhe, A., Getman, Y., Virkkunen, A., AlGhezi, R., Lennes, M., Grósz, T., Lindén, K., & Kurimo, M. (2022) · Language Resources and Evaluation · doi:10.1007/s10579-022-09606-3 · Cited by 24
Weld, G., Ayton, E., Althoff, T., & Glenski, M. (2021) · Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda (NLP4IF) · doi:10.18653/v1/2021.nlp4if-1.5 · Cited by 2
Protecting marginalized communities by mitigating discrimination in toxic language detection
Faal, F., Schmitt, K., & Yu, J. Y. (2021) · IEEE International Symposium on Technology and Society (ISTAS) · doi:10.1109/istas52410.2021.9629201 · Cited by 1
Offensiveness Analysis of Chinese Group Addressing Terms and Dataset Construction
Zhu, S., & Liu, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-0586-3_27 · Cited by 1
Bias Detection and Mitigation in Textual Data: A Study on Fake News and Hate Speech Detection
Kasampalis, A., Chatzakou, D., Tsikrika, T., Vrochidis, S., & Kompatsiaris, I. Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56063-7_29 · Cited by 1
Evaluation of Gender Bias in Amharic Word Embedding Model
Zenebe, B., Gizaw, S., & Abgaz, Y. M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70242-6_17 · Cited by 1
Examining inferred author and textual correlates of harmful language annotation
Korre, K., Yenikent, S., Basile, A., Spallaccia, B., Franco-Salvador, M., & Barrón‐Cedeño, A. (2025) · Language Resources and Evaluation · doi:10.1007/s10579-025-09822-7 · Cited by 3
“For example, Park et al. (2018) note that phrases such as “You are a good woman” are marked as ‘sexist’ probably because of the word ‘woman’, underlining that this unfair bias can inhibit the robustness of models and therefore their efficient practical use.”
Debiasing Hate Speech Classification Models for Queer Language Through Keyword Analysis
Yahathugoda, D. S., Wijesinghe, R., & Weerasinghe, R. (2025) · International Journal on Advances in ICT for Emerging Regions (ICTer) · doi:10.4038/icter.v18i2.7296 · Cited by 1
“A study proposed a framework to measure and mitigate the bias of toxicity detection models towards different demographic groups, such as race, gender, religion, etc[13].”
Large Language Model Annotation Bias in Hate Speech Detection
Okpala, E., & Cheng, L. (2025) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v19i1.35879 · Cited by 4
“…2017; Golbeck et al. 2017; Founta et al. 2018; Zampieri et al. 2019; Caselli et al. 2020; Basile et al. 2019) widely used in hate speech detection (Park, Shin, and Fung 2018; Lee, Yoon, and Jung 2018; Van Aken et al. 2018; Kapoor et al. 2019; Caselli et al. 2021; Li et al. 2021) and racial bias…”
Taso, F. T. D. S., dos Reis, V. Q., & Martinez, F. V. (2025) · Journal on Interactive Systems · doi:10.5753/jis.2025.5958 · Cited by 1
Enhancing RLHF in LLMs: Comparing BERT, XGBoost, and Deep Models for Bias Detection
Torfa, W., Elgendy, O., & Nassif, A. B. (2025) · International Conference on Developments in eSystems Engineering (DeSE) · doi:10.1109/dese68208.2025.11367955 · Cited by 1
“In [13] Park et al. addressed the issue of gender bias in abusive language detection models, which often misclassify benign sentences containing identity terms (e.g., "woman") as offensive due to the biased training data.”
EquiSent: A Scalable Solution for Bias-Aware Gender-Fair Sentiment Modeling
Kundu, P., & Senthilselvi, A. (2025) · International Conference on Advances in Computation, Communication and Information Technology (ICAICCIT) · doi:10.1109/icaiccit68829.2025.11434462 · Cited by 1
“…adversarial training; for example, Fleisig and Fellbaum [1] applied it to machine translation, Liu et al. [2] extended it to dialogue generation, and Park et al. [4] as well as Sharma et al. [5] demonstrated reductions in disparities for abusive language detection and natural language inference.”
Adversarial Discriminator to Mitigate Gender Bias in Abusive Language Detection
Park, J. S., & Cho, S.-B. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230473 · Cited by 1
Self-citationBang, J., Lee, B., & Park, P. (2023) · Journal of Broadcast Engineering · doi:10.5909/jbe.2023.28.5.545 · Cited by 1
Mitigating Societal Harms in Large Language Models
Kumar, S., Balachandran, V., Njoo, L., Anastasopoulos, A., & Tsvetkov, Y. (2023) · doi:10.18653/v1/2023.emnlp-tutorial.5 · Cited by 1
“…2017; Field and Tsvetkov, 2020 • Harm Detection - Research on Toxic text detection (Vaidya et al., 2020; Han and Tsvetkov, 2020), fact-checking (Zhou et al., 2021), hal-lucination detection (Kryscinski et al., 2020; Goyal and Durrett, 2020), bias-detection (Sun et al., 2019; Park et al., 2018).”
BackgroundMitigating Bias in Image Classification - Pretraining on Diverse Data for Improved Generalization
Shirvalkar, R., & Geetha, M. K. (2023) · International Conference on Electronics, Materials Engineering & Nano-Technology (IEMENTech) · doi:10.1109/iementech60402.2023.10423435 · Cited by 1
“According to the literature on bias and fairness, the data used to train the models can contain biases that affect their accuracy, especially for certain groups or categories [6].”
MethodEnronSR: A Benchmark for Evaluating AI-Generated Email Replies
Shay, M., Davidson, R., & Grinberg, N. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31448 · Cited by 3
“…et al. 2016; Brunet et al. 2019; Zhao et al. 2019; Basta, Costa-juss`a, and Casas 2019; Kaneko and Bollegala 2019), hate speech and abusive language detection (Park, Shin, and Fung 2018; Sap et al. 2019; Mozafari, Farahbakhsh, and Crespi 2020), and coreference resolution (Zhao et al. 2018).”
BackgroundHafner, F. S., Hafner, L., & Corizzo, R. (2025) · Journal of Computational Social Science · doi:10.1007/s42001-025-00382-y · Cited by 3
“In this way, explicit bias has been extensively proven to exist (e.g. [4, 5, 7, 8]), with the most comprehensive study based on over 200 SA tools, finding that most provide consistently higher sentiment intensity predictions for one race or one gender [6].”
An Analytical Approach for Recognizing the Occupational Gender Bias in the Data Systems
Rai, P., & Kumar, P. N. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-51859-2_59 · Cited by 1
On Gender Biases in Offensive Language Classification Models
Marcé, S., & Poliak, A. (2022) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.19 · Cited by 1
“While previous research has considered identity-based bias against a gendered subject in abusive language tasks (Park et al., 2018; Prabhakaran et al., 2019) and gender-based biases among annotations (Excell and Al Moubayed, 2021), how the perceived gender of a speaker or author affects output model classification rem…”
BackgroundInfluentialLee, L., Li, J., Ku, S.-W., & Tseng, Y. (2023) · Journal of Information Science · doi:10.1177/01655515231160041 · Cited by 1
“Offensive language use is a common form of abusive behaviour in online social media that can result in personal trauma, cyberbullying, hate crimes, and discrimination [33].”
Stahl, M., Spliethöver, M., & Wachsmuth, H. (2022) · Workshop on Natural Language Processing and Computational Social Science (NLP+CSS) · doi:10.18653/v1/2022.nlpcss-1.6 · Cited by 1
“Park et al. (2018) augmented data with gendered sentences to reduce bias in word embeddings, and Zmigrod et al. (2019) aimed to convert between masculine and feminine inflected sentences without parallel data.”
BackgroundFIND: Human-in-the-Loop Debugging Deep Text ClassifiersPreprint
Lertvittayakumjorn, P., Specia, L., & Toni, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.04987 · Cited by 13
Cross-geographic Bias Detection in Toxicity Modeling.Preprint
Ghosh, S., Baker, D., Jurgens, D., & Prabhakaran, V. (2021) · arXiv (Cornell University) · Cited by 1
Improving Counterfactual Generation for Fair Hate Speech DetectionPreprint
Davani, A. M., Omrani, A., Kennedy, B., Atari, M., Ren, X., & Dehghani, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.01721 · Cited by 2
“To mitigate this bias, data augmentation approaches are proposed to create balanced labels for each SGT or to prevent biases from propagating to the learned model (Dixon et al., 2018; Zhao et al., 2018; Park et al., 2018).”
MethodIndependent Ethical Assessment of Text Classification Models: A Hate Speech Detection Case StudyPreprint
Singh, A., Chen, J., Zhang, L., Rasekh, A., Golbin, I., & Rao, A. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.07627 · Cited by 2
“Quantitative models that focus on different ethical components have also been developed, such as the techniques for bias detection in embedding models by [7, 19, 25] and the metrics for evaluating bias in text classification datasets and models by [2, 5, 9, 20, 27].”
MethodContrastive Clustering: Toward Unsupervised Bias Reduction for Emotion and Sentiment ClassificationPreprint
Mowery, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.07448 · Cited by 1
“Studies may also examine multiple techniques, such as exploring debiased word embeddings, swapping gender-specific words in training data, and using larger corpora [17].”
BackgroundGuarín, D. L., Bandini, A., Dempster, A., Wang, H. E., Rezaei, S., Taati, B., & Yunusova, Y. (2021) · doi:10.36227/techrxiv.12950279.v2 · Cited by 1
“One approach to mitigate the bias in a pre-trained FA model is to fine-tune the model using representative data applying transfer learning techniques [43], [44].”
MethodUnderstanding Unconscious Bias by Large-scale Data Analysis
Tang, S. (2019) · eScholarship (California Digital Library)
Mitigating Gender Bias in Neural Machine Translation Using Counterfactual Data
Wong, A. C. ‐. (2020) · CUNY Academic Works (City University of New York)
Small Business Classification By Name: Addressing Gender and Geographic Origin BiasesPreprint
Shapiro, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.10348 · Cited by 1
“fine-tuning bias with a larger less biased dataset [16] or by other weight adjustment techniques [17], or removing gender-specific factors from the model [18].”
BackgroundAdversarial Scrubbing of Demographic Information for Text ClassificationPreprint
Chowdhury, S. B. R., Ghosh, S., Li, Y., Oliva, J. B., Srivastava, S., & Chaturvedi, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.08613 · Cited by 4
“Prior work has analysed bias in different NLP systems like machine translation (Park et al., 2018; Stanovsky et al., 2019; Font and Costa-Jussa, 2019; Saunders and Byrne, 2020), NLI (Rudinger et al., 2017), text classification (Dixon et al., 2018; Kiritchenko and Mohammad, 2018; Sap et al., 2019;…”
MethodGuarín, D. L., Bandini, A., Dempster, A., Wang, H. E., Rezaei, S., Taati, B., & Yunusova, Y. (2020) · doi:10.36227/techrxiv.12950279
Towards the Detection of Hateful Sentiment in Social Networks
González, S. G., González, A. B. G., & López, V. F. (2022) · Advances in intelligent systems and computing · doi:10.1007/978-3-031-14859-0_13
Exploring the Impact of Gender Bias Mitigation Approaches on a Downstream Classification Task
Sobhani, N., & Delany, S. J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16564-1_10 · Cited by 1
“Following work by [16] the templates, the identity term pairs and the nouns and adjectives we used are given in Table 1.”
InfluentialMitigating Multi-class Unintended Demographic Bias in Text Classification with Adversarial Learning
Pan, L., Yao, L., Zhang, W., & Wang, X. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20891-1_27
A Comparative Study of Gender Bias Mitigation Techniques and the Role of Implicit StereotypesPreprint
Sobhani, N., & Delany, S. J. (2023) · Research Square · doi:10.21203/rs.3.rs-3126256/v1 · Cited by 1
Gender Bias Propagation on Hate Speech: An Analysis at Feature-LevelPreprint
Nascimento, F. R. S., Cavalcanti, G. D. C., & Costa‐Abreu, M. D. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4517546
Mitigating Unintended Identity Bias in Toxicity Detection Language ModelPreprint
Saryazdi, A. H., & Sharifpour, K. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4561446
Beinborn, L., & Hollenstein, N. (2023) · Synthesis lectures on human language technologies · doi:10.1007/978-3-031-43260-6_6
Metadata
- Authors
- Park, J. H., Shin, J., & Fung, P.
- Affiliations
- Hong Kong University of Science and Technology (HK)
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 2799-2804
- OpenAlex
- W2888167352
- Cited by
- 323
- References
- 13
- Citations by year
- 2026: 1 · 2025: 23 · 2024: 26 · 2023: 56 · 2022: 53 · 2021: 75 · 2020: 53 · 2019: 34 · 2018: 2
- FWCI
- 16.75
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Hong Kong University of Science and Technology
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- offensive language detection, gender bias, model bias, bias mitigation, fine-tuning, imbalanced training data, word embeddings
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
