The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Reducing Gender Bias in Abusive Language Detection
Open accessAbstract
Abusive language detection models tend to have a problem of being biased toward identity words of a certain group of people because of imbalanced training datasets.For example, "You are a good woman" was considered "sexist" when trained on an existing dataset.Such model bias is an obstacle for models to be robust enough for practical use.In this work, we measure gender biases on models trained with different abusive language datasets, while analyzing the effect of different pre-trained word embeddings and model architectures.We also experiment with three bias mitigation methods: (1) debiased word embeddings, (2) gender swap data augmentation, and (3) fine-tuning with a larger corpus.These methods can effectively reduce gender bias by 90-98% and can be extended to correct model bias in other scenarios.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Prior work has shown that toxicity classifiers can pick up on biases in the training data and spuriously associate toxicity with the mention of certain demographics (Park et al., 2018; Dixon et al., 2018).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Park et al., 2018
In the citing paper
“models are trained with historical data and can present issues such as gender bias and racial bias in AI models [68,73], potentially exacerbating structural inequalities [10]”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [68]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
Talat, Z., & Hovy, D. (2016) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n16-2013 · Cited by 1,919 · cited 2×
“So far, many efforts were put into defining and constructing abusive language datasets from different sources and labeling them through crowd-sourcing or user moderation (Waseem and Hovy, 2016; Waseem, 2016; Founta et al., 2018; Wulczyn et al., 2017).”
BackgroundDeep Learning for Hate Speech Detection in Tweets
Badjatiya, P., Gupta, S., Gupta, M., & Varma, V. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3041021.3054223 · Cited by 1,253
Felbo, B., Mislove, A. E., Søgaard, A., Rahwan, I., & Lehmann, S. (2017) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 712
Gender bias in coreference resolution: Evaluation and debiasing methods
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2003 · Cited by 684 · cited 3×
“This simple method removes correlation between gender and classification decision and has proven to be effective for correcting gender biases in co-reference resolution task (Zhao et al., 2018).”
BackgroundMethodInfluentialMeasuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674 · cited 5×
“Therefore, we generate a separate unbiased test set for each gender, male and female, using the identity term template method proposed in Dixon et al. (2017).”
BackgroundMethodInfluentialEx Machina: Personal Attacks Seen at Scale
Wulczyn, E., Thain, N., & Dixon, L. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052591 · Cited by 636 · cited 1×
“So far, many efforts were put into defining and constructing abusive language datasets from different sources and labeling them through crowd-sourcing or user moderation (Waseem and Hovy, 2016; Waseem, 2016; Founta et al., 2018; Wulczyn et al., 2017).”
BackgroundLarge Scale Crowdsourcing and Characterization of Twitter Abusive Behavior
Founta, A.-M., Djouvas, C., Chatzakou, D., Ilias Leontiadis, A., Blackburn, J., Stringhini, G., Vakali, A. I., Sirivianos, M., & Kourtellis, N. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.14991 · Cited by 605
Flagged first, then by citations
Research works printed but not linked: 6
- Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Ad- vances in Neural Information Processing Systems, pages 4349-4357.
- Svetlana Kiritchenko and Saif M Mohammad. 2018. Examining gender and race bias in two hundred sen- timent analysis systems. Proceedings of the 7th Joint Conference on Lexical and Computational Se- mantics(*SEM), New Orleans, USA.
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013. Distributed representa- tions of words and phrases and their compositional- ity. In NIPS, pages 3111-3119.
- Ji Ho Park and Pascale Fung. 2017. One-step and two- step classification for abusive language detection on twitter. ALW1: 1st Workshop on Abusive Language Online to be held at the annual meeting of the Asso- ciation of Computational Linguistics (ACL) 2017.
- Zeerak Waseem. 2016. Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter. In Proceedings of the first workshop on NLP and computational social science, pages 138- 142.
- Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with ad- versarial learning. Proceedings of AAAI/ACM Con- ference on Ethics and Society(AIES) 2018.
Cited by
Sorted by citations
Items 101–200 of 322
Srinivasan, K. B., Danescu-Niculescu-Mizil, C., Lee, L., & Tan, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.09563 · Cited by 31
“to these questions surrounding tools for algorithmically-assisted moderation. Meanwhile, we caution against automatic content moderation given the inherent biases and potentially complex consequences [31,36,42]. Methodologicalimplications.Our study engages with methodological challenges in using observational data to…”
BackgroundData, Power and Bias in Artificial IntelligencePreprint
Leavy, S., O’Sullivan, B., & Σιαπέρα, Ε. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.07341 · Cited by 97
“, 2016] and by swapping the gender of entities in text [Zhao et al., 2018; Park et al., 2018].”
MethodMisogynoir: challenges in detecting intersectional hate
Kwarteng, J., Perfumi, S. C., Farrell, T., Third, A., & Fernandez, M. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00993-7 · Cited by 30
Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Framework
Halevy, M., Harris, C., Bruckman, A., Yang, D., & Howard, A. (2021) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3465416.3483299 · Cited by 36
“[24] attempts to reduce gender bias in the toxic-language”
When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization
Ladhak, F., Durmus, E., Suzgun, M., Zhang, T., Jurafsky, D., McKeown, K., & Hashimoto, T. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.234 · Cited by 30
“These approaches include designing data augmentation methods (Zhao et al., 2018a; Lee et al., 2017, 2018; Zhao et al., 2018b; Park et al., 2018), tagging training data with gender labels (Prates et al., 2018; Van-massenhove et al., 2018), debiasing word embed-dings (Bolukbasi et al., 2016; Zhao et…”
MethodIntersectional Bias in Hate Speech and Abusive Language DatasetsPreprint
Kim, J., Ortiz, C., Nam, S., Santiago, S., & Datta, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.05921 · Cited by 49
“Nevertheless, these studies examine this problem either from an exclusively racial (Waseem 2016; Waseem and Hovy 2016; Davidson, Bhattacharya, and Weber 2019; Sap et al. 2019) or genderbias perspective (Tatman 2017; Park, Shin, and Fung 2018; Dixon et al. 2018).”
BackgroundRuddit: Norms of Offensiveness for English Reddit Comments
Hada, R., Sudhir, S., Mishra, P., Yannakoudakis, H., Mohammad, S. M., & Shutova, E. (2021) · doi:10.18653/v1/2021.acl-long.210 · Cited by 30
“Identity terms A common criticism of the existing offensive language datasets is that in those datasets, certain identity terms (particularly those referring to minority groups) occur mainly in texts that are offensive (Sap et al., 2019; Davidson et al., 2019; Wiegand et al., 2019; Park et al., 2018; Dixon et al., 201…”
BackgroundThe (de)biasing Effect of GAN-Based Augmentation Methods on Skin Lesion Images
Mikołajczyk, A., Majchrowska, S., & Limeros, S. C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16452-1_42 · Cited by 27
“Some works on measuring bias suggested simply comparing the performance metrics on biased and unbiased dataset [20].”
BackgroundAyo, F. E., Folorunso, O. O., Ibharalu, F. T., & Osinuga, I. A. (2020) · International Journal of Intelligent Computing and Cybernetics · doi:10.1108/ijicc-06-2020-0061 · Cited by 30
Detecting Cross-Geographic Biases in Toxicity Modeling on Social Media
Ghosh, S., Baker, D., Jurgens, D., & Prabhakaran, V. (2021) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/2021.wnut-1.35 · Cited by 35
“…recent research has shown that these models often encode various societal biases against marginalized groups (Waseem, 2016; Caliskan et al., 2017; Park et al., 2018; van Aken et al., 2018; Dixon et al., 2018; Sap et al., 2019; Davidson et al., 2019; Hutchinson et al., 2020; Zhou et al., 2021)…”
BackgroundAn information theoretic approach to reducing algorithmic bias for machine learning
Kim, J.-Y., & Cho, S. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.09.081 · Cited by 31
The Authors Matter: Understanding and Mitigating Implicit Bias in Deep Text Classification
Liu, H., Jin, W., Karimi, H., Liu, Z., & Tang, J. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.7 · Cited by 33
Measure Utility, Gain Trust: Practical Advice for XAI Researchers
Davis, B., Glenski, M., Sealy, W. I. N., & Arendt, D. (2020) · IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX) · doi:10.1109/trex51495.2020.00005 · Cited by 31
“These domains include predictive policing [17], recidivism prediction [12,15], and hate speech and abusive language identification online [13,36,40].”
BackgroundNecessity and Sufficiency for Explaining Text Classifiers: A Case Study in Hate Speech Detection
Balkır, E., Nejadgholi, I., Fraser, K., & Kiritchenko, S. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.192 · Cited by 30
Recognizing misogynous memes: Biased models and tricky archetypes
Rizzi, G., Gasparini, F., Saibene, A., Rosso, P., & Fersini, E. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103474 · Cited by 30
Comparative Evaluation of Label-Agnostic Selection Bias in Multilingual Hate Speech Datasets
Ousidhoum, N. D., Song, Y., & Yeung, D. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.199 · Cited by 26
“…classi-fication performance and investigate how to debias the detection given false positives caused by gender group identity words such as “women” (Park et al., 2018), racial terms reclaimed by communities in certain contexts (Davidson et al., 2019), or names of groups that belong to the…”
BackgroundMethodInfluentialChiril, P., Moriceau, V., Benamara, F., Mari, A., Origgi, G., & Coulomb‐Gully, M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.373 · Cited by 26
Investigating Societal Biases in a Poetry Composition SystemPreprint
Sheng, E., & Uthus, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.02686 · Cited by 26
“, 2017), abusive language detection (Park et al., 2018), sentiment analysis (Shen et al.”
BackgroundMethodEvaluating Gender Bias in Natural Language InferencePreprint
Sharma, S., Dey, M., & Sinha, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.05541 · Cited by 36
“…many subtasks of NLP, ranging from learned word embeddings (Bolukbasi et al., 2016; Brunet et al., 2019), natural language inference (He et al., 2019a), hate speech detection (Park et al., 2018), dialog (Henderson et al., 2018; Dinan et al., 2019), and coreference resolution (Zhao et al., 2018b).”
BackgroundFuzzE: Fuzzy Fairness Evaluation of Offensive Language Classifiers on African-American English
Rios, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5434 · Cited by 23
Measuring and mitigating language model biases in abusive language detection
Song, R., Giunchiglia, F., Li, Y., Shi, L., & Xu, H. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103277 · Cited by 23
Features or Spurious Artifacts? Data-centric Baselines for Fair and Robust Hate Speech Detection
Ramponi, A., & Tonelli, S. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.221 · Cited by 22
“…the task has been addressed through a number of approaches, e.g., via adversarial feature learning (Vaidya et al., 2020), by using debiased word embeddings and gender swap data augmentation (Park et al., 2018) or by adding non-toxic examples to better balance the data (Dixon et al., 2018).”
BackgroundUtilizing subjectivity level to mitigate identity term bias in toxic comments classification
Zhao, Z., Zhang, Z., & Hopfgartner, F. (2022) · Online Social Networks and Media · doi:10.1016/j.osnem.2022.100205 · Cited by 26
Examining and mitigating gender bias in text emotion detection task
Odbal, O., Zhang, G., & Ananiadou, S. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.04.057 · Cited by 21
Gender-based multi-aspect sentiment detection using multilabel learning
Kumar, J. A., Trueman, T. E., & Cambria, E. (2022) · Information Sciences · doi:10.1016/j.ins.2022.05.057 · Cited by 21
“Related works The gender-based model has attracted researchers in the fields of natural language processing (NLP), linguistics [20, 30], psychology [66], image processing [67], abusive language detection [50], e-commerce [80], science and engineering education [11, 72, 2, 68], and healthcare [51, 24, 23].”
BackgroundBalancing out Bias: Achieving Fairness Through Balanced Training
Han, X., Baldwin, T., & Cohn, T. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.779 · Cited by 50
“…learn spurious correlations with other demographics and socio-economic factors (Hen-dricks et al., 2018; Lu et al., 2018; Bolukbasi et al., 2016; Park et al., 2018), leading to disparities across author demographics in contexts including coreference resolution, sentiment analysis, and hate…”
BackgroundDoes Gender Matter in the News? Detecting and Examining Gender Bias in News Articles
Dacon, J., & Liu, H. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442442.3452325 · Cited by 20
“Existingworks attempt to address the issue of fairness in various NLP tasks such as text classification [5, 32, 39], word embedding [2, 15, 43], coreference resolution [35, 42], language modeling [4], machine translation [13], semantic role labeling [41], dialogue generation [23, 24], etc.”
BackgroundSelective Classification Can Magnify Disparities Across GroupsPreprint
Jones, E., Sagawa, S., Koh, P. W., Kumar, A., & Liang, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.14134 · Cited by 34
“In toxicity comment detection, models have been shown to latch onto spurious correlations between the toxicity and mention of certain demographic groups (Park et al., 2018; Dixon et al., 2018).”
BackgroundInvestigating Sampling Bias in Abusive Language Detection
Razo, D., & Kübler, S. (2020) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2020.alw-1.9 · Cited by 19
“Park et al. (2018) discuss methods to decrease the gender bias in abusive language detection.”
MethodMitigating Biases in Toxic Language Detection through Invariant Rationalization
Chuang, Y.-S., Gao, M., Luo, H., Glass, J. R., Lee, H.-Y., Chen, Y.-N., & Li, S.-W. D. (2021) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2021.woah-1.12 · Cited by 24
Chiril, P., Benamara, F., & Moriceau, V. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.242 · Cited by 21
“der bias in word embeddings or word association graphs (Bolukbasi et al., 2016; Park et al., 2018; Madaan et al., 2018; Dev and Phillips, 2019; Du et al., 2019) as well as to identify disparity across gender in various applications like co-reference”
BackgroundMethodChoose Your Lenses: Flaws in Gender Bias Evaluation
Orgad, H., & Belinkov, Y. (2022) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.17 · Cited by 24
“…et al. (2019); Qian et al. (2019) Emami et al. (2019); Habash et al. (2019) Dinan et al. (2020); Costa-jussà and de Jorge (2020) Basta et al. (2020) 2 Park et al. (2018); Stafanovičs et al. (2020) Saunders and Byrne (2020); Saunders et al. (2020) Kaneko and Bollegala (2021a); Jin et al. (2021) 3”
Kaur, S., Singh, S., & Kaushal, S. (2024) · International Journal of Cognitive Computing in Engineering · doi:10.1016/j.ijcce.2024.02.002 · Cited by 19
Measuring and Mitigating Name Biases in Neural Machine Translation
Wang, J., Rubinstein, B. I. P., & Cohn, T. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.184 · Cited by 18
“Gender bias is common in automated systems (Park et al., 2018; Borkan et al., 2019; Stanovsky et al., 2019; Saunders and Byrne, 2020), with a leading cause being training corpora that include far more sentences referring to men than to women.”
BackgroundPersian offensive language detection
Kebriaei, E., Homayouni, A., Faraji, R., Razavi, A., Shakery, A., Faili, H., & Yaghoobzadeh, Y. (2023) · Machine Learning · doi:10.1007/s10994-023-06370-5 · Cited by 18
“Due to the annotation complexity and the keyword-based sampling, most of the datasets are generally biased towards specific keywords (Madukwe et al. 2020; Park et al. 2018; Dixon et al. 2018).”
Automatic Misogyny Detection in Social Media Platforms using Attention-based Bidirectional-LSTM
Rahali, A., Akhloufi, M. A., Therien-Daniel, A.-M., & Brassard-Gourdeau, É. (2021) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc52423.2021.9659158 · Cited by 16
“A very interesting approach that we used on the EN-EVALITA challenge is Gender Swap (GS) augmentation [24], which augments the training data by identifying male entities and swapping them with equivalent female entities and vice-versa.”
MethodA GENDERED PERSPECTIVE ON ARTIFICIAL INTELLIGENCE
Parsheera, S. (2018) · ITU Kaleidoscope Academic Conference (ITU K) · doi:10.23919/itu-wt.2018.8597618 · Cited by 16
Hartmann, D., Oueslati, A., Staufer, D., Pohlmann, L., Munzert, S., & Heuer, H. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713998 · Cited by 29
“For instance, extensive research was conducted to mitigate biases against specific groups [19, 76, 78, 89].”
Bias and comparison framework for abusive language datasets
Wich, M., Eder, T., Kuwatly, H. A., & Groh, G. (2021) · AI and Ethics · doi:10.1007/s43681-021-00081-0 · Cited by 16
“[30] evaluated gender bias on the [44] and [16] datasets.”
BackgroundAnoop, K. R., Gangan, M. P., Deepak, P., & Lajish, V. L. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-4453-6_2 · Cited by 17
“Sentiment analysis [52], abusive language detection [81], text clas-sification [33], machine translation [38,112], personalized medicine [90], corefer-Bias ence resolution [95,128], crime recidivism prediction systems [27], automating resume screening [56], online advertisements delivery [57,109],…”
BackgroundMethodInfluentialDetecting Unintended Social Bias in Toxic Language Datasets
Sahoo, N., Gupta, H., & Bhattacharyya, P. (2022) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2022.conll-1.10 · Cited by 25
Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future
Yang, L., Song, Y., Ren, X., Lyu, C., Wang, Y., Zhuo, J., Liu, L., Wang, J., Foster, J., & Zhang, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.276 · Cited by 21
Sandu, A., Cotfas, L., Delcea, C., Ioanăş, C., Florescu, M., & Orzan, M. (2024) · Electronics · doi:10.3390/electronics13224352 · Cited by 18
Ashktorab, Z., Hoover, B., Agarwal, M., Dugan, C., Geyer, W., Yang, H., & Yurochkin, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581227 · Cited by 21
“In the Group View task as well as our exploration task, we use group identifier keywords as a proxy for text content, which is a common way to find groups in text classification [20, 34, 48, 58].”
MethodGendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019
Webster, K., Costa‐jussà, M. R., Hardmeier, C., & Radford, W. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3801 · Cited by 21
Pruksachatkun, Y., Krishna, S., Dhamala, J., Gupta, R., & Chang, K.-W. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.294 · Cited by 21
On Measuring Social Biases in Prompt-Based Multi-Task Learning
Akyürek, A. F., Paik, S., Kocyigit, M. Y., Akbıyık, S., Runyun, Ş. L., & Wijaya, D. T. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.42 · Cited by 21
Debiasing pretrained text encoders by paying attention to paying attention
Gaci, Y., Benatallah, B., Casati, F., & Benabdeslem, K. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.651 · Cited by 14
“Admittedly, common social biases have also been shown to exist in hate speech detection models, for example in associating toxicity to frequently attacked groups (such as "muslim" or "gay") even if the text itself is not toxic (Dixon et al., 2018; Park et al., 2018).”
Muralikumar, M. D., Yang, Y. S., & McDonald, D. W. (2023) · ACM Transactions on Social Computing · doi:10.1145/3582568 · Cited by 14
“…hoc evaluations on how AI/ML models perform across different racial, cultural, demographic, and/or intersectional groups (e.g., References [36, 37, 56]) and developing more nuanced metrics to measure unintended biases [29] are important efforts in support of model fairness and transparency [54].”
BackgroundGender Bias in Pretrained Swedish Embeddings
Sahlgren, M., & Olsson, F. (2019) · DSpace repository (University of Tartu) · Cited by 15
Deep Learning on a Healthy Data Diet: Finding Important Examples for Fairness
Zayed, A., Parthasarathi, P., Mordido, G., Palangi, H., Shabanian, S., & Chandar, S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26706 · Cited by 19
Quantifying 60 Years of Gender Bias in Biomedical Research with Word Embeddings
Rios, A., Joshi, R., & Shin, H. (2020) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2020.bionlp-1.1 · Cited by 15
“Bias has been found in word embeddings (Boluk-basi et al., 2016; Zhao et al., 2018, 2019), text clas-sification models (Dixon et al., 2018; Park et al., 2018; Badjatiya et al., 2019; Rios, 2020), and in machine translation systems (Font and Costa-juss`a, 2019; Escud´e Font, 2019).”
BackgroundGeBioToolkit: Automatic Extraction of Gender-Balanced Multilingual Corpus of Wikipedia BiographiesPreprint
Costa‐jussà, M. R., Lin, P. L., & España-Bonet, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.04778 · Cited by 13
“Recent research is being devoted to correct the asymmetries mainly by data augmentation techniques in fields such as coreference resolution (Rudinger et al., 2018; Zhao et al., 2018; Webster et al., 2018) or abusive language detection (Park et al., 2018).”
CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models
Zhao, J., Fang, M., Shi, Z., Li, Y., Chen, L., & Pechenizkiy, M. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.757 · Cited by 14
Improving Gender-Related Fairness in Sentence Encoders: A Semantics-Based Approach
Dolci, T., Azzalini, F., & Tanelli, M. (2023) · Data Science and Engineering · doi:10.1007/s41019-023-00211-0 · Cited by 12
“The social bias in the geometry of the model is then of course reflected in downstream applications like web search, CV parsing or hate speech detection [3, 7, 43].”
An Empirical Investigation of Bias in the Multimodal Analysis of Financial Earnings Calls
Sawhney, R., Aggarwal, A., & Shah, R. R. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.294 · Cited by 12
“In terms of natural language representation, embeddings such as word2vec and GloVe, trained on news articles may inherit gender stereotypes (Packer et al., 2018; Bolukbasi et al., 2016; Park et al., 2018).”
BackgroundSocietal Biases in Language Generation: Progress and ChallengesPreprint
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.04054 · Cited by 12
Cyberbully Detection Using BERT with Augmented Texts
Guo, X., Anjum, U., & Zhan, J. (2022) · 2022 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata55660.2022.10020581 · Cited by 12
“[25] to reduce gender bias and [28] to improve detection of denigrating speech in cyberbullying.”
BackgroundWhich Stereotypes Are Moderated and Under-Moderated in Search Engine Autocompletion?
Leidinger, A., & Rogers, R. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594062 · Cited by 12
“Methodologically, bias has been quantified using intrinsic measures [9, 11, 66] which operate on word embeddings or extrinsic measures that examine how bias manifests itself in downstream tasks such as sentiment analysis [34, 37] or hate speech detection [19, 52].”
BackgroundExtracting or Guessing? Improving Faithfulness of Event Temporal Relation Extraction
Wang, H., Zhang, H., Deng, Y., Gardner, J. R., Roth, D., & Chen, M. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.39 · Cited by 12
Dual-Teacher De-Biasing Distillation Framework for Multi-Domain Fake News Detection
Li, J., Feng, X., Gu, T., & Chang, L. (2024) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00279 · Cited by 12
Mitigation of Diachronic Bias in Fake News Detection Dataset
Murayama, T., Wakamiya, S., & Aramaki, E. (2021) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/2021.wnut-1.21 · Cited by 15
Removing AI’s sentiment manipulation of personalized news delivery
Wu, C., Wu, F., Qi, T., Zhang, W.-Q., Xie, X., & Huang, Y. (2022) · Humanities and Social Sciences Communications · doi:10.1057/s41599-022-01473-1 · Cited by 12
Social norm bias: residual harms of fairness-aware algorithms
Cheng, M., De‐Arteaga, M., Mackey, L. W., & Kalai, A. T. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00910-8 · Cited by 15
“Various fairness-aware algorithms aim to mitigate these issues from a group fairness perspective (Park et al. 2018; Kumar et al. 2020; Bordia and Bowman 2019; Gonen and Goldberg 2019).”
BackgroundGender bias detection on hate speech classification: an analysis at feature-level
Nascimento, F. R. S., Cavalcanti, G. D. C., & Costa‐Abreu, M. D. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10841-8 · Cited by 13
“Regarding the unintended bias evaluation, we investigated different metrics widely used in the literature [9, 14].”
InfluentialFan, A., & Gardent, C. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.586 · Cited by 21
Can We Improve Model Robustness through Secondary Attribute Counterfactuals?
Balashankar, A., Wang, X., Packer, B., Thain, N., Ed, H., & Beutel, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.386 · Cited by 11
“…generation approaches have been built on, as well as coupled with regularization, to improve counterfactual fairness (Kusner et al., 2017), such as preventing models from being overly sensitive to identity terms (Garg et al., 2018; Prabhakaran et al., 2019; Kurita et al., 2019; Park et al., 2018).”
BackgroundAlGhafa Evaluation Benchmark for Arabic Language Models
Almazrouei, E., Cojocaru, R., Baldo, M., Malartic, Q., Alobeidli, H., Mazzotta, D., Penedo, G., Campesan, G., Farooq, M., Alhammadi, M., et al. (2023) · Arabic Natural Language Processing Conference (ArabicNLP) · doi:10.18653/v1/2023.arabicnlp-1.21 · Cited by 17
“Since toxic language classifiers can exhibit a built-in bias toward specific terms including the names of certain social groups (Sap et al., 2019), (Park et al., 2018), (Hutchinson et al., 2020), the toxicity probing is performed in two steps.”
BackgroundLatent Space Smoothing for Individually Fair Representations
Peychev, M., Ruoss, A., Balunović, M., Baader, M., & Vechev, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19778-9_31 · Cited by 11
Enhancing the fairness of offensive memes detection models by mitigating unintended political bias
Kumari, G., Sinha, A., Ekbal, A., Chatterjee, A. R., & Vinutha, B. N. (2024) · Journal of Intelligent Information Systems · doi:10.1007/s10844-023-00834-9 · Cited by 9
“Utilizing these models without adequately considering the social context surrounding meme creation and dissemination poses a substantial risk of exacerbating biases, promoting exclusionary practices, and further marginalizing certain social groups (Park et al., 2018; Davidson et al., 2019).”
InfluentialDialect Diversity in Text Summarization on Twitter
Keswani, V., & Celis, L. E. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450108 · Cited by 13
“Yet, automated language tools are often unable to handle the dialect diversity in Twitter, leading to issues like disparate accuracy of language identification between posts written in African-American English (AAE) and standard English [6], or dialect-based discrepancies in abusive speech detection [56, 61].”
BackgroundUse of Artificial Intelligence in Regulatory Decision-Making
Jago, R., van der Gaag, A., Stathis, K., Petej, I., Lertvittayakumjorn, P., Krishnamurthy, Y., Gao, Y., Silva, J. C., Webster, M., Gallagher, A., et al. (2021) · Journal of Nursing Regulation · doi:10.1016/s2155-8256(21)00112-5 · Cited by 11
A study of bias mitigation strategies for speaker recognition
Peri, R., Somandepalli, K., & Narayanan, S. S. (2022) · Computer Speech & Language · doi:10.1016/j.csl.2022.101481 · Cited by 17
Rating Sentiment Analysis Systems for Bias Through a Causal Lens
Lakkaraju, K., Srivastava, B., & Valtorta, M. (2024) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2024.3375519 · Cited by 9
“Additionally, in [31], the authors aim to reduce gender bias in abusive language detection models without explicitly measuring the bias in the models.”
BackgroundGuarín, D. L., Bandini, A., Dempster, A., Wang, H. E., Rezaei, S., Taati, B., & Yunusova, Y. (2020) · doi:10.36227/techrxiv.12950279.v1 · Cited by 8
“the lack of representative data in the databases used to train FA models [29], [37], [40]–[42] One approach to mitigate the bias in a pre-trained FA model is to fine-tune the model using representative data applying transfer learning techniques [43], [44].”
MethodTowards Ethics by Design in Online Abusive Content DetectionPreprint
Kiritchenko, S., & Nejadgholi, I. (2020) · arXiv (Cornell University) · doi:10.4224/40002027 · Cited by 15
Balancing out Bias: Achieving Fairness Through Training Reweighting.Preprint
Han, X., Baldwin, T., & Cohn, T. (2021) · arXiv (Cornell University) · Cited by 9
Antonopoulou, C. (2023) · Technoetic Arts · doi:10.1386/tear_00109_1 · Cited by 9
Improving Counterfactual Generation for Fair Hate Speech Detection
Davani, A. M., Omrani, A., Kennedy, B., Atari, M., Ren, X., & Dehghani, M. (2021) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2021.woah-1.10 · Cited by 9
Beyond plain toxic: building datasets for detection of flammable topics and inappropriate statements
Babakov, N., Logacheva, V., & Panchenko, A. (2023) · Language Resources and Evaluation · doi:10.1007/s10579-023-09682-z · Cited by 16
“Xia et al. (2020) train their model to detect two objectives: toxicity and presence of the toxicity-provoking topic, (Zhang et al., 2020) perform re-weighing of instances, (Park et al., 2018) create pseudo-data to level off the number of instances of different classes.”
Adversarial Scrubbing of Demographic Information for Text Classification
Chowdhury, S. B. R., Ghosh, S., Li, Y., Oliva, J. B., Srivastava, S., & Chaturvedi, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.43 · Cited by 13
“Prior work has analysed bias in different NLP systems like machine translation (Park et al., 2018; Stanovsky et al., 2019; Font and Costa-Jussa, 2019; Saunders and Byrne, 2020), NLI (Rudinger et al., 2017), text classification (Dixon et al., 2018; Kiritchenko and Mohammad, 2018; Sap et al., 2019;…”
MethodEasy Adaptation to Mitigate Gender Bias in Multilingual Text Classification
Huang, X. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.52 · Cited by 8
Bias Mitigation for Toxicity Detection via Sequential Decisions
Cheng, L., Mosallanezhad, A., Silva, Y. N., Hall, D. L., & Liu, H. (2022) · Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3477495.3531945 · Cited by 8
Controlling Bias Exposure for Fair Interpretable Predictions
He, Z., Wang, Y., McAuley, J., & Majumder, B. P. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.431 · Cited by 11
Finding good representations of emotions for text classification
Park, J. H. (2018) · doi:10.14711/thesis-991012636967803412 · Cited by 10
Self-citationFrom physical space to cyberspace: Recessive gender biases in social media mirror the real world
Shen, K., Ding, L., Kong, L., & Liu, X. (2024) · Cities · doi:10.1016/j.cities.2024.105149 · Cited by 7
Symbolic signals on Instagram: how visual media shapes engagement, emotion, trust, and diffusion
Gurung, M. I., Agarwal, N., Bhuiyan, M. M. I., & Poudel, D. (2025) · Social Network Analysis and Mining · doi:10.1007/s13278-025-01469-0 · Cited by 21
“These variations were likely due to the distinct training datasets and methodologies employed by each model (Park et al. 2018).”
Gupta, S. S., Lee, S., De‐Arteaga, M., & Lease, M. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583290 · Cited by 10
“For this reason, a “one-size-fits-all” modeling approach may yield sub-optimal performance and more specifically raise concerns of algorithmic fairness (Arango et al., 2019; Park et al., 2018; Sap et al., 2019).”
Socially Responsible Hate Speech Detection: Can Classifiers Reflect Social Stereotypes?
Vargas, F., Carvalho, I., Hürriyetoğlu, A., KNAW Humanities Cluster, D., Pardo, T. A. S., & Benevenuto, F. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_126 · Cited by 8
How Far Can It Go? On Intrinsic Gender Bias Mitigation for Text Classification
Tokpo, E. K., Delobelle, P., Berendt, B., & Calders, T. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.248 · Cited by 9
Measuring Model Fairness under Noisy Covariates: A Theoretical Perspective
Prost, F., Awasthi, P., Blumm, N., Kumthekar, A., Potter, T., Li, W., Wang, X., Ed, H., Chen, J., & Beutel, A. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462603 · Cited by 7
“For the sake of clarity, let’s return to our example from above of a toxicity model for online comments, which predicts whether a comment contains toxic language (y = 1 if toxic) and for which prior works have demonstrated that these models often exhibit biases towards certain demographics [11, 19, 30, 31].”
Online Abuse of UK MPs from 2015 to 2019: Working PaperPreprint
Greenwood, M., Bakir, M. E., Gorrell, G., Song, X., Roberts, I. G., & Bontcheva, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.11230 · Cited by 11
Towards Debiasing Sentence RepresentationsPreprint
Liang, P. P., Li, I., Zheng, E., Lim, Y. C., Salakhutdinov, R., & Morency, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08100 · Cited by 65
“In particular, Zhao et al. (2019), Park et al. (2018), and Garg et al. (2019) are not able to perform post-hoc debiasing and require changing the data or underlying word embeddings and retraining which is costly.”
BackgroundSorato, D., Lundsteen, M., Ventura, C. C., & Zavala‐Rojas, D. (2024) · Journal of Computational Social Science · doi:10.1007/s42001-023-00243-6 · Cited by 8
Race and Religion in Online Abuse towards UK Politicians: Working Paper
Gorrell, G., Bakir, M. E., Greenwood, M., Roberts, I. G., & Bontcheva, K. (2019) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.48550/arxiv.1910.00920 · Cited by 12
“05), a result that echoes Pew (2017). Ethnic minority MPs received significantly more racist abuse (%, σ= vs.”
BackgroundOriola, O., & Kotzé, E. (2020) · South African Computer Journal · doi:10.18489/sacj.v32i2.847 · Cited by 6
“…on the automatic detection of abusive language, such as hate speech (Yuan et al., 2016; Zimmerman et al., 2018), cyberbullying (Kargutkar & Chitre, 2020), racist (Greevy & Smeaton, 2004), sexist (Park et al., 2018), and toxic contents (van Aken et al., 2018), using machine learning techniques.”
MethodLearning to Generate Equitable Text in Dialogue from Biased Training Data
Sicilia, A., & Alikhani, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.163 · Cited by 6
Metadata
- Authors
- Park, J. H., Shin, J., & Fung, P.
- Affiliations
- Hong Kong University of Science and Technology (HK)
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 2799-2804
- OpenAlex
- W2888167352
- Cited by
- 323
- References
- 13
- Citations by year
- 2026: 1 · 2025: 23 · 2024: 26 · 2023: 56 · 2022: 53 · 2021: 75 · 2020: 53 · 2019: 34 · 2018: 2
- FWCI
- 16.75
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Hong Kong University of Science and Technology
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- offensive language detection, gender bias, model bias, bias mitigation, fine-tuning, imbalanced training data, word embeddings
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
