Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics
Open accessThe DOI matches its Crossref record; 83 of the 84 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Measuring bias is key for better understanding and addressing unfairness in NLP/ML models. This is often done via fairness metrics, which quantify the differences in a model’s behaviour across a range of demographic groups. In this work, we shed more light on the differences and similarities between the fairness metrics used in NLP. First, we unify a broad range of existing metrics under three generalized fairness metrics, revealing the connections between them. Next, we carry out an extensive empirical comparison of existing metrics and demonstrate that the observed differences in bias measurement can be systematically explained via differences in parameter choices for our generalized metrics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Most of the existing bias evaluation metrics have been for specific categories of bias, such as gender bias (Czarnowska et al. 2021).”
From Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models (2024) · cites this paper as Czarnowska et al. 2021
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other
Mann, H. B., & Whitney, D. R. (1947) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177730491 · Cited by 14,320 · cited 1×
“(2019) define their comparison function using the Mann-Whitney U test statistic (Mann and Whitney, 1947).”
MethodConditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878 · cited 2×
“Models We experiment on three RoBERTa (Liu et al., 2019) based models: 6 (i) a binary classifier 6 Our preliminary experiments also used models based on Electra (Clark et al., 2020) as well as those trained on SST-2 and SST-3 datasets (Socher et al., 2013).”
MethodA Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812 · cited 1×
“Surveys of Mehrabi et al. (2019), Hutchinson and Mitchell (2019) and Chouldechova and Roth (2020) cover a broad scope of literature on algorithmic fairness.”
BackgroundA Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019) · arXiv · doi:10.48550/arxiv.1908.09635 · Cited by 5,812
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · cited 2×
“A related line of research measures bias present in sentence or word representations (Bolukbasi et al., 2016; Caliskan et al., 2017; Kurita et al., 2019; Sedoc and Ungar, 2019; Chaloner and Maldonado, 2019; Dev and Phillips, 2019; Gonen and Goldberg, 2019; Hall Maudslay et al., 2019; Liang et al., 2020; Shin et al., 2…”
BackgroundIntroduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition
Tjong Kim Sang, E. F., & De Meulder, F. (2003) · Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003 - · doi:10.3115/1119176.1119195 · Cited by 2,788
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 2×
“The original versions target demographic parity (Dwork et al., 2012), while the TC versions target equality of opportunity (Hardt et al., 2016) and can pinpoint the existence of bias more precisely, as we show later (Section 6).”
BackgroundMan is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 1×
“A related line of research measures bias present in sentence or word representations (Bolukbasi et al., 2016; Caliskan et al., 2017; Kurita et al., 2019; Sedoc and Ungar, 2019; Chaloner and Maldon-ado, 2019; Dev and Phillips, 2019; Gonen and Goldberg, 2019; Hall Maudslay et al., 2019; Liang et al.,…”
BackgroundDesign challenges and misconceptions in named entity recognition
Ratinov, L., & Roth, D. (2009) · Proceedings of the Thirteenth Conference on Computational Natural Language Learning - CoNLL '09 · doi:10.3115/1596374.1596399 · Cited by 1,448 · cited 1×
“In NER experiments we use the BILOU labeling scheme (Ratinov and Roth, 2009) and, for the probability-based metrics, we use the probabilities from the encoder’s output.”
MethodLanguage (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050 · cited 3×
“We arrive at this list by reviewing 146 papers that study bias from the survey of Blodgett et al. (2020) and selecting metrics that meet three criteria: (i) the metric is extrinsic; i.e., it is applied to at least one downstream NLP task, 4 (ii) it quan-tifies the difference in performance across…”
BackgroundKusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995 · cited 1×
“…fairness requires parity for two or more versions of an individual, one from the actual world and others from counterfactual worlds in which the individual belongs to a different protected group ; i.e., it requires invariance to the change of the protected group (Kusner et al., 2017).”
BackgroundBeyond accuracy: behavioral testing of NLP models with CheckList
Ribeiro, M. T., Wu, T., Guestrin, C., & Singh, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.442 · Cited by 890
The Risk of Racial Bias in Hate Speech Detection
Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1163 · Cited by 817 · cited 1×
“…protected group or for each split of a challenge dataset (Hovy and Søgaard, 2015; Rudinger et al., 2018; Zhao et al., 2018; Garimella et al., 2019; Sap et al., 2019; Bagdasaryan et al., 2019; Stafanoviˇcs et al., 2020; Tan et al., 2020; Mehrabi et al., 2020; Nadeem et al., 2020; Cao and Daumé…”
BackgroundSemEval-2018 Task 1: Affect in Tweets
Mohammad, S., Bravo-Marquez, F., Salameh, M., & Kiritchenko, S. (2018) · Proceedings of The 12th International Workshop on Semantic Evaluation · doi:10.18653/v1/s18-1001 · Cited by 746 · cited 3×
“We ex-periment on synthetic data created using hand-crafted templates, as is common in the literature (Dixon et al., 2018; Kiritchenko and Mohammad, 2018; Kurita et al., 2019; Huang et al., 2020a).”
BackgroundMethodGender bias in coreference resolution: Evaluation and debiasing methods
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2003 · Cited by 684 · cited 2×
“…elaborate measure independently for each protected group or for each split of a challenge dataset (Hovy and Søgaard, 2015; Rudinger et al., 2018; Zhao et al., 2018; Garimella et al., 2019; Sap et al., 2019; Bagdasaryan et al., 2019; Stafanoviˇcs et al., 2020; Tan et al., 2020; Mehrabi et al.,…”
BackgroundMeasuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674 · cited 7×
“We ex-periment on synthetic data created using hand-crafted templates, as is common in the literature (Dixon et al., 2018; Kiritchenko and Mohammad, 2018; Kurita et al., 2019; Huang et al., 2020a).”
BackgroundMethodInfluentialStereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.416 · Cited by 643 · cited 1×
“…group or for each split of a challenge dataset (Hovy and Søgaard, 2015; Rudinger et al., 2018; Zhao et al., 2018; Garimella et al., 2019; Sap et al., 2019; Bagdasaryan et al., 2019; Stafanovičs et al., 2020; Tan et al., 2020; Mehrabi et al., 2020; Nadeem et al., 2020; Cao and Daumé III, 2020).”
BackgroundStereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2020) · arXiv · doi:10.48550/arxiv.2004.09456 · Cited by 643
Mitigating Gender Bias in Natural Language Processing: Literature Review
Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W., & Wang, W. Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1159 · Cited by 492 · cited 1×
“Sun et al. (2019) provide a comprehensive overview of addressing gender bias in NLP.”
BackgroundGonen, H., & Goldberg, Y. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1061 · Cited by 454 · cited 1×
“A related line of research measures bias present in sentence or word representations (Bolukbasi et al., 2016; Caliskan et al., 2017; Kurita et al., 2019; Sedoc and Ungar, 2019; Chaloner and Maldonado, 2019; Dev and Phillips, 2019; Gonen and Goldberg, 2019; Hall Maudslay et al., 2019; Liang et al., 2020; Shin et al., 2…”
BackgroundCrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Nangia, N., Vania, C., Bhalerao, R., & Bowman, S. R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.154 · Cited by 446
Gender Bias in Coreference Resolution
Rudinger, R., Naradowsky, J., Leonard, B., & Van Durme, B. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2002 · Cited by 443 · cited 2×
“…F1 or accuracy, or a more elaborate measure independently for each protected group or for each split of a challenge dataset (Hovy and Søgaard, 2015; Rudinger et al., 2018; Zhao et al., 2018; Garimella et al., 2019; Sap et al., 2019; Bagdasaryan et al., 2019; Stafanoviˇcs et al., 2020; Tan et al.,…”
BackgroundRacial Bias in Hate Speech and Abusive Language Detection Datasets
Davidson, T., Bhattacharya, D., & Weber, I. (2019) · Proceedings of the Third Workshop on Abusive Language Online · doi:10.18653/v1/w19-3504 · Cited by 414 · cited 1×
“Just like fairness metrics, statistical significance tests can also detect the presence of systematic differences in the behavior of a model, and hence are often employed as alternative means to quantify bias (Mohammad et al., 2018; Davidson et al., 2019; Zhiltsova et al., 2019).”
BackgroundExamining Gender and Race Bias in Two Hundred Sentiment Analysis Systems
Kiritchenko, S., & Mohammad, S. (2018) · Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s18-2005 · Cited by 384
A snapshot of the frontiers of fairness in machine learning
Chouldechova, A., & Roth, A. (2020) · Communications of the ACM · doi:10.1145/3376898 · Cited by 354 · cited 2×
“Surveys of Mehrabi et al. (2019), Hutchinson and Mitchell (2019) and Chouldechova and Roth (2020) cover a broad scope of literature on algorithmic fairness.”
BackgroundMeasuring Bias in Contextualized Word Representations
Kurita, K., Vyas, N., Pareek, A., Black, A. W., & Tsvetkov, Y. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3823 · Cited by 348 · cited 2×
“A related line of research measures bias present in sentence or word representations (Bolukbasi et al., 2016; Caliskan et al., 2017; Kurita et al., 2019; Sedoc and Ungar, 2019; Chaloner and Maldon-ado, 2019; Dev and Phillips, 2019; Gonen and Goldberg, 2019; Hall Maudslay et al., 2019; Liang et al.,…”
BackgroundMethodBias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
De-Arteaga, M., Romanov, A., Wallach, H., Chayes, J., Borgs, C., Chouldechova, A., Geyik, S., Kenthapadi, K., & Kalai, A. T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287572 · Cited by 337 · cited 4×
“, 2017), 11 (Blodgett and O’Connor, 2017; Bhaskaran and Bhallamudi, 2019), 12 (De-Arteaga et al., 2019), 13 (Stanovsky et al.”
BackgroundInfluentialBias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
De-Arteaga, M., Romanov, A., Wallach, H., Chayes, J., Borgs, C., Chouldechova, A., Geyik, S., Kenthapadi, K., & Kalai, A. T. (2019) · arXiv · doi:10.48550/arxiv.1901.09451 · Cited by 337 · cited 4×
“, 2017), 11 (Blodgett and O’Connor, 2017; Bhaskaran and Bhallamudi, 2019), 12 (De-Arteaga et al., 2019), 13 (Stanovsky et al.”
BackgroundInfluentialAbsolutely relative: How research results are summarized can affect treatment decisions
Forrow, L., Taylor, W. C., & Arnold, R. M. (1992) · The American Journal of Medicine · doi:10.1016/0002-9343(92)90100-p · Cited by 334 · cited 1×
“Indeed, such effects have also been widely discussed in the context of reporting health risks (Forrow et al., 1992; Stegenga, 2015; Noordzij et al., 2017).”
BackgroundNuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification
Borkan, D., Dixon, L., Sorensen, J., Thain, N., & Vasserman, L. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3317593 · Cited by 309 · cited 5×
“A number of papers have published evidence of uneven treatment of different demographics (Dixon et al., 2018; Zhao et al., 2018; Rudinger et al., 2018; Garg et al., 2019; Borkan et al., 2019; Stanovsky et al., 2019; Gonen and Webster, 2020; Huang et al., 2020a; Nangia et al., 2020), which can reportedly cause a variet…”
BackgroundInfluential50 Years of Test (Un)fairness: Lessons for Machine Learning
Hutchinson, B., & Mitchell, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287600 · Cited by 297 · cited 2×
“Surveys of Mehrabi et al. (2019), Hutchinson and Mitchell (2019) and Chouldechova and Roth (2020) cover a broad scope of literature on algorithmic fairness.”
BackgroundEvaluating Gender Bias in Machine Translation
Stanovsky, G., Smith, N. A., & Zettlemoyer, L. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1164 · Cited by 292 · cited 1×
“Indeed, such effects have also been widely discussed in the context of reporting health risks (Forrow et al., 1992; Stegenga, 2015; Noordzij et al., 2017).”
BackgroundSocial Biases in NLP Models as Barriers for Persons with Disabilities
Hutchinson, B., Prabhakaran, V., Denton, E., Webster, K., Zhong, Y., & Denuyl, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.487 · Cited by 274
Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
Beutel, A., Chen, J., Zhao, Z., & Chi, E. H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.00075 · Cited by 274
Social Biases in NLP Models as Barriers for Persons with Disabilities
Hutchinson, B., Prabhakaran, V., Denton, E., Webster, K., Zhong, Y., & Denuyl, S. (2020) · arXiv · doi:10.48550/arxiv.2005.00813 · Cited by 274 · cited 2×
“This is reflected in our choice of identity terms, which we have gathered from Wikipedia, Wiktionary as well as (Dixon et al., 2018) and (Hutchinson et al., 2020) (see Table 4 for an example).”
MethodPredictive biases in natural language processing models: A conceptual framework and overview
Shah, D. S., Schwartz, H. A., & Hovy, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.468 · Cited by 241 · cited 1×
“Shah et al. (2020) offer both a survey of bias in NLP as well as a conceptual framework for studying bias.”
BackgroundGonen, H., & Goldberg, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03862 · Cited by 225 · cited 1×
“A related line of research measures bias present in sentence or word representations (Bolukbasi et al., 2016; Caliskan et al., 2017; Kurita et al., 2019; Sedoc and Ungar, 2019; Chaloner and Maldonado, 2019; Dev and Phillips, 2019; Gonen and Goldberg, 2019; Hall Maudslay et al., 2019; Liang et al., 2020; Shin et al., 2…”
BackgroundMind the GAP: A balanced corpus of gendered ambiguous pronouns
Webster, K., Recasens, M., Axelrod, V., & Baldridge, J. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00240 · Cited by 223
Differential Privacy Has Disparate Impact on Model Accuracy
Bagdasaryan, E., & Shmatikov, V. (2019) · arXiv · doi:10.48550/arxiv.1905.12101 · Cited by 219 · cited 1×
“…group or for each split of a challenge dataset (Hovy and Søgaard, 2015; Rudinger et al., 2018; Zhao et al., 2018; Garimella et al., 2019; Sap et al., 2019; Bagdasaryan et al., 2019; Stafanoviˇcs et al., 2020; Tan et al., 2020; Mehrabi et al., 2020; Nadeem et al., 2020; Cao and Daumé III, 2020).”
BackgroundCounterfactual Fairness in Text Classification through Robustness
Garg, S., Perot, V., Limtiaco, N., Taly, A., Chi, E. H., & Beutel, A. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3317950 · Cited by 198 · cited 4×
“…of papers have published evidence of uneven treatment of different demographics (Dixon et al., 2018; Zhao et al., 2018; Rudinger et al., 2018; Garg et al., 2019; Borkan et al., 2019; Stanovsky et al., 2019; Gonen and Webster, 2020; Huang et al., 2020a; Nangia et al., 2020), which can…”
BackgroundInfluentialTowards Controllable Biases in Language Generation
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.291 · Cited by 197
Relative risk versus absolute risk: one cannot be interpreted without the other
Noordzij, M., van Diepen, M., Caskey, F. C., & Jager, K. J. (2017) · Nephrology Dialysis Transplantation · doi:10.1093/ndt/gfw465 · Cited by 172 · cited 1×
“Indeed, such effects have also been widely discussed in the context of reporting health risks (Forrow et al., 1992; Stegenga, 2015; Noordzij et al., 2017).”
BackgroundGender Bias in Machine Translation
Savoldi, B., Gaido, M., Bentivogli, L., Negri, M., & Turchi, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00401 · Cited by 165 · cited 1×
“There are also many task specific surveys, e.g., for language generation (Sheng et al., 2021) or machine translation (Savoldi et al., 2021).”
BackgroundBlodgett, S. L., & O'Connor, B. (2017) · arXiv · doi:10.48550/arxiv.1707.00061 · Cited by 153 · cited 2×
“, 2017), 11 (Blodgett and O’Connor, 2017; Bhaskaran and Bhallamudi, 2019), 12 (De-Arteaga et al.”
An annotated dataset of literary entities
Bamman, D., Popat, S., & Shen, S. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1220 · Cited by 149
Towards Debiasing Sentence Representations
Liang, P. P., Li, I. M., Zheng, E., Lim, Y. C., Salakhutdinov, R., & Morency, L.-P. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.488 · Cited by 147
Societal Biases in Language Generation: Progress and Challenges
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.330 · Cited by 131 · cited 1×
“There are also many task specific surveys, e.g., for language generation (Sheng et al., 2021) or machine translation (Savoldi et al., 2021).”
BackgroundReducing Sentiment Bias in Language Models via Counterfactual Evaluation
Huang, P.-S., Zhang, H., Jiang, R., Stanforth, R., Welbl, J., Rae, J., Maini, V., Yogatama, D., & Kohli, P. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.7 · Cited by 124
50 Years of Test (Un)fairness: Lessons for Machine Learning
Hutchinson, B., & Mitchell, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.10104 · Cited by 118 · cited 2×
“Surveys of Mehrabi et al. (2019), Hutchinson and Mitchell (2019) and Chouldechova and Roth (2020) cover a broad scope of literature on algorithmic fairness.”
BackgroundAttenuating Bias in Word VectorsPreprint
Dev, S., & Phillips, J. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.07656 · Cited by 117 · cited 1×
“…or word representations (Bolukbasi et al., 2016; Caliskan et al., 2017; Kurita et al., 2019; Sedoc and Ungar, 2019; Chaloner and Maldon-ado, 2019; Dev and Phillips, 2019; Gonen and Goldberg, 2019; Hall Maudslay et al., 2019; Liang et al., 2020; Shin et al., 2020; Liang et al., 2020;…”
BackgroundPerturbation Sensitivity Analysis to Detect Unintended Model Biases
Prabhakaran, V., Hutchinson, B., & Mitchell, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1578 · Cited by 115 · cited 2×
“Out of the 26 metrics we find, only four do not fit within our framework: BPSN and BNSP (Borkan et al., 2019), the metric (De-Arteaga et al., 2019) and Perturbation Label Distance (Prabhakaran et al., 2019).”
BackgroundMethodPutting Fairness Principles into Practice: Challenges, Metrics, and Improvements
Beutel, A., Chen, J., Doshi, T., Qian, H., Woodruff, A., Luu, C., Kreitmann, P., Bischof, J., & Chi, E. H. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314234 · Cited by 111
Tagging Performance Correlates with Author Age
Hovy, D., & Søgaard, A. (2015) · Annual Meeting of the Association for Computational Linguistics · doi:10.3115/v1/p15-2079 · Cited by 110 · cited 1×
“…metric, e.g., F1 or accuracy, or a more elaborate measure independently for each protected group or for each split of a challenge dataset (Hovy and Søgaard, 2015; Rudinger et al., 2018; Zhao et al., 2018; Garimella et al., 2019; Sap et al., 2019; Bagdasaryan et al., 2019; Stafanoviˇcs et…”
BackgroundIt’s All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution
Hall Maudslay, R., Gonen, H., Cotterell, R., & Teufel, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1530 · Cited by 103 · cited 2×
“…(Bolukbasi et al., 2016; Caliskan et al., 2017; Kurita et al., 2019; Sedoc and Ungar, 2019; Chaloner and Maldonado, 2019; Dev and Phillips, 2019; Gonen and Goldberg, 2019; Hall Maudslay et al., 2019; Liang et al., 2020; Shin et al., 2020; Liang et al., 2020; Papakyriakopoulos et al., 2020).”
Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem
Saunders, D., & Byrne, B. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.690 · Cited by 97
Stereotypical Bias Removal for Hate Speech Detection Task using Knowledge-based Generalizations
Badjatiya, P., Gupta, M., & Varma, V. (2019) · The World Wide Web Conference · doi:10.1145/3308558.3313504 · Cited by 93 · cited 1×
“In yet another line of research, Badjatiya et al. (2019) detect bias through identifying bias sensitive words.”
BackgroundPapakyriakopoulos, O., Hegelich, S., Serrano, J. C. M., & Marco, F. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372843 · Cited by 88 · cited 1×
“…(Bolukbasi et al., 2016; Caliskan et al., 2017; Kurita et al., 2019; Sedoc and Ungar, 2019; Chaloner and Maldon-ado, 2019; Dev and Phillips, 2019; Gonen and Goldberg, 2019; Hall Maudslay et al., 2019; Liang et al., 2020; Shin et al., 2020; Liang et al., 2020; Papakyriakopoulos et al., 2020).”
It’s Morphin’ Time! Combating Linguistic Discrimination with Inflectional Perturbations
Tan, S., Joty, S., Kan, M.-Y., & Socher, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.263 · Cited by 84 · cited 1×
“…group or for each split of a challenge dataset (Hovy and Søgaard, 2015; Rudinger et al., 2018; Zhao et al., 2018; Garimella et al., 2019; Sap et al., 2019; Bagdasaryan et al., 2019; Stafanoviˇcs et al., 2020; Tan et al., 2020; Mehrabi et al., 2020; Nadeem et al., 2020; Cao and Daumé III, 2020).”
BackgroundStegenga, J. (2015) · Studies in History and Philosophy of Science Part C Studies in History and Philosophy of Biological and Biomedical Sciences · doi:10.1016/j.shpsc.2015.06.003 · Cited by 84 · cited 1×
“Indeed, such effects have also been widely discussed in the context of reporting health risks (Forrow et al., 1992; Stegenga, 2015; Noordzij et al., 2017).”
BackgroundMan is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition
Mehrabi, N., Gowda, T., Morstatter, F., Peng, N., & Galstyan, A. (2020) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3372923.3404804 · Cited by 79
Good Secretaries, Bad Truck Drivers? Occupational Gender Stereotypes in Sentiment Analysis
Bhaskaran, J., & Bhallamudi, I. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3809 · Cited by 71 · cited 1×
“, 2017), 11 (Blodgett and O’Connor, 2017; Bhaskaran and Bhallamudi, 2019), 12 (De-Arteaga et al.”
Mitigating Gender Bias in Machine Translation with Target Gender AnnotationsPreprint
Stafanovičs, A., Pinnis, M., & Bergmanis, T. (2020) · Conference on Machine Translation · doi:10.48550/arxiv.2010.06203 · Cited by 71 · cited 1×
“…group or for each split of a challenge dataset (Hovy and Søgaard, 2015; Rudinger et al., 2018; Zhao et al., 2018; Garimella et al., 2019; Sap et al., 2019; Bagdasaryan et al., 2019; Stafanoviˇcs et al., 2020; Tan et al., 2020; Mehrabi et al., 2020; Nadeem et al., 2020; Cao and Daumé III, 2020).”
BackgroundTowards Understanding Gender Bias in Relation Extraction
Gaut, A., Sun, T., Tang, S., Huang, Y., Qian, J., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W., et al. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.265 · Cited by 54
Cyberbullying Detection With Fairness Constraints
Gencoglu, O. (2021) · IEEE Internet Computing · doi:10.1109/mic.2020.3032461 · Cited by 53 · cited 1×
“This is all the more important for FPED and FNED, as they have been very influential, with many works relying exclusively on these metrics (Rios, 2020; Huang et al., 2020b; Gencoglu, 2021; Rios and Lwowski, 2020).”
BackgroundWasserstein Fair Classification
Jiang, R., Pacchiano, A., Stepleton, T., Jiang, H., & Chiappa, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.12059 · Cited by 53 · cited 1×
“(2020a) choose Wasserstein-1 distance (Jiang et al., 2020) 3 21 , while Borkan et al.”
BackgroundChaloner, K., & Maldonado, A. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3804 · Cited by 50
Twitter Universal Dependency Parsing for African-American and Mainstream American English
Blodgett, S. L., Wei, J., & O’Connor, B. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1131 · Cited by 49 · cited 3×
“, 2019; Saunders and Byrne, 2020), 14 (Blodgett et al., 2018), 15 (Bamman et al.”
BackgroundDebiasing Embeddings for Reduced Gender Bias in Text Classification
Prost, F., Thain, N., & Bolukbasi, T. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3810 · Cited by 44
Garimella, A., Banea, C., Hovy, D., & Mihalcea, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1339 · Cited by 41
Mitigating Gender Bias in Machine Translation with Target Gender Annotations
Stafanovičs, A., Bergmanis, T., & Pinnis, M. (2020) · Proceedings of the Fifth Conference on Machine Translation · doi:10.18653/v1/2020.wmt-1.73 · Cited by 35
The Limits of Abstract Evaluation Metrics: The Case of Hate Speech Detection
Olteanu, A., Talamadupula, K., & Varshney, K. R. (2017) · Proceedings of the 2017 ACM on Web Science Conference · doi:10.1145/3091478.3098871 · Cited by 30 · cited 1×
“In line with the recommendations of Olteanu et al. (2017) and Blodgett et al. (2020), we assert that fairness metrics need to be grounded in the application domain and carefully matched to the type of studied bias to offer meaningful insights.”
BackgroundExamining Gender and Race Bias in Two Hundred Sentiment Analysis Systems
Kiritchenko, S., & Mohammad, S. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.04508 · Cited by 28
Automatically Identifying Gender Issues in Machine Translation using Perturbations
Gonen, H., & Webster, K. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.180 · Cited by 28
The Role of Protected Class Word Lists in Bias Identification of Contextualized Word Representations
Sedoc, J., & Ungar, L. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3808 · Cited by 26 · cited 1×
“…line of research measures bias present in sentence or word representations (Bolukbasi et al., 2016; Caliskan et al., 2017; Kurita et al., 2019; Sedoc and Ungar, 2019; Chaloner and Maldon-ado, 2019; Dev and Phillips, 2019; Gonen and Goldberg, 2019; Hall Maudslay et al., 2019; Liang et al.,…”
BackgroundShin, S., Song, K., Jang, J., Kim, H., Joo, W., & Moon, I. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.280 · Cited by 25
FuzzE: Fuzzy Fairness Evaluation of Offensive Language Classifiers on African-American English
Rios, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5434 · Cited by 23 · cited 1×
“This is all the more important for FPED and FNED, as they have been very influential, with many works relying exclusively on these metrics (Rios, 2020; Huang et al., 2020b; Gencoglu, 2021; Rios and Lwowski, 2020).”
BackgroundThe meaning and measurement of bias: lessons from natural language processing
Jacobs, A. Z., Blodgett, S. L., Barocas, S., Daumé, H., & Wallach, H. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375671 · Cited by 23 · cited 1×
“Next, we empirically investigate the role of different metrics in detecting the systemic differences in performance for different demographic groups, i.e., differences in quality of service (Jacobs et al., 2020).”
MethodToward Gender-Inclusive Coreference Resolution
Cao, Y. T., & Daumé III, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.418 · Cited by 22
Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition
Huang, X., Xing, L., Dernoncourt, F., & Paul, M. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10361 · Cited by 21
Mitigation of Unintended Biases against Non-Native English Texts in Sentiment Analysis.
Zhiltsova, A., Caton, S. J., & Mulway, C. (2019) · IASC · Cited by 15
Joint Multiclass Debiasing of Word Embeddings
Popović, R., Lemmerich, F., & Strohmaier, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59491-6_8 · Cited by 10
An Empirical Study of the Downstream Reliability of Pre-Trained Word Embeddings
Rios, A., & Lwowski, B. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.299 · Cited by 3 · cited 1×
“This is all the more important for FPED and FNED, as they have been very influential, with many works relying exclusively on these metrics (Rios, 2020; Huang et al., 2020b; Gencoglu, 2021; Rios and Lwowski, 2020).”
Background
Research works printed but not linked: 3
- Clark, 2020, ELECTRA: Pre-training text encoders as discriminators rather than generators, The International Conference on Learning Representations (ICLR)
- Goldfarb-Tarrant, 2021, Intrinsic bias metrics do not correlate with application bias, 10.18653/v1/2021.acl-long.150 · DOI · Paper page
- Liu, 2019, RoBERTa: A robustly optimized BERT pretraining approach, CoRR
Cited by
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Biases in Large Language Models: Origins, Inventory, and Discussion
Navigli, R., Conia, S., & Ross, B. (2023) · Journal of Data and Information Quality · doi:10.1145/3597307 · Cited by 364
“Importantly, it has been found that the various sets of metrics used in hundreds of papers dealing with social bias can be uniied under three generalized fairness metrics: pairwise comparison, background comparison, and multi-group comparison metrics [41].”
BackgroundTowards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10896-y · Cited by 141
Human-LLM Collaborative Annotation Through Effective Verification of LLM Labels
Wang, X., Kim, H., Rahman, S., Mitra, K., & Miao, Z. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641960 · Cited by 128
Policy advice and best practices on bias and fairness in AI
Álvarez, J. M., Colmenarejo, A. B., Elobaid, A., Fabbrizzi, S., Fahimi, M., Ferrara, A., Ghodsi, S., Mougan, C., Papageorgiou, I. T., Lobo, P. R., et al. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09746-w · Cited by 160
“A few global events are targeted at multidisciplinary aspects of bias, fairness and other ethical issues in AI and algorithmic decision making.”
Metadata
- Authors
- Czarnowska, P., Vyas, Y., & Shah, K.
- Affiliations
- University of Cambridge (GB)
- Amazon (United States) (US)
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2021
- Volume
- 9
- Pages
- 1249-1267
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W3174651299
- Cited by
- 72
- References
- 82
- Citations by year
- 2026: 9 · 2025: 10 · 2024: 22 · 2023: 19 · 2022: 12
- FWCI
- 16.59
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- fairness metrics, algorithmic bias, demographic bias, bias measurement, parameter selection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.