Auto-Debias: Debiasing Masked Language Models with Automated Biased Prompts
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
Human-like biases and undesired social stereotypes exist in large pretrained language models.Given the wide adoption of these models in real-world applications, mitigating such biases has become an emerging and important task.In this paper, we propose an automatic method to mitigate the biases in pretrained language models.Different from previous debiasing work that uses external corpora to finetune the pretrained models, we instead directly probe the biases encoded in pretrained models through prompts.Specifically, we propose a variant of the beam search method to automatically search for biased prompts such that the cloze-style completions are the most different with respect to different demographic groups.Given the identified biased prompts, we then propose a distribution alignment loss to mitigate the biases.Experiment results on standard datasets and metrics show that our proposed Auto-Debias approach can significantly reduce biases, including gender and racial bias, in pretrained language models such as BERT, RoBERTa and ALBERT.Moreover, the improvement in fairness does not decrease the language models' understanding abilities, as shown using the GLUE benchmark.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308 · cited 11×
“In the gender debias experiments, we use BERT-base-uncased, RoBERTa-base, and ALBERT-large-v2.”
MethodInfluentialGLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
Practical Black-Box Attacks against Machine Learning
Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z. B., & Swami, A. (2017) · Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security · doi:10.1145/3052973.3053009 · Cited by 3,495 · cited 1×
“Discussion: Another perspective for Auto-Debias is that the debiasing method resembles adversarial training (Goodfellow et al., 2014; Papernot et al., 2017).”
MethodSemantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Language Models as Knowledge Bases?
Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834
Mitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387 · cited 1×
“Recent research has adopted the adversarial training idea to remove biases from sensitive features, representations and classification models (Zhang et al., 2018; Elazar and Goldberg, 2018; Beutel et al., 2017; Han et al., 2021).”
MethodAutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., & Singh, S. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.346 · Cited by 1,279
Word embeddings quantify 100 years of gender and ethnic stereotypes
Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720347115 · Cited by 1,065 · cited 1×
“Prior work has found that traditional static word embeddings contain human-like biases and stereotypes (Caliskan et al., 2017; Bolukbasi et al., 2016; Garg et al., 2018; Manzini et al., 2019; Gonen and Goldberg, 2019).”
BackgroundLanguage (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050 · cited 2×
“Unfortunately, pretrained language models, which are trained on large human-written corpora, also inherit human-like biases and undesired social stereotypes (Caliskan et al., 2017; Bolukbasi et al., 2016; Blodgett et al., 2020).”
BackgroundMeasuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674 · cited 1×
“Bias in NLP systems can stem from training data (Dixon et al., 2018), pre-trained word embeddings or can be amplified by the machine learning models.”
BackgroundThe Woman Worked as a Babysitter: On Biases in Language Generation
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1339 · Cited by 537 · cited 2×
“Prompts have been an effective tool in probing the internal knowledge relations of language models (Petroni et al., 2019), and they can also reflect the stereotypes encompassed in PLMs (Ousidhoum et al., 2021; Sheng et al., 2019).”
BackgroundMethodOn Measuring Social Biases in Sentence Encoders
May, C., Wang, A., Bordia, S., Bowman, S. R., & Rudinger, R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1063 · Cited by 461 · cited 3×
“Recent work in language model prompting shows that through cloze-style prompts, one can probe and analyze the knowledge (Petroni et al., 2019), biases (May et al., 2019) or toxic content (Ousidhoum et al., 2021) in PLMs. Motivated by this, instead of refer-1012 .”
BackgroundMethodGonen, H., & Goldberg, Y. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1061 · Cited by 454 · cited 1×
“Prior work has found that traditional static word embeddings contain human-like biases and stereotypes (Caliskan et al., 2017; Bolukbasi et al., 2016; Garg et al., 2018; Manzini et al., 2019; Gonen and Goldberg, 2019).”
BackgroundCrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Nangia, N., Vania, C., Bhalerao, R., & Bowman, S. R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.154 · Cited by 446
Gender Bias in Coreference Resolution
Rudinger, R., Naradowsky, J., Leonard, B., & Van Durme, B. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2002 · Cited by 443 · cited 1×
“…or even amplified in downstream NLP tasks such as sentiment classification (Kiritchenko and Moham-mad, 2018), co-reference resolution (Zhao et al., 2019; Rudinger et al., 2018), clinical text classification (Zhang et al., 2020) and psychometric analysis (Abbasi et al., 2021; Ahmad et al., 2020).”
BackgroundExamining Gender and Race Bias in Two Hundred Sentiment Analysis Systems
Kiritchenko, S., & Mohammad, S. (2018) · Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s18-2005 · Cited by 384
Measuring Bias in Contextualized Word Representations
Kurita, K., Vyas, N., Pareek, A., Black, A. W., & Tsvetkov, Y. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3823 · Cited by 348 · cited 1×
“Researchers have also reported similar human-like biases and stereotypes in contextual embedding PLMs (May et al., 2019; Kurita et al., 2019; Tan and Celis, 2019; Hutchinson et al., 2020; Guo and Caliskan, 2021; Wolfe and Caliskan, 2021) or in the text generation tasks (Schick et al., 2021; Sheng…”
BackgroundNull It Out: Guarding Protected Attributes by Iterative Nullspace Projection
Ravfogel, S., Elazar, Y., Gonen, H., Twiton, M., & Goldberg, Y. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.647 · Cited by 287
Factual Probing Is [MASK]: Learning vs. Learning to Recall
Zhong, Z., Friedman, D., & Chen, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.398 · Cited by 274
Social Biases in NLP Models as Barriers for Persons with Disabilities
Hutchinson, B., Prabhakaran, V., Denton, E., Webster, K., Zhong, Y., & Denuyl, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.487 · Cited by 274
Social Biases in NLP Models as Barriers for Persons with Disabilities
Hutchinson, B., Prabhakaran, V., Denton, E., Webster, K., Zhong, Y., & Denuyl, S. (2020) · arXiv · doi:10.48550/arxiv.2005.00813 · Cited by 274 · cited 1×
“…also reported similar human-like biases and stereotypes in contextual embedding PLMs (May et al., 2019; Kurita et al., 2019; Tan and Celis, 2019; Hutchinson et al., 2020; Guo and Caliskan, 2021; Wolfe and Caliskan, 2021) or in the text generation tasks (Schick et al., 2021; Sheng et al., 2019).”
BackgroundSelf-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP
Schick, T., Udupa, S., & Schütze, H. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00434 · Cited by 227 · cited 1×
“…also reported similar human-like biases and stereotypes in contextual embedding PLMs (May et al., 2019; Kurita et al., 2019; Tan and Celis, 2019; Hutchinson et al., 2020; Guo and Caliskan, 2021; Wolfe and Caliskan, 2021) or in the text generation tasks (Schick et al., 2021; Sheng et al., 2019).”
BackgroundMeasuring and Reducing Gendered Correlations in Pre-trained Models
Webster, K., Wang, X., Tenney, I., Beutel, A., Pitler, E., Pavlick, E., Chen, J., Chi, E., & Petrov, S. (2020) · arXiv · doi:10.48550/arxiv.2010.06032 · Cited by 195
Guo, W., & Caliskan, A. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462536 · Cited by 180 · cited 1×
“…also reported similar human-like biases and stereotypes in contextual embedding PLMs (May et al., 2019; Kurita et al., 2019; Tan and Celis, 2019; Hutchinson et al., 2020; Guo and Caliskan, 2021; Wolfe and Caliskan, 2021) or in the text generation tasks (Schick et al., 2021; Sheng et al., 2019).”
BackgroundPerturbation Sensitivity Analysis to Detect Unintended Model Biases
Prabhakaran, V., Hutchinson, B., & Mitchell, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1578 · Cited by 115 · cited 2×
“Our idea is motivated by the assumption that a fair NLP system should produce scores that are independent to the choice of identities mentioned in the text (Prabhakaran et al., 2019).”
BackgroundMethodHurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings
Zhang, H., Lu, A. X., Abdalla, M., McDermott, M., & Ghassemi, M. (2020) · arXiv · doi:10.48550/arxiv.2003.11515 · Cited by 37
Cited by
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10896-y · Cited by 141
Survey on Machine Learning Biases and Mitigation Techniques
Siddique, S., Haque, M. A., George, R., Gupta, K. D., Gupta, D., & Faruk, M. J. H. (2023) · Digital · doi:10.3390/digital4010001 · Cited by 78
Zhou, F., Mao, Y., Yu, L., Yang, Y., & Zhong, T. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.232 · Cited by 29
“Consider gender debiasing using Auto-Debias (Guo et al., 2022) as an example.”
BackgroundMethodInfluentialSelf-citationDisparities in seizure outcomes revealed by large language models
(2024) · doi:10.1093/jamia/ocae047 · Cited by 17
Metadata
- Authors
- Guo, Y., Yang, Y., & Abbasi, A.
- Venue
- Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2022
- Pages
- 1012-1023
- OpenAlex
- W4285192297
- Cited by
- 113
- References
- 31
- FWCI
- 27.73
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.