Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP
Open accessThe DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Abstract
⚠ This paper contains prompts and model outputs that are offensive in nature. When trained on large, unfiltered crawls from the Internet, language models pick up and reproduce all kinds of undesirable biases that can be found in the data: They often generate racist, sexist, violent, or otherwise toxic language. As large models require millions of training examples to achieve good performance, it is difficult to completely prevent them from being exposed to such content. In this paper, we first demonstrate a surprising finding: Pretrained language models recognize, to a considerable degree, their undesirable biases and the toxicity of the content they produce. We refer to this capability as self-diagnosis. Based on this finding, we then propose a decoding algorithm that, given only a textual description of the undesired behavior, reduces the probability of a language model producing problematic text. We refer to this approach as self-debiasing. Self-debiasing does not rely on manually curated word lists, nor does it require any training data or changes to the model’s parameters. While we by no means eliminate the issue of language models generating biased text, we believe our approach to be an important step in this direction.1
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, it has limitations as it cannot be applied to downstream tasks.”
From Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models (2024) · cites this paper as Schick et al. (2021)
In the citing paper
“Importantly, this method does not compromise the language model and requires no additional training.”
From Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models (2024) · cites this paper as Schick et al. (2021)
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 16×
“…illustrating that both static (e.g., Mikolov et al., 2013; Bojanowski et al., 2017) and contextualized word embeddings (e.g., Peters et al., 2018; Devlin et al., 2019) pretrained in a self-supervised fashion exhibit all kinds of unfair and discriminative biases (Bolukbasi et al., 2016; Caliskan…”
BackgroundMethodInfluentialEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 1×
“There is a large body of work illustrating that both static (e.g., Mikolov et al., 2013; Bojanowski et al., 2017) and contextualized word embeddings (e.g., Peters et al., 2018; Devlin et al., 2019) pretrained in a self-supervised fashion exhibit all kinds of unfair and discriminative biases…”
BackgroundLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 1×
“…a large body of work illustrating that both static (e.g., Mikolov et al., 2013; Bojanowski et al., 2017) and contextualized word embeddings (e.g., Peters et al., 2018; Devlin et al., 2019) pretrained in a self-supervised fashion exhibit all kinds of unfair and discriminative biases (Bolukbasi et…”
BackgroundExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 13×
“Figure 1: Most probable continuations according to T5-3B (Raffel et al., 2020) and GPT2-XL (Radford et al.”
BackgroundMethodInfluentialEnriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645 · cited 1×
“There is a large body of work illustrating that both static (e.g., Mikolov et al., 2013; Bojanowski et al., 2017) and contextualized word embeddings (e.g., Peters et al., 2018; Devlin et al., 2019) pretrained in a self-supervised fashion exhibit all kinds of unfair and discriminative biases…”
BackgroundTransformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“Building training datasets with more care and deliberation, an alternative solution discussed by Bender et al. (2021), is important, especially for improving linguistic and cultural diversity in online and other forms of communication.”
BackgroundEnergy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1355 · Cited by 3,684 · cited 1×
“It can also necessitate building different training sets and, accordingly, training different models for each desired behavior, which can result in high environmental impact (Strubell et al., 2019).”
BackgroundSemantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · cited 1×
“…2018; Devlin et al., 2019) pretrained in a self-supervised fashion exhibit all kinds of unfair and discriminative biases (Bolukbasi et al., 2016; Caliskan et al., 2017; Zhao et al., 2017; Rudinger et al., 2018; Gonen and Goldberg, 2019; Bordia and Bowman, 2019; Sheng et al., 2019; Basta et al.,…”
BackgroundZhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.11 · Cited by 2,264 · cited 3×
“(2020) presume that this is because BERT was trained only on Wikipedia and BookCorpus (Zhu et al., 2015), whereas RoBERTa was additionally trained on OpenWebText (Gokaslan and Cohen, 2019), which likely has a much higher incidence of biased text than the other two sources (Gehman et al.”
BackgroundResultZhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06724 · Cited by 2,264 · cited 3×
“(2020) presume that this is because BERT was trained only on Wikipedia and BookCorpus (Zhu et al., 2015), whereas RoBERTa was additionally trained on OpenWebText (Gokaslan and Cohen, 2019), which likely has a much higher incidence of biased text than the other two sources (Gehman et al.”
BackgroundResultDon’t Stop Pretraining: Adapt Language Models to Domains and Tasks
Gururangan, S., Marasović, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., & Smith, N. A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.740 · Cited by 2,103
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 3×
“For static word embeddings, various algorithms for debiasing have been proposed (Bolukbasi et al., 2016; Zhao et al., 2018; Ravfogel et al., 2020; Go-nen and Goldberg, 2019), many of them being based on predefined word lists or other external resources.”
BackgroundMethodExploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference
Schick, T., & Schütze, H. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.20 · Cited by 1,305
Self-citationTransformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., et al. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.5347031 · Cited by 1,152
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., D., M., Châtel, G., Chaumond, J., Rault, T., Voss, C., Wang, F., et al. (2019) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3385997 · Cited by 1,037
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Fedus, W., Zoph, B., & Shazeer, N. (2021) · arXiv · doi:10.48550/arxiv.2101.03961 · Cited by 1,005 · cited 1×
“With model sizes continually increasing (Radford et al., 2019; Raffel et al., 2020; Brown et al., 2020; Fedus et al., 2021), ever-larger pretraining datasets are necessary both to prevent overfit-1 Our implementation is publicly available at https:// github.com/timoschick/self-debiasing .”
BackgroundHow Can We Know What Language Models Know?
Jiang, Z., Xu, F. F., Araki, J., & Neubig, G. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00324 · Cited by 1,002
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1323 · Cited by 829 · cited 1×
“…2019) pretrained in a self-supervised fashion exhibit all kinds of unfair and discriminative biases (Bolukbasi et al., 2016; Caliskan et al., 2017; Zhao et al., 2017; Rudinger et al., 2018; Gonen and Goldberg, 2019; Bordia and Bowman, 2019; Sheng et al., 2019; Basta et al., 2019; Nangia et al.,…”
BackgroundCTRL: A Conditional Transformer Language Model for Controllable Generation
Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., & Socher, R. (2019) · arXiv · doi:10.48550/arxiv.1909.05858 · Cited by 751 · cited 1×
“See Table 1 for attribute descriptions (sdg, sdb 1 ) and §4.2 for bias types (sdb 2 ). proaches for controllable text generation (Keskar et al., 2019; Schick and Schütze, 2020; He et al., 2020) but these approaches (i) require either a customized pretraining objective or labeled training data, and…”
InfluentialIt’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners
Schick, T., & Schütze, H. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.185 · Cited by 747
Self-citationRealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.416 · Cited by 643 · cited 1×
“For contextualized embeddings, similar methods to alleviate the issue of undesirable biases and toxicity have been proposed (Dev et al., 2020; Nangia et al., 2020; Nadeem et al., 2020; Krause et al., 2020; Liang et al., 2020; Kaneko and Bollegala, 2021a).”
MethodStereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2020) · arXiv · doi:10.48550/arxiv.2004.09456 · Cited by 643
The Woman Worked as a Babysitter: On Biases in Language Generation
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1339 · Cited by 537
Plug and Play Language Models: A Simple Approach to Controlled Text Generation
Dathathri, S., Madotto, A., Lan, J., Hung, J., Frank, E., Molino, P., Yosinski, J., & Liu, R. (2019) · arXiv · doi:10.48550/arxiv.1912.02164 · Cited by 508 · cited 1×
“For text generation, Gehman et al. (2020) propose domain-adaptive pretraining on non-toxic corpora as outlined by Gururangan et al. (2020) and consider plug and play language models (Dathathri et al., 2020).”
MethodPersistent Anti-Muslim Bias in Large Language Models
Abid, A., Farooqi, M., & Zou, J. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462624 · Cited by 476 · cited 1×
“…et al., 2016; Caliskan et al., 2017; Zhao et al., 2017; Rudinger et al., 2018; Gonen and Goldberg, 2019; Bordia and Bowman, 2019; Sheng et al., 2019; Basta et al., 2019; Nangia et al., 2020, i.a. ) and are prone to generating toxic texts (Brown et al., 2020; Gehman et al., 2020; Abid et al., 2021).”
BackgroundGonen, H., & Goldberg, Y. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1061 · Cited by 454 · cited 1×
“…exhibit all kinds of unfair and discriminative biases (Bolukbasi et al., 2016; Caliskan et al., 2017; Zhao et al., 2017; Rudinger et al., 2018; Gonen and Goldberg, 2019; Bordia and Bowman, 2019; Sheng et al., 2019; Basta et al., 2019; Nangia et al., 2020, i.a. ) and are prone to generating…”
BackgroundCrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Nangia, N., Vania, C., Bhalerao, R., & Bowman, S. R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.154 · Cited by 446
Gender Bias in Coreference Resolution
Rudinger, R., Naradowsky, J., Leonard, B., & Van Durme, B. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2002 · Cited by 443 · cited 1×
“…a self-supervised fashion exhibit all kinds of unfair and discriminative biases (Bolukbasi et al., 2016; Caliskan et al., 2017; Zhao et al., 2017; Rudinger et al., 2018; Gonen and Goldberg, 2019; Bordia and Bowman, 2019; Sheng et al., 2019; Basta et al., 2019; Nangia et al., 2020, i.a. ) and are…”
BackgroundPointer Sentinel Mixture Models
Merity, S., Xiong, C., Bradbury, J., & Socher, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.07843 · Cited by 427 · cited 8×
“We measure language modeling performance on Wikitext-2 for RoBERTa and its self-debiased variant.”
MethodInfluentialLearning Gender-Neutral Word Embeddings
Zhao, J., Zhou, Y., Li, Z., Wang, W., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1521 · Cited by 345 · cited 1×
“For static word embeddings, various algorithms for debiasing have been proposed (Bolukbasi et al., 2016; Zhao et al., 2018; Ravfogel et al., 2020; Go-nen and Goldberg, 2019), many of them being based on predefined word lists or other external resources.”
MethodNull It Out: Guarding Protected Attributes by Iterative Nullspace Projection
Ravfogel, S., Elazar, Y., Gonen, H., Twiton, M., & Goldberg, Y. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.647 · Cited by 287
Gonen, H., & Goldberg, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03862 · Cited by 225 · cited 1×
“…exhibit all kinds of unfair and discriminative biases (Bolukbasi et al., 2016; Caliskan et al., 2017; Zhao et al., 2017; Rudinger et al., 2018; Gonen and Goldberg, 2019; Bordia and Bowman, 2019; Sheng et al., 2019; Basta et al., 2019; Nangia et al., 2020, i.a. ) and are prone to generating…”
BackgroundNegated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly
Kassner, N., & Schütze, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.698 · Cited by 223 · cited 1×
“Unfortunately, negation is often not understood well by current generations of language models (Kassner and Schütze, 2020).”
BackgroundSelf-citationGeDi: Generative Discriminator Guided Sequence Generation
Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., & Rajani, N. F. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.424 · Cited by 220
Identifying and Reducing Gender Bias in Word-Level Language Models
Bordia, S., & Bowman, S. R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop · doi:10.18653/v1/n19-3002 · Cited by 219
CTRLsum: Towards Generic Controllable Text Summarization
He, J., Kryściński, W., McCann, B., Rajani, N. F., & Xiong, C. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.396 · Cited by 201
Salazar, J., Liang, D., Nguyen, T. Q., & Kirchhoff, K. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.240 · Cited by 191 · cited 2×
“Nangia et al. (2020) use pseudo-log-likelihood (Wang and Cho, 2019; Salazar et al., 2020) to assign scores to sentences using MLMs. Bias in an MLM is then measured as the proportion of entries for which the MLM assigns a higher score to the more stereotypical sentence; an ideal model that does not…”
MethodEvaluating the Underlying Gender Bias in Contextualized Word Embeddings
Basta, C., Costa-jussà, M. R., & Casas, N. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3805 · Cited by 189 · cited 2×
“…et al., 2016; Caliskan et al., 2017; Zhao et al., 2017; Rudinger et al., 2018; Gonen and Goldberg, 2019; Bordia and Bowman, 2019; Sheng et al., 2019; Basta et al., 2019; Nangia et al., 2020, i.a. ) and are prone to generating toxic texts (Brown et al., 2020; Gehman et al., 2020; Abid et al., 2021).”
BackgroundCTRLsum: Towards Generic Controllable Text SummarizationPreprint
He, J.-X., Kryscinski, W., McCann, B., Rajani, N., & Xiong, C. (2020) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2012.04281 · Cited by 169 · cited 1×
“…(sdg, sdb 1 ) and §4.2 for bias types (sdb 2 ). proaches for controllable text generation (Keskar et al., 2019; Schick and Schütze, 2020; He et al., 2020) but these approaches (i) require either a customized pretraining objective or labeled training data, and (ii) use natural language…”
BackgroundInfluentialGonen, H., & Goldberg, Y. (2019) · arXiv (Cornell University) · Cited by 156
Toxicity Detection: Does Context Really Matter?
Pavlopoulos, J., Sorensen, J., Dixon, L., Thain, N., & Androutsopoulos, I. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.396 · Cited by 117
Few-Shot Text Generation with Pattern-Exploiting TrainingPreprint
Schick, T., & Schütze, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.11926 · Cited by 113 · cited 1×
“It is also similar to prompt- or keyword-based approaches for controllable text generation (Keskar et al., 2019; Schick and Schütze, 2020; He et al., 2020) but these approaches (i) require either a customized pretraining objective or labeled training data, and (ii) use natural language prompts to inform a language mod…”
MethodSelf-citationDebiasing Pre-trained Contextualised Embeddings
Kaneko, M., & Bollegala, D. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.107 · Cited by 87
Zero-shot Text Classification With Generative Language Models
Puri, R., & Catanzaro, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.10165 · Cited by 70 · cited 1×
“Our approach for performing self-diagnosis builds heavily on recent work that explores zero-shot learning using task descriptions (Radford et al., 2019; Puri and Catanzaro, 2019; Schick and Schütze, 2021a).”
MethodNeural Interactive Translation Prediction
Knowles, R., & Koehn, P. (2016) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 68
Dictionary-based Debiasing of Pre-trained Word Embeddings
Kaneko, M., & Bollegala, D. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.16 · Cited by 48
Models and Inference for Prefix-Constrained Machine Translation
Wuebker, J., Green, S., DeNero, J., Hasan, S., & Luong, M.-T. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1007 · Cited by 47 · cited 1×
“Our proposed self-debiasing algorithm bears some resemblance with prefix-constrained decoding used in interactive machine translation for completing partial translations (Knowles and Koehn, 2016; Wuebker et al., 2016).”
MethodMonolingual and Multilingual Reduction of Gender Bias in Contextualized Representations
Liang, S., Dufter, P., & Schütze, H. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.446 · Cited by 29
Self-citationOn measuring and mitigating biased inferences of word embeddings
Dev, S., Li, T., Phillips, J. M., & Srikumar, V. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6267 · Cited by 14
Monolingual and Multilingual Reduction of Gender Bias in Contextualized Representations
Liang, S., Dufter, P., & Schütze, H. (2020) · Open access LMU (Ludwid Maxmilian's Universitat Munchen) · doi:10.5282/ubm/epub.74040 · Cited by 5 · cited 1×
“For contextualized embeddings, similar methods to alleviate the issue of undesirable biases and toxicity have been proposed (Dev et al., 2020; Nangia et al., 2020; Nadeem et al., 2020; Krause et al., 2020; Liang et al., 2020; Kaneko and Bollegala, 2021a).”
MethodSelf-citationDictionary-based Debiasing of Pre-trained Word EmbeddingsPreprint
Kaneko, M., & Bollegala, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.09525 · Cited by 5
Artificial Intelligence and the Cultural Problem of Online Extreme Speech
Udupa, S. (2020) · Cited by 1
Self-citation
Research works printed but not linked: 1
- Wang, 2019, BERT has a mouth, and it must speak: BERT as a Markov random field language model, 30, 10.18653/v1/W19-2304 · doi:10.18653/v1/w19-2304
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey
Min, B., Ross, H., Sulem, E., Veyseh, A. P. B., Nguyen, T. H., Sainz, O., Agirre, E., Heintz, I., & Roth, D. (2024) · ACM Computing Surveys · doi:10.1145/3605943 · Cited by 1,387
“[168] showed that PLMs are able to recognize the toxicity of the text they produce (self-diagnosis).”
BackgroundBias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Using large language models in psychology
Demszky, D., Yang, D., Yeager, D. S., Bryan, C. J., Clapper, M., Chandhok, S., Eichstaedt, J. C., Hecht, C., Jamieson, J., Johnson, M., et al. (2023) · Nature Reviews Psychology · doi:10.1038/s44159-023-00241-5 · Cited by 369
“For example, researchers are developing approaches for LLMs to self-diagnose 59 and self-criticize based on principles written by people 60 to de-bias their output.”
BackgroundRed Teaming Language Models with Language Models
Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., & Irving, G. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.225 · Cited by 363
Metadata
- Authors
- Schick, T., Udupa, S., & Schütze, H.
- Affiliations
- Ludwig-Maximilians-Universität München (DE)
- Corresponding author
- Schick, Timo
- Udupa, Sahana
- Schütze, Hinrich
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2021
- Volume
- 9
- Pages
- 1408-1424
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W3134354193
- Cited by
- 227
- References
- 58
- Citations by year
- 2026: 29 · 2025: 40 · 2024: 36 · 2023: 70 · 2022: 36 · 2021: 16
- FWCI
- 13.96
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion, cc-by)
- Universitätsbibliothek der LMU
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- self-diagnosis, bias in language models, toxicity, bias detection, decoding algorithms, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.