Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP
Open accessThe DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Abstract
⚠ This paper contains prompts and model outputs that are offensive in nature. When trained on large, unfiltered crawls from the Internet, language models pick up and reproduce all kinds of undesirable biases that can be found in the data: They often generate racist, sexist, violent, or otherwise toxic language. As large models require millions of training examples to achieve good performance, it is difficult to completely prevent them from being exposed to such content. In this paper, we first demonstrate a surprising finding: Pretrained language models recognize, to a considerable degree, their undesirable biases and the toxicity of the content they produce. We refer to this capability as self-diagnosis. Based on this finding, we then propose a decoding algorithm that, given only a textual description of the undesired behavior, reduces the probability of a language model producing problematic text. We refer to this approach as self-debiasing. Self-debiasing does not rely on manually curated word lists, nor does it require any training data or changes to the model’s parameters. While we by no means eliminate the issue of language models generating biased text, we believe our approach to be an important step in this direction.1
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, it has limitations as it cannot be applied to downstream tasks.”
From Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models (2024) · cites this paper as Schick et al. (2021)
In the citing paper
“Importantly, this method does not compromise the language model and requires no additional training.”
From Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models (2024) · cites this paper as Schick et al. (2021)
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 16×
“…illustrating that both static (e.g., Mikolov et al., 2013; Bojanowski et al., 2017) and contextualized word embeddings (e.g., Peters et al., 2018; Devlin et al., 2019) pretrained in a self-supervised fashion exhibit all kinds of unfair and discriminative biases (Bolukbasi et al., 2016; Caliskan…”
BackgroundMethodInfluentialEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 1×
“There is a large body of work illustrating that both static (e.g., Mikolov et al., 2013; Bojanowski et al., 2017) and contextualized word embeddings (e.g., Peters et al., 2018; Devlin et al., 2019) pretrained in a self-supervised fashion exhibit all kinds of unfair and discriminative biases…”
BackgroundLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 1×
“…a large body of work illustrating that both static (e.g., Mikolov et al., 2013; Bojanowski et al., 2017) and contextualized word embeddings (e.g., Peters et al., 2018; Devlin et al., 2019) pretrained in a self-supervised fashion exhibit all kinds of unfair and discriminative biases (Bolukbasi et…”
BackgroundExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 13×
“Figure 1: Most probable continuations according to T5-3B (Raffel et al., 2020) and GPT2-XL (Radford et al.”
BackgroundMethodInfluentialEnriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645 · cited 1×
“There is a large body of work illustrating that both static (e.g., Mikolov et al., 2013; Bojanowski et al., 2017) and contextualized word embeddings (e.g., Peters et al., 2018; Devlin et al., 2019) pretrained in a self-supervised fashion exhibit all kinds of unfair and discriminative biases…”
BackgroundTransformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“Building training datasets with more care and deliberation, an alternative solution discussed by Bender et al. (2021), is important, especially for improving linguistic and cultural diversity in online and other forms of communication.”
Background
Research works printed but not linked: 1
- Wang, 2019, BERT has a mouth, and it must speak: BERT as a Markov random field language model, 30, 10.18653/v1/W19-2304 · doi:10.18653/v1/w19-2304
Cited by
Items 101–200 of 237
A First Look at Toxicity Injection Attacks on Open-domain Chatbots
Weeks, C., Cheruvu, A., Abdullah, S. M., Kanchi, S., Yao, D. D., & Viswanath, B. (2023) · Annual Computer Security Applications Conference · doi:10.1145/3627106.3627122 · Cited by 16
“Toxic language detection has been studied mainly for LLMs [14, 18, 20, 32, 38, 49], and there is limited work on mitigating toxicity in chatbots [7, 55, 64].”
BackgroundUnsupervised Neural Machine Translation with Generative Language Models OnlyPreprint
Han, J. M., Babuschkin, I., Edwards, H. S., Neelakantan, A., Xu, T., Polu, S., Ray, A., Shyam, P., Ramesh, A., Radford, A., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05448 · Cited by 45
“The role of few-shot prompting and distillation in our method is related to recent work on unsupervised data augmentation using language models (Anaby-Tavor et al., 2020; Schick et al., 2021; Kumar et al., 2020; Papanikolaou & Pierleoni, 2020; Schick & Schütze, 2021; Yang et al., 2020) and is also in the same spirit a…”
MethodLIBRA: Measuring Bias of Large Language Model from a Local Context
Pang, B., Qiao, T., Walker, C. G., Cunningham, C. W., & Koh, Y. S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88708-6_1 · Cited by 17
Semantic-Oriented Unlabeled Priming for Large-Scale Language Models
Liu, Y., Schick, T., & Schtze, H. (2023) · Workshop on Simple and Efficient Natural Language Processing (SustaiNLP) · doi:10.18653/v1/2023.sustainlp-1.2 · Cited by 19
Self-citationA Brief Survey on Safety of Large Language Models
Gao, Z., Liu, X., Lan, Y.-S., & Yang, Z. (2024) · Journal of Computing and Information Technology · doi:10.20532/cit.2024.1005778 · Cited by 15
“As LLMs require millions of training examples to achieve good performance, it is difficult to completely prevent them from being exposed to such content [37].”
A Communication Theory Perspective on Prompting Engineering Methods for Large Language Models
Song, Y., He, Y., Zhao, X.-F., Gu, H., Jiang, D., Yang, H., & Fan, L. (2024) · Journal of Computer Science and Technology · doi:10.1007/s11390-024-4058-8 · Cited by 14
Hate Speech Detection: Leveraging LLM-GPT2 with Fine-Tuning and Multi-Shot Techniques
Choudhary, M., Agarwal, B., & Goyal, V. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.04.542 · Cited by 12
Automating Bias Testing of LLMs
Morales, S., Clarisó, R., & Cabot, J. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00018 · Cited by 11
“An alternative approach, by Schick et al. [24], explicitly adds instructions in the prompt to ask the LLM not to generate a biased response.”
MethodSTILE: Exploring and Debugging Social Biases in Pre-trained Text Representations
Kabir, S., Li, L., & Zhang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642111 · Cited by 8
“Since the seminal work on gender bias in word embeddings [6], many techniques have been proposed to detect social biases in pre-trained text representations [12, 41, 45–47, 51, 55, 63].”
BackgroundSpeciesist language and nonhuman animal bias in English Masked Language Models
Takeshita, M., Rzepka, R., & Araki, K. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103050 · Cited by 14
“, 2020) and Self-debias (Schick et al., 2021), which do not require a specific word list, may be useful (cf.”
BackgroundOn Evaluating and Mitigating Gender Biases in Multilingual Settings
Vashishtha, A., Ahuja, K., & Sitaram, S. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.21 · Cited by 47
Security and Privacy of Large Language Models: Threat Taxonomy, Ethical Implications, and Governance
Pribisalić, M., & Martinčić-Ipšić, S. (2026) · AI · doi:10.3390/ai7050152 · Cited by 9
Harrag, F., Mahdadi, C., & Ziad, A. N. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-37249-0_13 · Cited by 7
Conceptor-Aided Debiasing of Large Language Models
Li, Y., Ungar, L., & Sedoc, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.661 · Cited by 14
“D EBIAS : by using specific templates to encourage LLMs to generate toxic output and then modifying the original output distribution of the model by a decoding algorithm, Schick et al. (2021) makes use of the internal knowledge of language model to debias in a post-hoc manner.”
MethodA Conceptual Framework for Quality Assurance of LLM-based Socio-critical Systems
Baresi, L., Camilli, M., Dolci, T., & Quattrocchi, G. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695306 · Cited by 14
“According to Schick et al. [26], these solutions fail to prevent language models from generating biased, unfair, or unethical text.”
DeSCoVeR: Debiased Semantic Context Prior for Venue Recommendation
Rajanala, S., Pal, A., Singh, M., Phan, R. C. ‐., & Wong, K. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531877 · Cited by 7
“However, they have been shown to be less effective [36] since train-dataset cleaning is a costly process, and it does not guarantee the non-susceptibility of the model when exposed to biased out-of-distribution data in the future.”
BackgroundDisclosure and Mitigation of Gender Bias in LLMs
Dong, X., Wang, Y., Yu, P. S., & Caverlee, J. (2024) · arXiv · doi:10.48550/arxiv.2402.11190 · Cited by 7
“Since our conditional generation probing method uncovers bias even in the absence of explicitly mentioned genders or stereotypes, existing meth-ods, which only mitigate bias when genders or stereotypes are present, are not effective in such cases (Sheng et al., 2020; Schick et al., 2021).”
BackgroundInjecting Bias through Prompts: Analyzing the Influence of Language on LLMs
Torres, N., Ulloa, C., Araya, I., Ayala, M., & Jara, S. (2024) · International Conference of the Chilean Computer Science Society (SCCC) · doi:10.1109/sccc63879.2024.10767653 · Cited by 8
“Schick et al. [13] showed that LLMs could be steered towards generating false or biased information through carefully designed prompts, raising concerns about the potential misuse of these models.”
ImageCaptioner2: Image Captioner for Image Captioning Bias Amplification Assessment
Abdelrahman, E., Sun, P., Li, L. E., & Elhoseiny, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30080 · Cited by 15
Attanasio, G., Plaza-del-Arco, F. M., Nozza, D., & Lauscher, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.243 · Cited by 37
Understanding the Effect of Model Compression on Social Bias in Large Language Models
Gonçalves, G., & Strubell, E. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.161 · Cited by 41
“The Self-Debias baseline was proposed by Schick et al. (2021), and uses prompts to encourage models to generate toxic text and learns to give less weight to the generate toxic tokens.”
MethodDANTE: Dialog graph enhanced prompt learning for conversational question answering over KGs
Li, J., Song, S., Yan, S., Hu, G., Lai, C., & Zhou, Y. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112294 · Cited by 5
Bias, machine learning, and conceptual engineering
Rudolph, R. E., Shech, E., & Tamir, M. (2025) · Philosophical Studies · doi:10.1007/s11098-024-02273-w · Cited by 7
“A second family was self-de-biasing (Schick et al., 2021), which leverages directed generation capabilities of LLMs to identify and actively reduce outputs generated from representational bias present in the pretrained model.”
Performance Analysis of IndoBERT for Detection of Online Gambling Promotion in YouTube Comments
Kamdan, K., Anugrah, M. P., Almutaali, M. J., Ramdani, R., & Kharisma, I. L. (2025) · International Global Conference Series on ICT Integration in Technical Education & Smart Society · doi:10.3390/engproc2025107066 · Cited by 5
Dong, X., Zhu, Z., Wang, Z., Teleki, M., & Caverlee, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.390 · Cited by 28
StereoMap: Quantifying the Awareness of Human-like Stereotypes in Large Language Models
Jeoung, S., Ge, Y., & Diesner, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.752 · Cited by 30
On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlook
Fan, M., Wang, C., Chen, C., Liu, Y., & Huang, J. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02375-w · Cited by 50
“Schick et al. [183] used LLMs themselves for diagnosis and debiasing purposes.”
The (Undesired) Attenuation of Human Biases by Multilinguality
España-Bonet, C., & Barrón‐Cedeño, A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.133 · Cited by 13
“Lots of work has been done on detecting social cultural biases and trying to mitigate them (Bolukbasi et al., 2016; Zhao et al., 2018; Gonen and Goldberg, 2019; Ravfogel et al., 2020; Dev et al., 2020; Schick et al., 2021; Zhou et al., 2022).”
BackgroundAdding Instructions during Pretraining: Effective way of Controlling Toxicity in Language Models
Prabhumoye, S., Patwary, M. A., Shoeybi, M., & Catanzaro, B. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.193 · Cited by 31
Self-Preference Bias in LLM-as-a-Judge
Wataoka, K., Takahashi, T., & Ri, R. (2024) · arXiv · doi:10.48550/arxiv.2410.21819 · Cited by 5
“LLMs are increasingly employed in dialogue evaluation due to their ability to provide flexible assessments from perspectives such as utility and safety [Zheng et al., 2024, Sottana et al., 2023, Wang et al., 2024, Schick et al., 2021].”
Jailbreaking large language models: models, origins, and evolution of attacks and defenses
Li, X. H., Wu, J., Zheng, Q., Wang, H., Fan, M. Y., Hu, S., Guo, J., & Ting, L. I. U. (2025) · Scientia Sinica Informationis · doi:10.1360/ssi-2024-0196 · Cited by 4
Aoyagui, P. A., Stemmler, K., Ferguson, S. A., Kim, Y., & Kuzminykh, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713248 · Cited by 18
“…[107], creative writing [29], or conflict resolution [99]), recent explorations focused on online content moderation [38, 44, 72], where AI, in general, [73], and LLMs specifically [71, 82, 98, 119], assist humans in identifying hate speech targeting specific groups or populations [3, 5, 27, 43].”
Wei, D., Chau, M., & Li, Z. (2025) · MIS Quarterly · doi:10.25300/misq/2025/18416 · Cited by 6
Slovene and Croatian word embeddings in terms of gender occupational analogies
Ulčar, M., Supej, A., Robnik‐Šikonja, M., & Pollak, S. (2021) · Slovenščina 2 0 empirične aplikativne in interdisciplinarne raziskave · doi:10.4312/slo2.0.2021.1.26-59 · Cited by 8
“Schick et al. (2021) recently proposed a self-diagnosis and self-debiasing model where large language models examine their outputs regarding the potential presence of undesirable attributes.”
BackgroundAdapting LLMs to Downstream Applications
Kucharavy, A. (2024) · Springer eBooks · doi:10.1007/978-3-031-54827-7_2 · Cited by 7
Legibility Diffuser: Offline Imitation for Intent Expressive Motion
Bronars, M., Cheng, S., & Xu, D. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3421849 · Cited by 12
Cantini, R., Cosenza, G., Orsino, A., & Talia, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-78977-9_4 · Cited by 23
“Mitigation strategies should involve curating more balanced and representative training datasets [31,33], while also implementing robust bias detection [32,36] and alignment mechanisms [40,41], incorporating fairness guidelines.”
BackgroundFairness Mediator: Neutralize Stereotype Associations to Mitigate Bias in Large Language Models
Xiao, Y., Liu, A., Liang, S., Liu, X., & Tao, D. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728881 · Cited by 16
“Drawing on adversarial attack techniques [28, 57], FairMed employs gradient-guided iteration to make precise adjustments for equal associations, unlike other approaches that rely on matrix projections [48, 73] or debiasing prompts [32, 74].”
InfluentialGPT3Mix: Leveraging Large-scale Language Models for Text AugmentationPreprint
Yoo, K. M., Park, D., Kang, J., Lee, S.-W., & Park, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08826 · Cited by 103
“Recent body of work has shown that handcrafted dictionaries can be employed to suppress the selection of offensive words (Gehman et al., 2020) and that language models can implicitly learn to identify biases through self-diagnosis, which can be exploited for self-debiasing (Schick et al., 2021).”
BackgroundRL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs
Akyürek, A. F., Akyürek, E., Madaan, A., Kalyan, A., Clark, P., Wijaya, D., & Tandon, N. (2023) · arXiv · doi:10.48550/arxiv.2305.08844 · Cited by 4
COBIAS: Assessing the Contextual Reliability of Bias Benchmarks for Language Models
Govil, P., Jain, H., Bonagiri, V. K., Chadha, A. R., Kumaraguru, P., Gaur, M., & Dey, S. (2025) · ACM Web Science Conference (WebSci) · doi:10.1145/3717867.3717923 · Cited by 13
“Existing works on mitigating LLM bias fall into one of two major categories – debiasing [22, 34, 57], and benchmarking [4, 43, 44, 52, 76].”
Leveraging Bias in Pre-Trained Word Embeddings for Unsupervised Microaggression Detection
Ògúnremí, T., Sabri, N., Basile, V., & Caselli, T. (2022) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.10749 · Cited by 3
PREADD: Prefix-Adaptive Decoding for Controlled Text Generation
Pei, J., Yang, K., & Klein, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.636 · Cited by 50
Trade-Offs Between Fairness and Privacy in Language Modeling
Matzken, C., Eger, S., & Habernal, I. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.434 · Cited by 10
Efficient Multilingual Language Model Compression through Vocabulary Trimming
Ushio, A., Zhou, Y. X., & Camacho-Collados, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.981 · Cited by 27
Measuring and Mitigating Stereotype Bias in Language Models: An Overview of Debiasing Techniques
Sokolová, Z., Harahus, M., Staš, J., Kupcová, E., Sokol, M., Koctúrová, M., & Juhár, J. (2024) · International Symposium ELMAR (ELMAR) · doi:10.1109/elmar62909.2024.10694175 · Cited by 6
“In the paper Schick et al. [21], the authors demonstrated that large language models (LLMs) can perform self-diagnosis, meaning they can assess their outputs for undesirable attributes using only their internal knowledge and textual descriptions.”
BackgroundRESPECT: A framework for promoting inclusive and respectful conversations in online communications
Raza, S., Muaad, A. Y. M., Hasan, E., Garg, M., Al-Zanbouri, Z., & Bashir, S. R. (2025) · Natural Language Processing Journal · doi:10.1016/j.nlp.2025.100126 · Cited by 5
Axolotl: Fairness through Assisted Prompt Rewriting of Large Language Model Outputs
Ebrahimi, S., Chen, K., Asudeh, A., Das, G., & Koudas, N. (2024) · IEEE International Conference on Knowledge Graph (ICKG) · doi:10.1109/ickg63256.2024.00017 · Cited by 4
“These methods include modifying the architecture and incorporating modules with a lower number of parameters into the layers of pre-trained language models (PLMs) (43). c) Post-training: Schick et al.(26) demonstrates that language models are aware of their biases and can recognize undesired behaviors, such as biases…”
InfluentialIRepair: An Intent-Aware Approach to Repair Data-Driven Errors in Large Language Models
Imtiaz, S. M., Singh, A., Batole, F., & Rajan, H. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715775 · Cited by 5
Cyberbullying Detection on Social Media
Huang, H., & QI, D. (2023) · Higher Education and Oriental Studies · doi:10.54435/heos.v3i1.95 · Cited by 3
Foveate, Attribute, and Rationalize: Towards Physically Safe and Trustworthy AI
Mei, A., Levy, S., & Wang, W. Y. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.701 · Cited by 10
“…further inference steps to reach physical harm such as hate speech and cyberbullying (Jurgens et al., 2019; Xu et al., 2012; Chatza-kou et al., 2019; Breitfeller et al., 2019; Schick et al., 2021; Dinan et al., 2022b; Kiritchenko et al., 2021; Schmidt and Wiegand, 2017; Salawu et al., 2020).”
BackgroundTargeting the Source: Selective Data Curation for Debiasing NLP Models
Gaci, Y., Benatallah, B., Casati, F., & Benabdeslem, K. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43415-0_17 · Cited by 7
Towards Agile Text Classifiers for Everyone
Mozes, M., Hoffmann, J., Tomanek, K., Kouate, M., Thain, N., Yuan, A., Bolukbasi, T., & Dixon, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.30 · Cited by 20
Villate-Castillo, G., Ser, J. D., & Sanz, B. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11548-z · Cited by 16
Fine-grained detoxification framework via instance-level prefixes for large language models
Xin, Y., Wang, L., Wang, X., & He, L. J. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128684 · Cited by 11
Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generation
Zhang, Z., Cheng, J., Sun, H., Deng, J., Mi, F., Wang, Y., Shang, L., & Huang, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.270 · Cited by 18
Wacker, T., Kreppein, Y., & Lanquillon, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93418-6_14 · Cited by 2
You, J., Lee, J., Lee, S., & Kwon, H.-Y. (2026) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3773966.3777990 · Cited by 4
Detoxifying Language Models with a Toxic Corpus
Park, Y., & Rudzicz, F. (2022) · Workshop on Language Technology for Equality, Diversity, Inclusion (LT-EDI) · doi:10.18653/v1/2022.ltedi-1.6 · Cited by 9
From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models
Li, L., Lin, Y., Ren, X., Zhao, G., Li, P., Zhou, J., & Sun, X. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.477 · Cited by 2
FedPerC: Federated Learning for Language Generation with Personal and Context Preference Embeddings
Silva, A., Tambwekar, P., & Gombolay, M. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.64 · Cited by 7
“While this problem is certainly not unique to FedPerC, it is important to consider the safety and fairness implications of improved language generation, and future work must address biases inherent to large language models (Schick et al., 2021; Ravfogel et al., 2020).”
BackgroundMehrotra, S., Raghavendra, P., Prajesh, B., Kambale, H., & Majumdar, P. (2025) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3768292.3770355 · Cited by 7
“Post-processing techniques—like bias detection filters and rephrasing algorithms modify outputs in real time to reduce bias [26].”
Xu, Q., Tian, Z., Wu, H., Huang, Z., Liang, D., Song, Y., Pan, Z., Liu, F., & Li, D. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3626228 · Cited by 5
“Some research designs templates to filter sensitive words [23], [35].”
Mitigating Covertly Unsafe Text within Natural Language Systems
Mei, A., Kabir, A., Levy, S., Subbiah, M., Allaway, E., Judge, J., Patton, D. U., Bimber, B., McKeown, K. R., & Wang, W. Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.211 · Cited by 13
Know your audience: specializing grounded language models with listener subtraction
Singh, A. K., Ding, D., Saxe, A., Hill, F., & Lampinen, A. K. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.279 · Cited by 7
“could be debiasing the underlying (frozen) language model (Schick et al., 2021; Meade et al., 2022), as the speaker’s propensity for producing offensive language is inherited from this underlying model.”
BackgroundRevisiting Queer Minorities in Lexicons
Ramesh, K., Kumar, S., & KhudaBukhsh, A. R. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.23 · Cited by 2
“…and practices on censoring (or not censoring) historically charged words (Cannon, 2005; Stephens-Davidowitz and Pabon, 2017; Sap et al., 2020; Schick et al., 2021). from true experts on this issue that may significantly influence how a safe web may look like for sexual minorities in the…”
BackgroundDon’t Just Clean It, Proxy Clean It: Mitigating Bias by Proxy in Pre-Trained Models
Panda, S., Kobren, A., Wick, M., & Shen, Q. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.372 · Cited by 19
“Prior work on de-biasing pre-trained language models largely focus on upstream mitigation of intrinsic biases (Meade et al., 2022; Kaneko and Bollegala, 2021; Schick et al., 2021).”
BackgroundComparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examples
Kaneko, M., Bollegala, D., & Okazaki, N. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.209 · Cited by 14
Inferring Offensiveness In Images From Natural Language SupervisionPreprint
Schramowski, P., & Kersting, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04222 · Cited by 4
“However, Schick et al. (2021) demonstrated that these models can recognize, to a considerable degree, their undesirable retained knowledge and the toxicity of the content they produce.”
BackgroundZhou, Y. X., Camacho-Collados, J., & Bollegala, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.683 · Cited by 15
A Meta-thinking Approach to Mitigating Linguistic Sycophancy in Vision-Language Models
Hoang, C., Roberts, N., & Hasan, M. R. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8298-0_30 · Cited by 3
PG-CE: A Progressive Generation Dataset with Constraint Enhancement for Controllable Text Generation
Zhuang, Y., & Sun, Y. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228294 · Cited by 1
“…Utilizes expert and anti-expert models, combining their token predictions through weighted probability aggregation c) Prompt-based Methods: • SD [31]: Guides generation through specific prompt templates, comparing outputs with and without prompts to filter undesired content • REI [32]:…”
Chand, S., Baca, F., & Ferrara, E. (2026) · AI · doi:10.3390/ai7010024 · Cited by 9
Mei, Y., Duan, Y., & Nguyen, H. D. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.115752 · Cited by 1
A review of fairness challenges in natural language processing
Zadid, T. H., Jahin, S. M., Dhruba, A. R., Tanim, S. A., Hamja, A. M., Hasan, M. R., Miah, M. S. U., Islam, M. I., & Mansur, A. A. (2026) · Array · doi:10.1016/j.array.2026.100773 · Cited by 4
Akbulut, M. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.104899 · Cited by 1
Online Domain-Aware LLM Decoding for Continual Domain Evolution
Abu-Shaira, M., & Shi, W. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-1468-6_40 · Cited by 2
“Auxiliary controllers (e.g., PPLM [7], GeDi [13]) rely on external models, while internal methods ( DExperts [19], Self-Debiasing [29], FUDGE [33]) use contrastive or self-generated signals.”
Hu, J., Li, Y., Song, R., Lyu, K., & Chang, Y. (2026) · Artificial Intelligence · doi:10.1016/j.artint.2026.104625 · Cited by 1
Leveraging Bias in Pre-trained Word Embeddings for Unsupervised Microaggression Detection
Ògúnremí, T., Basile, V., & Caselli, T. (2022) · Italian Journal of Computational Linguistics · doi:10.4000/ijcol.1066 · Cited by 1
An Invariant Learning Characterization of Controlled Text Generation
Zheng, C., Shi, C., Vafa, K., Feder, A., & Blei, D. M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.179 · Cited by 14
Morabito, R., Kabbara, J., & Emami, A. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.280 · Cited by 10
Analyzing the Limits of Self-Supervision in Handling Bias in Language
Bauer, L., Gopalakrishnan, K., Gella, S., Liu, Y., Bansal, M., & Hakkani-Tür, D. Z. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.545 · Cited by 3
Chintam, A., Beloch, R., Zuidema, W. H., Hanna, M., & van der Wal, O. (2023) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2023.blackboxnlp-1.29 · Cited by 29
Societal Versus Encoded Stereotypes in Text Encoders
Gaci, Y., Benatallah, B., Casati, F., & Benabdeslem, K. (2023) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai59109.2023.00015 · Cited by 2
“…count: finetuning with specific fairness objectives [48, 7, 40, 29, 22], using adapters [37], Counterfactual Data Augmentation (CDA) [54] to balance group mentions in training data, dropout [43], zero-shot learning [51], contrastive learning [9], or regularizing the entropy of attention weights [3].”
MethodAll Should Be Equal in the Eyes of LMs: Counterfactually Aware Fair Text Generation
Banerjee, P., Java, A., Jandial, S., Shahid, S., Furniturewala, S., Krishnamurthy, B., & Bhatia, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29719 · Cited by 10
“To address these challenges, a much more practical alternative is to perform inference time transformations to the output probabilities (Schick, Udupa, and Sch¨utze 2021; Hallinan et al. 2023) or consider common prompt-based interventions (Chain-of-Thought reasoning (Kojima et al. 2023) and…”
BackgroundMethodInfluentialFrom Bias to Fairness: The Role of Domain-Specific Knowledge and Efficient Fine-Tuning
Zheng, W., Yang, A., Lin, N., & Zhou, D. M. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5669-8_29 · Cited by 1
Debiasing Low-Resource Language Models Via Cross-Lingual Transfer Learning
Chauhan, A. (2024) · IEEE MIT Undergraduate Research Technology Conference (URTC) · doi:10.1109/urtc65039.2024.10937594 · Cited by 1
“Similarly, self-debiasing leverages a model’s ability to identify and adjust for its own biases without additional training or parameter changes, using self-diagnosis prompts to gauge potential bias and adjust accordingly [13].”
Overview of NLPCC 2022 Shared Task 7: Fine-Grained Dialogue Social Bias Measurement
Zhou, J., Mi, F., Meng, H. M. L., & Deng, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17189-5_31 · Cited by 3
Plot Writing From Pre-Trained Language Models
Jin, Y., Kadam, V., & Wanvarie, D. (2022) · International Conference on Natural Language Generation · doi:10.18653/v1/2022.inlg-main.5 · Cited by 7
Suicide Prevention Technologies and Social Media Platforms: Legal, Social and Ethical ImplicationsPreprint
Stephenson, A., Cohen, G. A., Beller, Y., & Greenbaum, D. S. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.4194066 · Cited by 2
Towards Unified Objectives for Self-Reflective AIPreprint
Samwald, M., Praas, R., & Hebenstreit, K. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4446991 · Cited by 3
Plot Writing From Pre-Trained Language Models
Jin, Y., Kadam, V., & Wanvarie, D. (2022) · arXiv · doi:10.48550/arxiv.2206.03021 · Cited by 1
“We apply self-debiasing (Schick et al., 2021) to generate distinct geographical units and male/female names using each task description.”
MethodThe Presence of Bias Based on Pre-trained Language Models
Liu, R., Wu, Y., & Xing, Y. (2023) · Applied and Computational Engineering · doi:10.54254/2755-2721/8/20230250
Performance and Risk Trade-offs for Multi-word Text Prediction at Scale
Vashishtha, A., Prasad, S. S., Bajaj, P., Chaudhary, V., Cook, K., Dandapat, S., Sitaram, S., & Choudhury, M. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.167 · Cited by 6
“…that propose techniques for measurement and mitigation of biases of LLMs (Raffel et al., 2020; NLLB Team et al., 2022; Geva et al., 2022; Dathathri et al., 2019; Schick et al., 2021; Lu et al., 2022), there are very few that analyze such harms in context of real-world downstream applications.”
BackgroundInfluentialAutoReply: Detecting Nonsense in Dialogue with Discriminative Replies
Shi, W., Dinan, E., Renduchintala, A., Fried, D., Jacob, A. P., Zhou, Y., & Lewis, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.23 · Cited by 1
Multilingual Fairness Analysis and Research based on Pre-Trained Models
(2023) · Journal of Innovation and Social Science Research · doi:10.53469/jissr.2023.10(12).07
DialCL-Bias: A Semantic and Contextual Framework to Identify Social Bias in Dialogue
Cai, Z., Wu, D., & Li, P. (2023) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53992.2023.10394325
“Shick [14] argues that pre-trained language models have been successful in identifying undesirable biases and toxic content in text classification tasks.”
BackgroundStudent-Teacher Prompting for Red Teaming to Improve Guardrails
Llaca, R. R., Leskoschek, V., Paiva, V. C., Lupău, C., Lippmann, P., & Yang, J. (2023) · doi:10.18653/v1/2023.artofsafety-1.2 · Cited by 5
“…correct this undesired behaviour, various strategies are employed, such as removing content associated with specific key-words from the training data (Schick et al., 2021) or employing red-teaming to simulate adversarial scenarios for refining the model (Perez et al., 2022a; Ganguli et al., 2022).”
MethodEfficient Detection of Generative Chinese Statements Using Feature Compression Methods
Gao, Y., Huang, T., Cai, N., & Yang, D. (2024) · International Conference on Mobile Internet, Cloud Computing and Information Security (MICCIS) · doi:10.1109/miccis63508.2024.00014
“Prompt-based Self-Detection (PSELFDET) the model utilizes its internal knowledge acquired during pre-training to identify offensive language[8][9].”
Method
Metadata
- Authors
- Schick, T., Udupa, S., & Schütze, H.
- Affiliations
- Ludwig-Maximilians-Universität München (DE)
- Corresponding author
- Schick, Timo
- Udupa, Sahana
- Schütze, Hinrich
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2021
- Volume
- 9
- Pages
- 1408-1424
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W3134354193
- Cited by
- 227
- References
- 58
- Citations by year
- 2026: 29 · 2025: 40 · 2024: 36 · 2023: 70 · 2022: 36 · 2021: 16
- FWCI
- 13.96
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion, cc-by)
- Universitätsbibliothek der LMU
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- self-diagnosis, bias in language models, toxicity, bias detection, decoding algorithms, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.