The DOI matches its Crossref record; 71 of the 74 references it cites were found in a registry or an index.
Debiasing Pretrained Text Encoders by Paying Attention to Paying Attention
Open accessAbstract
Natural Language Processing (NLP) models are found to exhibit discriminatory stereotypes across many social constructs, e.g.gender and race.In comparison to the progress made in reducing bias from static word embeddings, fairness in sentence-level text encoders received little consideration despite their wider applicability in contemporary NLP tasks.In this paper, we propose a debiasing method for pretrained text encoders that both reduces social stereotypes, and inflicts next to no semantic damage.Unlike previous studies that directly manipulate the embeddings, we suggest to dive deeper into the operation of these encoders, and pay more attention to the way they pay attention to different social groups.We find that stereotypes are also encoded in the attention layer.Then, we work on model debiasing by redistributing the attention scores of a text encoder such that it forgets any preference to historically advantaged groups, and attends to all social classes with the same intensity.Our experiments confirm that reducing bias from attention effectively mitigates it from the model's text representations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“understanding that attention mechanisms play a key role in encoding bias informs attention-targeting mitigations such as Attanasio et al. (2022), Gaci et al. (2022), and Zayed et al. (2023a).”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Gaci et al. (2022)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
Knowledge Distillation: A Survey
Gou, J., Yu, B., Maybank, S. J., & Tao, D. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01453-z · Cited by 3,931
Flagged first, then by citations
3 of the 74 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to home- maker? debiasing word embeddings. Advances in neural information processing systems, 29:4349- 4357.
- C Hansen, M Tosik, G Goossen, C Li, L Bayeva, F Berbain, and M Rotaru. 2015. How to get the best word vectors for resume parsing. In SNN Adaptive Intelligence/Symposium: Machine Learning.
- Paul Pu Liang, Irene Mengze Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, and Louis- Philippe Morency. 2020a. Towards debiasing sen- tence representations. In Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, pages 5502-5515.
- Zachary C Lipton. 2018. The mythos of model inter- pretability: In machine learning, the concept of in- terpretability is both important and slippery. Queue, 16(3):31-57.
- Jesse Vig. 2019a. Bertviz: A tool for visualizing multi- head self-attention in the bert model. In Proceedings of the 2019 ICLR Workshop on Debugging Machine Learning Models.
Cited by
Sorted by citations
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
ROUGE-SEM: Better evaluation of summarization using ROUGE combined with semantics
Zhang, M., Li, C., Wan, M., Zhang, X., & Zhao, Q. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121364 · Cited by 61
Deng, C., Duan, Y., Jin, X., Chang, H., Tian, Y., Liu, H., Wang, Y., Gao, K., Zou, H. P., Jin, Y., et al. (2025) · AI and Ethics · doi:10.1007/s43681-025-00797-3 · Cited by 72
Debiasing large language models: research opportunities*
Yogarajan, V., Dobbie, G., & Keegan, T. T. (2024) · Journal of the Royal Society of New Zealand · doi:10.1080/03036758.2024.2398567 · Cited by 18
Exploring the occupational biases and stereotypes of Chinese large language models
Jiang, L., Zhu, G., Sun, J., Cao, J., & Wu, J. X. (2025) · Scientific Reports · doi:10.1038/s41598-025-03893-w · Cited by 9
Metadata
- Authors
- Gaci, Y., Benatallah, B., Casati, F., & Benabdeslem, K.
- Affiliations
- Lyon 1 Université (FR)
- UNSW Sydney (AU)
- ServiceNow (United States) (US)
- Venue
- Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
- Published
- 2022
- Pages
- 9582-9602
- OpenAlex
- W4318046968
- Cited by
- 14
- References
- 74
- Citations by year
- 2025: 5 · 2024: 3 · 2023: 6
- FWCI
- 3.20
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Iris (University of Trento) (submittedVersion, cc-by-nc-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- bias mitigation, attentional bias, social stereotypes, gender bias, racial bias, fairness, text representation, transformer models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
