CoChat

Investigating Gender Bias in BERT

Open access
Bhardwaj, Rishabh, Majumder, Navonil, Poria, Soujanya · 2021 · Cognitive Computation

The DOI matches its Crossref record; 28 of the 29 references it cites were found in a registry or an index, and 1 are not checked yet.

Abstract

In this work, we analyze the gender bias induced by BERT in downstream tasks. We also propose solutions to reduce gender bias. Contextual language models (CLMs) have pushed the NLP benchmarks to a new height. It has become a new norm to utilize CLM-provided word embeddings in downstream tasks such as text classification. However, unless addressed, CLMs are prone to learn intrinsic gender bias in the dataset. As a result, predictions of downstream NLP models can vary noticeably by varying gender words, such as replacing “he” to “she”, or even gender-neutral words. In this paper, we focus our analysis on a popular CLM, i.e., BERT\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\text {BERT}$$\end{document}. We analyze the gender bias it induces in five downstream tasks related to emotion and sentiment intensity prediction. For each task, we train a simple regressor utilizing BERT\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\text {BERT}$$\end{document}’s word embeddings. We then evaluate the gender bias in regressors using an equity evaluation corpus. Ideally and from the specific design, the models should discard gender informative features from the input. However, the results show a significant dependence of the system’s predictions on gender-particular words and phrases. We claim that such biases can be reduced by removing gender-specific features from word embedding. Hence, for each layer in BERT, we identify directions that primarily encode gender information. The space formed by such directions is referred to as the gender subspace in the semantic space of word embeddings. We propose an algorithm that finds fine-grained gender directions, i.e., one primary direction for each BERT layer. This obviates the need of realizing gender subspace in multiple dimensions and prevents other crucial information from being omitted. Experiments show that removing embedding components in gender directions achieves great success in reducing BERT-induced bias in the downstream tasks. The investigation reveals significant gender bias a contextualized language model ( i.e., BERT\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\text {BERT}$$\end{document}) induces in downstream tasks. The proposed solution seems promising in reducing such biases.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s12559-021-09881-2

Cites

28 of 29 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 29 references

Research works printed but not linked: 6

  1. Hussain A, Tahir A, Hussain Z, Sheikh Z, Gogate M, Dashtipour K, Ali A, Sheikh A. Artificial intelligence-enabled analysis of public attitudes on facebook and twitter toward covid-19 vaccines in the united kingdom and the united states: Observational study. J Med Intern Res. 2021;23(4):e26627.
  2. Caliskan A, Bryson JJ, Narayanan A. Semantics derived automatically from language corpora contain human-like biases. Science. 2017;356(6334):183–6. · DOI · Paper page
  3. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V. Roberta: A robustly optimized bert pretraining approach. 2019. https://arxiv.org/abs/1907.11692. · arXiv
  4. Morcos A, Raghu M, Bengio S. Insights on representational similarity in neural networks with canonical correlation. In Advances in Neural Information Processing Systems. 2018:5727–5736.
  5. Morcos A, Raghu M, Bengio S. Insights on representational similarity in neural networks with canonical correlation. In Advances in Neural Information Processing Systems. 2018:5727–5736.
  6. Zhao J, Zhou Y, Li Z, Wang W, Chang KW. Learning gender-neutral word embeddings. 2018. https://arxiv.org/abs/1809.01496. · DOI · Paper page

Cited by

See all 112

Metadata

Authors
Bhardwaj, R., Majumder, N., & Poria, S.
Affiliations
Singapore University of Technology and Design (SG)
Corresponding author
Poria, Soujanya
Venue
Cognitive Computation
Published
2021-05-20
Volume
13
Issue
4
Pages
1008-1018
ISSN
  • 1866-9956 (ISSN-L)
  • 1866-9964
OpenAlex
W3084269450
Cited by
112
References
27
Citations by year
2026: 8 · 2025: 21 · 2024: 21 · 2023: 25 · 2022: 24 · 2021: 11
FWCI
25.35
Type
journal article
Language
en
Open access
Licence
Locations
  • Cognitive Computation (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
Keywords
gender bias, BERT, contextual language models, word embeddings, bias mitigation, emotion prediction, equity evaluation, downstream tasks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Investigating Gender Bias in BERT. https://app.cochat.ai/papers/10.1007/s12559-021-09881-2
The paper
Bhardwaj, R., Majumder, N., & Poria, S. (2021). Investigating Gender Bias in BERT. Cognitive Computation. https://doi.org/10.1007/s12559-021-09881-2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.