CoChat
Sign Up
PreprintCited by 84Ethics and Social Impacts of AITopic ModelingHate Speech and Cyberbullying Detection

Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias

Open access
Bartl, Marion, Nissim, Malvina, Gatt, Albert·2020·arXiv
DOI 10.48550/arxiv.2010.14534arXiv 2010.14534
Open this paper in CoChat — free accountOpen PDF

Abstract

Contextualized word embeddings have been replacing standard embeddings as the representational knowledge source of choice in NLP systems. Since a variety of biases have previously been found in standard word embeddings, it is crucial to assess biases encoded in their replacements as well. Focusing on BERT (Devlin et al., 2018), we measure gender bias by studying associations between gender-denoting target words and names of professions in English and German, comparing the findings with real-world workforce statistics. We mitigate bias by fine-tuning BERT on the GAP corpus (Webster et al., 2018), after applying Counterfactual Data Substitution (CDS) (Maudslay et al., 2019). We show that our method of measuring bias is appropriate for languages such as English, but not for languages with a rich morphology and gender-marking, such as German. Our results highlight the importance of investigating bias and mitigation techniques cross-linguistically, especially in view of the current emphasis on large-scale, multilingual language models.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2010.14534

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • BEC-Pro measures gender biases with respect to occupations, with 5,400 sentences constructed from templates containing a person word and one of 60 profession terms.

    “Bias Evaluation Corpus with Professions (BEC-Pro) (Bartl, Nissim, and Gatt 2020) measures gender biases with respect to occupations, with 5,400 sentences constructed from templates containing a person word and one of 60 profession terms.”

    From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Bartl, Nissim, and Gatt 2020

    Section 3.2 (page unavailable) confirms that BEC-Pro uses templates with a person word and a profession term, drawing on three groups of 20 professions. It reports 5,400 sentences for each language, English and German, rather than 5,400 for the bilingual corpus as a whole. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • For BEC-Pro evaluation, person and profession words are masked.

    “For evaluation, person and profession words are masked.”

    From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Bartl, Nissim, and Gatt 2020

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 25

  1. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of "bias" in NLP. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 5454– 5476, Online. Association for Computational Lin- guistics.
  2. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186.
  3. Marta R. Costa-jussà, Christian Hardmeier, Will Rad- ford, and Kellie Webster, editors. 2019. Proceed- ings of the First Workshop on Gender Bias in Natu- ral Language Processing. Association for Computa- tional Linguistics, Florence, Italy.
  4. Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. arXiv preprint arXiv:1903.03862. · arXiv
  5. Hila Gonen, Yova Kementchedjhieva, and Yoav Gold- berg. 2019. How does grammatical gender affect noun representations in gender-marking languages? arXiv preprint arXiv:1910.14161. · arXiv
  6. Dirk Hovy, Shannon Spruit, Margaret Mitchell, Emily M. Bender, Michael Strube, and Hanna Wal- lach, editors. 2017. Proceedings of the First ACL Workshop on Ethics in Natural Language Process- ing. Association for Computational Linguistics, Va- lencia, Spain.
  7. Dirk Hovy and Shannon L Spruit. 2016b. The social impact of natural language processing. In Proceed- ings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Pa- pers), pages 591–598.
  8. Svetlana Kiritchenko and Saif M Mohammad. 2018. Examining gender and race bias in two hun- dred sentiment analysis systems. arXiv preprint arXiv:1805.04508. · arXiv
  9. Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Measuring bias in con- textualized word representations. arXiv preprint arXiv:1906.07337. · arXiv
  10. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learn- ing of language representations. arXiv preprint arXiv:1909.11942. · arXiv
  11. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv:1907.11692. · arXiv
  12. Rowan Hall Maudslay, Hila Gonen, Ryan Cotterell, and Simone Teufel. 2019. It's all in the name: Mit- igating gender bias with name-based counterfactual data substitution. arXiv preprint arXiv:1909.00871. · arXiv
  13. Corinne A Moss-Racusin, John F Dovidio, Victoria L Brescoll, Mark J Graham, and Jo Handelsman. 2012. Science faculty's subtle gender biases favor male stu- dents. Proceedings of the national academy of sci- ences, 109(41):16474–16479.
  14. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word repre- sentations. arXiv preprint arXiv:1802.05365. · arXiv
  15. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8):9.
  16. Robert Rosenthal. 1991. Applied social research meth- ods series, Vol. 6. Meta-analytic procedures for so- cial research (Rev. ed.). Sage Publications, Inc. https://doi. org/10.4135/9781412984997. · DOI · Paper page
  17. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. arXiv preprint arXiv:1804.09301. · arXiv
  18. Deven Shah, H Andrew Schwartz, and Dirk Hovy. 2019. Predictive biases in natural language process- ing models: A conceptual framework and overview. arXiv preprint arXiv:1912.11078. · arXiv
  19. Paul Stocker. 2012. Nouns. In A Student Grammar of German, page 11–29. Cambridge University Press.
  20. Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Eliza- beth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural lan- guage processing: Literature review. arXiv preprint arXiv:1906.08976. · arXiv
  21. Kellie Webster, Marta Recasens, Vera Axelrod, and Ja- son Baldridge. 2018. Mind the gap: A balanced cor- pus of gendered ambiguous pronouns. In Transac- tions of the ACL, page to appear.
  22. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, R'emi Louf, Morgan Funtow- icz, and Jamie Brew. 2019. Huggingface's trans- formers: State-of-the-art natural language process- ing. ArXiv, abs/1910.03771. · arXiv
  23. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cot- terell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. arXiv preprint arXiv:1904.03310. · arXiv
  24. Pei Zhou, Weijia Shi, Jieyu Zhao, Kuan-Hao Huang, Muhao Chen, Ryan Cotterell, and Kai- Wei Chang. 2019. Examining gender bias in lan- guages with grammatical gender. arXiv preprint arXiv:1909.02224. · arXiv
  25. Ran Zmigrod, Sebastian J Mielke, Hanna Wallach, and Ryan Cotterell. 2019. Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology. arXiv preprint arXiv:1906.04571. · arXiv

Cited by

Sorted by citations

Authors
Bartl, M., Nissim, M., & Gatt, A.
Affiliations
University of Groningen (NL)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
Pages
1-16
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2010.14534
arXiv
2010.14534
OpenAlex
W3095105395
Cited by
84
References
9
Citations by year
2026: 5 · 2025: 14 · 2024: 11 · 2023: 19 · 2022: 27 · 2021: 8
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, other-oa)
  • arXiv (Cornell University) (cc-by)
Funders
Rijksuniversiteit Groningen
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
Keywords
gender bias, BERT, contextual word embeddings, gender stereotypes, bias mitigation, gendered occupations, multilingual language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review