The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Evaluating the Underlying Gender Bias in Contextualized Word Embeddings
Open accessAbstract
Gender bias is highly impacting natural language processing applications.Word embeddings have clearly been proven both to keep and amplify gender biases that are present in current data sources.Recently, contextualized word embeddings have enhanced previous word embedding techniques by computing word vector representations dependent on the sentence they appear in.In this paper, we study the impact of this conceptual change in the word embedding computation in relation with gender bias.Our analysis includes different measures previously applied in the literature to standard word embeddings.Our findings suggest that contextualized word embeddings are less biased than standard ones even when the latter are debiased.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Text suggestions based on language models have been known to inherit biases and dominant 'views' from the text corpus used to train the language model [22,39,43,55].”
From Interacting with Next-Phrase Suggestions: How Suggestion Systems Aid and Influence the Cognitive Processes of Writing (2023) · cites this paper as [22]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
UMAP: Uniform Manifold Approximation and Projection
McInnes, L., Healy, J., Saul, N., & Großberger, L. (2018) · Journal of Open Source Software · doi:10.21105/joss.00861 · Cited by 9,826 · cited 1×
“The clusters (for one particular experiment out of the 10 of them) are shown in Figure 2 after applying UMAP (McInnes et al., 2018; McInnes et al., 2018) to the contextualized embeddings.”
MethodUMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
McInnes, L., Healy, J., & Melville, J. (2018) · arXiv · doi:10.48550/arxiv.1802.03426 · Cited by 7,101
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100
Research works printed but not linked: 2
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013. Distributed representa- tions of words and phrases and their composition- ality. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Ad- vances in Neural Information Processing Systems 26, pages 3111-3119.
- Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies, Volume 1 (Long Papers), pages 2227-2237, New Orleans, Louisiana. Association for Computational Linguistics.
Cited by
Sorted by citations
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Language (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050
“Surveys, frameworks, and meta-analyses Hovy and Spruit (2016); Larson (2017); McCurdy and Serbetçi (2017); Schnoebelen (2017); Basta et al. (2019); Ethayarajh et al. (2019); Gonen and Goldberg (2019); Lauscher and Glavaš (2019); Loukina et al. (2019); Mayfield et al. (2019); Mirzaev et al. (2019);…”
BackgroundLund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24750 · Cited by 919
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
Metadata
- Authors
- Basta, C., Costa-jussà, M. R., & Casas, N.
- Affiliations
- Universitat Politècnica de Catalunya (ES)
- ORCID
- Basta, Christine
- Venue
- Proceedings of the First Workshop on Gender Bias in Natural Language Processing
- Published
- 2019
- Pages
- 33-39
- OpenAlex
- W2972668795
- Cited by
- 189
- References
- 25
- Citations by year
- 2026: 12 · 2025: 25 · 2024: 20 · 2023: 41 · 2022: 32 · 2021: 23 · 2020: 26 · 2019: 10
- FWCI
- 23.69
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Workshop on Gender Bias in Natural Language Processing (GeBNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- UPCommons institutional repository (Universitat Politècnica de Catalunya) (submittedVersion, public-domain)
- arXiv (Cornell University)
- Funders
- Agence Nationale de la Recherche (ANR-17-CHR2-0004)
- Ministerio de Economía y Competitividad (TEC2015-69266-P)
- Agència de Gestió d'Ajuts Universitaris i de Recerca
- European Regional Development Fund (TEC2015-69266-P)
- Agencia Estatal de Investigación
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Gender Studies › Gender Studies in Language
- Keywords
- gender bias, contextual word embeddings, word embeddings
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
