Quantifying gender bias towards politicians in cross-lingual language models
Open accessAbstract
Recent research has demonstrated that large pre-trained language models reflect societal biases expressed in natural language. The present paper introduces a simple method for probing language models to conduct a multilingual study of gender bias towards politicians. We quantify the usage of adjectives and verbs generated by language models surrounding the names of politicians as a function of their gender. To this end, we curate a dataset of 250k politicians worldwide, including their names and gender. Our study is conducted in seven languages across six different language modeling architectures. The results demonstrate that pre-trained language models’ stance towards politicians varies strongly across analyzed languages. We find that while some words such as dead , and designated are associated with both male and female politicians, a few specific words such as beautiful and divorced are predominantly associated with female politicians. Finally, and contrary to previous findings, our study suggests that larger language models do not tend to be significantly more gender-biased than smaller ones.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Gender bias manifests across diverse LLM contexts.
“Gender bias manifests across diverse LLM contexts, including how LLMs describe politicians [81], generate product descriptions [48], and respond to adversarial prompts [54].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [81]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,037 · see citing works
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195 · see citing works
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1× · see citing works
“However, alongside capturing the linguistic nuances of a language, they also perpetuate (and potentially amplify) biases (Zhao et al., 2017; Bender et al., 2021).”
BackgroundXLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · see citing works
Unsupervised Cross-lingual Representation Learning at Scale
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.747 · Cited by 5,637 · see citing works
(1998) · The MIT Press eBooks · doi:10.7551/mitpress/7287.001.0001 · Cited by 4,849 · see citing works
Vrandečić, D., & Krötzsch, M. (2014) · Communications of the ACM · doi:10.1145/2629489 · Cited by 3,367 · see citing works
Learning deep structured semantic models for web search using clickthrough data
Huang, P.-S., He, X., Gao, J., Deng, L., Acero, A., & Heck, L. P. (2013) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2505515.2505665 · Cited by 2,054 · see citing works
A primer in BERTology: What we know about how BERT works
Rogers, A., Kovaleva, O., & Rumshisky, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00349 · Cited by 1,552 · see citing works
Estimation of entropy and mutual information
Paninski, L. (2003) · Neural Computation · doi:10.1162/089976603321780272 · Cited by 1,442 · see citing works
Proceedings of the 24th annual meeting on Association for Computational Linguistics -
Biermann, A. W. (1986) · doi:10.3115/981131 · Cited by 1,083 · see citing works
BERTweet: A pre-trained language model for English Tweets
Nguyen, D. Q., Vu, T., & Nguyen, A. T. (2020) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2020.emnlp-demos.2 · Cited by 949 · see citing works
Ambati, B. R., Deoskar, T., Johnson, M., & Steedman, M. J. (2015) · The Association for Computational Linguistics · Cited by 860 · see citing works
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1323 · Cited by 829 · cited 1× · see citing works
“However, alongside capturing the linguistic nuances of a language, they also perpetuate (and potentially amplify) biases (Zhao et al., 2017; Bender et al., 2021).”
BackgroundLinguistic Knowledge and Transferability of Contextual Representations
Liu, N. F., Gardner, M., Belinkov, Y., Peters, M. E., & Smith, N. A. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1112 · Cited by 651 · see citing works
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.416 · Cited by 643 · see citing works
Miller, G. A., Leacock, C., Tengi, R., & Bunker, R. T. (1993) · DARPA Workshop on Speech and Natural Language (HLT) · doi:10.3115/1075671.1075742 · Cited by 612 · see citing works
(2019) · doi:10.18653/v1/n19-1419 · Cited by 601 · see citing works
Political Effects of the Internet and Social Media
Zhuravskaya, E., Petrova, M., & Enikolopov, R. (2020) · Annual Review of Economics · doi:10.1146/annurev-economics-081919-050239 · Cited by 560 · see citing works
Tenney, I., Xia, P., Chen, B., Wang, A., Poliak, A., McCoy, R. T., Kim, N., Durme, B. V., Bowman, S. R., Das, D., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.06316 · Cited by 480 · see citing works
Political Effects of the Internet and Social MediaPreprint
Zhuravskaya, E., Petrova, M., & Enikolopov, R. (2020) · Annual Review of Economics · doi:10.2139/ssrn.3439957 · Cited by 480 · see citing works
On Measuring Social Biases in Sentence Encoders
May, C., Wang, A., Bordia, S., Bowman, S. R., & Rudinger, R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1063 · Cited by 461 · see citing works
Posterior Regularization for Structured Latent Variable Models
Ganchev, K., Graça, J., Gillenwater, J., & Taskar, B. (2010) · Cited by 455 · see citing works
Stance and Sentiment in Tweets
Mohammad, S. M., Sobhani, P., & Kiritchenko, S. (2017) · ACM Transactions on Internet Technology · doi:10.1145/3003433 · Cited by 449 · see citing works
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Nangia, N., Vania, C., Bhalerao, R., & Bowman, S. R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.154 · Cited by 446 · see citing works
Permutation, Parametric, and Bootstrap Tests of Hypotheses
(2005) · Technometrics · doi:10.1198/tech.2005.s333 · Cited by 420 · see citing works
The Upward Bias in Measures of Information Derived from Limited Data Samples
Treves, A., & Panzeri, S. (1995) · Neural Computation · doi:10.1162/neco.1995.7.2.399 · Cited by 352 · see citing works
Sparse Additive Generative Models of Text
Eisenstein, J., Ahmed, A. A., & Xing, E. P. (2018) · Figshare · doi:10.1184/r1/6476342 · Cited by 279 · see citing works
Sentiment, emotion, purpose, and style in electoral tweets
Mohammad, S. M., Zhu, X., Kiritchenko, S., & Martin, J. (2014) · Information Processing & Management · doi:10.1016/j.ipm.2014.09.003 · Cited by 274 · see citing works
Order of subject, object, and verb
Huber, M., & Meeuwis, M. (2013) · Ghent University Academic Bibliography (Ghent University) · Cited by 273 · see citing works
Social Media and the Elections
Metaxas, P., & Mustafaraj, E. (2012) · Science · doi:10.1126/science.1230456 · Cited by 245 · see citing works
It's a Man's Wikipedia? Assessing Gender Inequality in an Online Encyclopedia
Wagner, C., García, D., Jadidi, M., & Strohmaier, M. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v9i1.14628 · Cited by 202 · see citing works
Posterior Regularization for Structured Latent Variable Models
GanchevKuzman, GraçaJoão, GillenwaterJennifer, & TaskarBen (2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1859918 · Cited by 201 · see citing works
Measuring and Reducing Gendered Correlations in Pre-trained Models
Webster, K., Wang, X., Tenney, I., Beutel, A., Pitler, E., Pavlick, E., Chen, J., Chi, E., & Petrov, S. (2020) · arXiv · doi:10.48550/arxiv.2010.06032 · Cited by 195 · see citing works
Social media and political discussion: when online presence silences offline conversation
Hampton, K. N., Shin, I., & Lu, W. (2016) · Information Communication & Society · doi:10.1080/1369118x.2016.1218526 · Cited by 193 · see citing works
Evaluating the Underlying Gender Bias in Contextualized Word Embeddings
Basta, C., Costa-jussà, M. R., & Casas, N. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3805 · Cited by 189 · see citing works
Hotel Arabic-Reviews Dataset Construction for Sentiment Analysis Applications
Elnagar, A., Khalifa, Y., & Einea, A. (2017) · Studies in computational intelligence · doi:10.1007/978-3-319-67056-0_3 · Cited by 169 · see citing works
Building Sentiment Lexicons for All Major Languages
Chen, Y., & Skiena, S. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-2063 · Cited by 161 · see citing works
Proceedings of the Ninth International Conference on Language Resources and Evaluation
Tsvetkov, Y., Schneider, N., Hovy, D., Bhatia, A., Faruqui, M., & Dyer, C. (2014) · Cited by 137 · see citing works
A Dataset of Syntactic-Ngrams over Time from a Very Large Corpus of English Books
Goldberg, Y., & Orwant, J. (2013) · Cited by 133 · see citing works
Conflict, criticism, or confidence
Collier, B. C., & Bear, J. B. (2012) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2145204.2145265 · Cited by 116 · see citing works
Perturbation Sensitivity Analysis to Detect Unintended Model Biases
Prabhakaran, V., Hutchinson, B., & Mitchell, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1578 · Cited by 115 · see citing works
AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic LanguagesPreprint
Kunchukuttan, A., Kakwani, D., Golla, S., Gokul, N. C., Bhattacharyya, A., Khapra, M. M., & Kumar, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.00085 · Cited by 103 · see citing works
FEEL: a French Expanded Emotion Lexicon
Abdaoui, A., Azé, J., Bringay, S., & Pascal Poncelet, B. (2016) · Language Resources and Evaluation · doi:10.1007/s10579-016-9364-5 · Cited by 99 · see citing works
Multi-Dimensional Gender Bias Classification
Dinan, E., Fan, A., Wu, L., Weston, J., Kiela, D., & Williams, A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.23 · Cited by 94 · see citing works
In Plain Sight: Media Bias Through the Lens of Factual Reporting
Fan, L., White, M., Sharma, E., Su, R., Choubey, P. K., Huang, R., & Wang, L. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1664 · Cited by 92 · see citing works
BabelSenticNet: A Commonsense Reasoning Framework for Multilingual Sentiment Analysis
Vilares, D., Peng, H., Satapathy, R., & Cambria, E. (2018) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2018.8628718 · Cited by 87 · see citing works
Huber, M., & Meeuwis, M. (2013) · Ghent University Academic Bibliography (Ghent University) · Cited by 68 · see citing works
Creating a General Russian Sentiment Lexicon
Loukachevitch, N., & Levchik, A. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2rzrgqnaka62 · Cited by 66 · see citing works
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender BiasPreprint
Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Sakenis, S., Huang, J., Singer, Y., & Shieber, S. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.12265 · Cited by 63 · see citing works
RtGender: A Corpus for Studying Differential Responses to Gender
Voigt, R., Jurgens, D., Prabhakaran, V., Jurafsky, D., & Tsvetkov, Y. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3di79eiki43x · Cited by 52 · see citing works
The Importance of Political Knowledge for Effective Citizenship
Kleinberg, M. S., & Lau, R. R. (2019) · Public Opinion Quarterly · doi:10.1093/poq/nfz025 · Cited by 45 · see citing works
“You are grounded!”: Latent Name Artifacts in Pre-trained Language Models
Shwartz, V., Rudinger, R., & Tafjord, O. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.556 · Cited by 41 · see citing works
Névéol, A., Dupont, Y., Bezançon, J., & Fort, K. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.583 · Cited by 38 · see citing works
Augmenting English Adjective Senses with Supersenses
Tsvetkov, Y., Schneider, N., Hovy, D., Bhatia, A., Faruqui, M., & Dyer, C. (2014) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5edop9rocohn · Cited by 34 · see citing works
Gender Bias in Masked Language Models for Multiple Languages
Kaneko, M., Imankulova, A., Bollegala, D., & Okazaki, N. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.197 · Cited by 31 · see citing works
Monolingual and Multilingual Reduction of Gender Bias in Contextualized Representations
liang, S. M., Dufter, P., & Schütze, H. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.446 · Cited by 30 · see citing works
A Spanish semantic orientation approach to domain adaptation for polarity classification
Molina-González, M. D., Martínez‐Cámara, E., Valdivia, M. T. M., & López, L. A. U. (2014) · Information Processing & Management · doi:10.1016/j.ipm.2014.10.002 · Cited by 30 · see citing works
Zhang, L., & Chen, C. (2016) · International Conference on Computational Intelligence and Security (CIS) · doi:10.1109/cis.2016.0046 · Cited by 28 · see citing works
Who Sides with Whom? Towards Computational Construction of Discourse Networks for Political Debates
Padó, S., Blessing, A., Blokker, N., Dayanık, E., Haunss, S., & Kuhn, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1273 · Cited by 21 · see citing works
Overview of TASS 2019: One More Further for the Global Spanish Sentiment Analysis Corpus.
Díaz–Galiano, M. C., Vega, M. G., Murillo, É. C., Chiruzzo, L., Cumbreras, M. Á. G., Martínez‐Cámara, E., Moctezuma, D., Montejo‐Ráez, A., Cabezudo, M. A. S., Téllez, E. S., et al. (2019) · IberLEF@SEPLN · Cited by 19 · see citing works
Naturalistic Causal Probing for Morpho-Syntax
Amini, A., Pimentel, T., Meister, C., & Cotterell, R. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00554 · Cited by 8 · see citing works
Self-citationCombining Sentiment Lexica with a Multi-View Variational Autoencoder
Hoyle, A., Wolf-Sonkin, L., Wallach, H., Cotterell, R., & Augenstein, I. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1065 · Cited by 7 · see citing works
Self-citationWhat is new? News media, General Elections, Sentiment, and Named Entities
Ahmad, K., Daly, N., & Liston, V. (2011) · Cited by 7 · see citing works
Shalkarbayuli, A., Kairbekov, A., & Amangeldi, Y. (2018) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1117/1/012002 · Cited by 4 · see citing works
Measuring Gender Bias in West Slavic Language Models
Martinková, S., Stańczak, K., & Augenstein, I. (2023) · Workshop on Slavic Natural Language Processing (SlavicNLP) · doi:10.18653/v1/2023.bsnlp-1.17 · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
7 of the 78 references couldn’t be matched to a known paper.
Research works printed but not linked: 14
- [8]Gotti F, Langlais P, Farzindar A. Translating Government Agencies’ Tweet Feeds: Specificities, Problems and (a few) Solutions In Proceedings of the Workshop on Language Analysis in Social Media. Atlanta, Georgia: Association for Computational Linguistics. 2013; 80– 89. Available from: https://aclanthology.org/W13-1109.
- [16]Stańczak K, Augenstein I. A Survey on Gender Bias in Natural Language Processing. arXiv preprint arXiv:211214168. 2021;abs/2112.14168. Available from: https://arxiv.org/abs/2112.14168. · arXiv
- [21]Vig J, Gehrmann S, Belinkov Y, Qian S, Nevo D, Singer Y, et al. Investigating Gender Bias in Language Models Using Causal Mediation Analysis. In Advances in Neural Information Processing Systems. Curran Associates, Inc. 2020; 33: 12388– 12401. Available from: https://proceedings.neurips.cc/paper/2020/file/92650b2e92217715fe312e6fa7b90d82-Paper.pdf.
- [29]Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, et al. PaLM: Scaling Language Modeling with Pathways. arXiv preprint arXiv:220402311. 2022;abs/2204.02311 Available from: https://arxiv.org/abs/2204.02311. · arXiv
- [34]Mohammad SM, Kiritchenko S, Sobhani P, Zhu X, Cherry C. A Dataset for Detecting Stance in Tweets. In Proceedings of the Tenth International Conference on Language Resources and Evaluation. Portorož, Slovenia: European Language Resources Association. 2016;3945–3952. Available from: https://www.aclweb.org/anthology/L16-1623.
- [40]Hennigen LT, Kim Y. Deriving Language Models from Masked Language Models. arXiv preprint arXiv:230515501. 2023;abs/2305.15501. Available from: https://arxiv.org/abs/2305.15501. · arXiv
- [41]Goyal K, Dyer C, Berg-Kirkpatrick T. Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis–Hastings. The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. Available from: https://openreview.net/forum?id=6PvWo1kEvlT.
- [44]Zeman D, Nivre J, Abrams M, Ackermann E, Aepli N, et al. Universal Dependencies 2.6. 2020. Available from: http://hdl.handle.net/11234/1-3226.
- [45]Wu Y, Schuster M, Chen Z, Le QV, Norouzi M, Macherey W, et al. Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv preprint arXiv:160908144. 2016;abs/1609.08144. Available from: https://arxiv.org/abs/1609.08144. · arXiv
- [47]Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In 8th International Conference on Learning Representations 2020, Addis Ababa, Ethiopia, April 26-30, 2020. 2020. Available from: https://openreview.net/forum?id=H1eA7AEtvS.
- [50]Kingma DP, Welling M. Auto-Encoding Variational Bayes. arXiv preprint arXiv:13126114. 2013;abs/1312.6114. Available from: https://arxiv.org/abs/1312.6114. · arXiv
- [53]Asgari E, Braune F, Roth B, Ringlstetter C, Mofrad M. UniSent: Universal Adaptable Sentiment Lexica for 1000+ Languages. In Proceedings of the 12th Language Resources and Evaluation Conference. Marseille, France: European Language Resources Association. 2020;4113–4120. Available from: https://www.aclweb.org/anthology/2020.lrec-1.506.
- [58]Chen CC, Huang HH, Chen HH. NTUSD-Fin: A Market Sentiment Dictionary for Financial Social Media Data Applications. In El-Haj M, Rayson P, Moore A, editors. Proceedings of the Eleventh International Conference on Language Resources and Evaluation. Paris, France: European Language Resources Association. 2018. Available from: http://lrec-conf.org/workshops/lrec2018/W27/pdf/1_W27.
- [73]Blodgett SL, Barocas S, Daumé III H, Wallach H. NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online: Association for Computational Linguistics. 2020;5454–5476. Available from: https://www.aclweb.org/anthology/2020.acl-main.485.
Cited by
Sorted by citations
Probing Pre-Trained Language Models for Cross-Cultural Differences in Values
Arora, A., Kaffee, L.-A., & Augenstein, I. (2023) · Workshop on Cross-Cultural Considerations in NLP (C3NLP) · doi:10.18653/v1/2023.c3nlp-1.12 · Cited by 91 · see citing works
Self-citationFairness and social bias quantification in Large Language Models for sentiment analysis
Radaideh, M. I., Kwon, O. H., & Radaideh, M. I. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113569 · Cited by 21 · see citing works
Anoop, K. R., Gangan, M. P., Deepak, P., & Lajish, V. L. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-4453-6_2 · Cited by 15 · see citing works
Quantifying gender biases towards politicians on Reddit
Marjanović, S. V., Stańczak, K., & Augenstein, I. (2022) · PLoS ONE · doi:10.1371/journal.pone.0274317 · Cited by 13 · see citing works
Self-citationXiang, A. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104554 · Cited by 11 · see citing works
The Elusiveness of Detecting Political Bias in Language Models
Lunardi, R., Barbera, D. L., & Roitero, K. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3680002 · Cited by 10 · see citing works
Ta, N., Zeng, J., & Li, Z. (2025) · Information Communication & Society · doi:10.1080/1369118x.2025.2537803 · Cited by 7 · see citing works
John vs. Ahmed: Debate-Induced Bias in Multilingual LLMs
Demidova, A., Atwany, H., Rabih, N., Sha’ban, S., & Abdul-Mageed, M. (2024) · Arabic Natural Language Processing Workshop (WANLP) · doi:10.18653/v1/2024.arabicnlp-1.18 · Cited by 6 · see citing works
Liu, Y., Wu, D., Ma, F., & Mao, W. (2025) · International Conference on Transportation Information and Safety · doi:10.1109/ictis68762.2025.11214874 · Cited by 3 · see citing works
The Ethics of Automating Legal Actors
Valvoda, J., Thompson, A., Cotterell, R., & Teufel, S. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00668 · Cited by 3 · see citing works
Self-citationMeasuring Gender Bias in West Slavic Language Models
Martinková, S., Stańczak, K., & Augenstein, I. (2023) · Workshop on Slavic Natural Language Processing (SlavicNLP) · doi:10.18653/v1/2023.bsnlp-1.17 · Cited by 3 · see citing works
Self-citationLe, N. T., & Sadat, F. (2024) · Conference on Human System Interactions · doi:10.1109/hsi61632.2024.10613583 · Cited by 2 · see citing works
UsingWikidata for Enhancing Compositionality in Pre-trained Language Models
Beloucif, M., Bansal, M., Biemann, C., & Universität Hamburg, G. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_019 · Cited by 1 · see citing works
All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs?
Kim, S., Janssen, M. A., Wang, J., Min-Venditti, A., Karanjia, N., & Anderies, J. M. (2026) · arXiv · doi:10.48550/arxiv.2604.17247
Large Language Models and Gender Bias: An Empirical AnalysisPreprint
ÇAKIR, P., & TEMİR, E. (2026) · Research Square · doi:10.21203/rs.3.rs-9577604/v1
Gender bias in LLMs: analyzing AI responses on STEM career guidance
Castro, L. M., & Castillo, G. P. (2026) · Cuadernos info · doi:10.7764/cdi.64.98546
Wilde, L. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-8424-5.ch004
Nursaid, N., Wulandari, E., & Ghaluh, B. M. (2026) · Australian Feminist Studies · doi:10.1080/08164649.2026.2712260
Wang, H., Lu, Y., Nan, K., Zhang, Q., Wang, J., & Qiu, L. (2026) · JMIR AI · doi:10.2196/106543
- Authors
- Stańczak, K., Ray Choudhury, S., Pimentel, T., Cotterell, R., & Augenstein, I.
- Affiliations
- University of Copenhagen (DK)
- University of Cambridge (GB)
- ETH Zurich (CH)
- Corresponding author
- Stańczak, Karolina
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2023-11-28
- Volume
- 18
- Issue
- 11
- Pages
- e0277640
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC10684026
- OpenAlex
- W3154775663
- Cited by
- 13
- References
- 78
- Citations by year
- 2026: 5 · 2025: 4 · 2024: 3 · 2022: 1
- FWCI
- 3.00
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
- Apollo (University of Cambridge) (submittedVersion, cc-by)
- ETH Zürich Research Collection
- Funders
- Danmarks Frie Forskningsfond (9130-00092B)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Gender Studies › Gender Studies in Language
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- gender bias, politicians, cross-lingual language models, pre-trained language models, adjective usage, verb usage, gender stereotypes, model size
- MeSH terms
- Female, Humans, Language, Male, Names, Bias, Multilingualism, Sexism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
