The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases
Open accessAbstract
With the starting point that implicit human biases are reflected in the statistical regularities of language, it is possible to measure biases in English static word embeddings. State-of-the-art neural language models generate dynamic word embeddings dependent on the context in which the word appears. Current methods measure pre-defined social and intersectional biases that occur in contexts defined by sentence templates. Dispensing with templates, we introduce the Contextualized Embedding Association Test (CEAT), that can summarize the magnitude of overall bias in neural language models by incorporating a random-effects model. Experiments on social and intersectional biases show that CEAT finds evidence of all tested biases and provides comprehensive information on the variance of effect magnitudes of the same bias in different contexts. All the models trained on English corpora that we study contain biased representations. GPT-2 contains the smallest magnitude of overall bias followed by GPT, BERT, and then ELMo, negatively correlating with the contextualization levels of the models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Much of the existing literature on diversity in AI asks whether different demographic, cultural, or political groups are represented or treated fairly (Hardt et al., 2016;Barocas et al., 2023;Guo & Caliskan, 2021;Wang et al., 2025).”
From On Epistemic Diversity in Large Language Models (2026) · cites this paper as Guo & Caliskan, 2021
Experimental — accuracy not yet measuredIn the citing paper
“Meta-learning, or 'learning to learn', centers on equipping AI models with the ability to rapidly adapt to new tasks and domains using limited data samples [251], [252].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [252]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Crenshaw, K. (2018) · Feminist Legal Theory · doi:10.4324/9780429500480-5 · Cited by 9,510
Measuring individual differences in implicit cognition: the implicit association test
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.74.6.1464 · Cited by 9,210
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Understanding and using the Implicit Association Test: I. An improved scoring algorithm.
Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.85.2.197 · Cited by 5,815
Research works printed but not linked: 28
- Su Lin Blodgett , Solon Barocas , Hal Daumé III, and Hanna Wallach . 2020 . Language (Technology) is Power : A Critical Survey of" Bias" in NLP. arXiv preprint arXiv:2005.14050 (2020). Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (Technology) is Power: A Critical Survey of" Bias" in NLP. arXiv preprint arXiv:2005.14050 (2020). · arXiv
- Bernd Bohnet , Ryan McDonald , Goncalo Simoes , Daniel Andor , Emily Pitler , and Joshua Maynez . 2018. Morphosyntactic tagging with a meta-bilstm model over context sensitive token encodings. arXiv preprint arXiv:1805.08237 ( 2018 ). Bernd Bohnet, Ryan McDonald, Goncalo Simoes, Daniel Andor, Emily Pitler, and Joshua Maynez. 2018. Morphosyntactic tagging with a meta-bilstm model over context sensitive token encodings. arXiv preprint arXiv:1805.08237 (2018). · arXiv
- Tolga Bolukbasi Kai-Wei Chang James Y Zou Venkatesh Saligrama and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems. 4349--4357. Tolga Bolukbasi Kai-Wei Chang James Y Zou Venkatesh Saligrama and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems. 4349--4357.
- Joy Buolamwini and Timnit Gebru . 2018 . Gender shades: Intersectional accuracy disparities in commercial gender classification . In Conference on fairness, accountability and transparency. 77--91 . Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. 77--91.
- Aylin Caliskan , Joanna J Bryson , and Arvind Narayanan . 2017 . Semantics derived automatically from language corpora contain human-like biases . Science , Vol. 356 , 6334 (2017), 183--186. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, Vol. 356, 6334 (2017), 183--186.
- Alex Campolo , Madelyn Sanfilippo , Meredith Whittaker , and Kate Crawford . 2017. AI now 2017 report . AI Now Institute at New York University ( 2017 ). Alex Campolo, Madelyn Sanfilippo, Meredith Whittaker, and Kate Crawford. 2017. AI now 2017 report. AI Now Institute at New York University (2017).
- Ciprian Chelba , Tomas Mikolov , Mike Schuster , Qi Ge , Thorsten Brants , Phillipp Koehn , and Tony Robinson . 2013. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005 ( 2013 ). Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. 2013. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005 (2013). · arXiv
- Rebecca DerSimonian and Nan Laird . 1986. Meta-analysis in clinical trials. Controlled clinical trials , Vol. 7 , 3 ( 1986 ), 177--188. Rebecca DerSimonian and Nan Laird. 1986. Meta-analysis in clinical trials. Controlled clinical trials, Vol. 7, 3 (1986), 177--188.
- John R Firth . 1957. A synopsis of linguistic theory , 1930 --1955. Studies in linguistic analysis (1957). John R Firth. 1957. A synopsis of linguistic theory, 1930--1955. Studies in linguistic analysis (1957).
- Anthony G Greenwald and Mahzarin R Banaji . 1995. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychological review , Vol. 102 , 1 ( 1995 ), 4. Anthony G Greenwald and Mahzarin R Banaji. 1995. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychological review, Vol. 102, 1 (1995), 4.
- Ange-Marie Hancock . 2007. When multiplication doesn't equal quick addition: Examining intersectionality as a research paradigm. Perspectives on politics , Vol. 5 , 1 ( 2007 ), 63--79. Ange-Marie Hancock. 2007. When multiplication doesn't equal quick addition: Examining intersectionality as a research paradigm. Perspectives on politics, Vol. 5, 1 (2007), 63--79.
- Rachel T Hare-Mustin and Jeanne Marecek . 1988. The meaning of difference: Gender theory, postmodernism, and psychology. American psychologist , Vol. 43 , 6 ( 1988 ), 455. Rachel T Hare-Mustin and Jeanne Marecek. 1988. The meaning of difference: Gender theory, postmodernism, and psychology. American psychologist, Vol. 43, 6 (1988), 455.
- Vasileios Hatzivassiloglou and Kathleen McKeown . 1997. Predicting the semantic orientation of adjectives . In 35th annual meeting of the association for computational linguistics and 8th conference of the european chapter of the association for computational linguistics. 174--181. Vasileios Hatzivassiloglou and Kathleen McKeown. 1997. Predicting the semantic orientation of adjectives. In 35th annual meeting of the association for computational linguistics and 8th conference of the european chapter of the association for computational linguistics. 174--181.
- Larry V Hedges and Ingram Olkin . 2014. Statistical methods for meta-analysis . Academic press . Larry V Hedges and Ingram Olkin. 2014. Statistical methods for meta-analysis .Academic press.
- Keita Kurita , Nidhi Vyas , Ayush Pareek , Alan W Black , and Yulia Tsvetkov . 2019 . Quantifying Social Biases in Contextual Word Representations. In 1st ACL Workshop on Gender Bias for Natural Language Processing . Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Quantifying Social Biases in Contextual Word Representations. In 1st ACL Workshop on Gender Bias for Natural Language Processing .
- Chandler May , Alex Wang , Shikha Bordia , Samuel R Bowman , and Rachel Rudinger . 2019. On measuring social biases in sentence encoders. arXiv preprint arXiv:1903.10561 ( 2019 ). Chandler May, Alex Wang, Shikha Bordia, Samuel R Bowman, and Rachel Rudinger. 2019. On measuring social biases in sentence encoders. arXiv preprint arXiv:1903.10561 (2019). · arXiv
- Moin Nadeem , Anna Bethke , and Siva Reddy . 2020. StereoSet: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456 ( 2020 ). Moin Nadeem, Anna Bethke, and Siva Reddy. 2020. StereoSet: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456 (2020). · arXiv
- Matthew E Peters , Mark Neumann , Mohit Iyyer , Matt Gardner , Christopher Clark , Kenton Lee , and Luke Zettlemoyer . 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365 ( 2018 ). Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018). · arXiv
- Alec Radford , Karthik Narasimhan , Tim Salimans , and Ilya Sutskever . 2018. Improving language understanding by generative pre-training. URL https://s3-us-west-2. amazonaws. com/openai-assets/researchcovers/languageunsupervised/language understanding paper. pdf ( 2018 ). Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. URL https://s3-us-west-2. amazonaws. com/openai-assets/researchcovers/languageunsupervised/language understanding paper. pdf (2018).
- Alec Radford , Jeffrey Wu , Rewon Child , David Luan , Dario Amodei , and Ilya Sutskever . 2019 . Language models are unsupervised multitask learners . OpenAI Blog , Vol. 1 , 8 (2019), 9 . Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, Vol. 1, 8 (2019), 9.
- M Raghavan and S Barocas . [n.d.]. Challenges for mitigating bias in algorithmic hiring . 2019 . URL https://www. brookings. edu/research/challenges-for-mitigating-bias-in-algorithmic-hiring ( [n.,d.]). M Raghavan and S Barocas. [n.d.]. Challenges for mitigating bias in algorithmic hiring. 2019. URL https://www. brookings. edu/research/challenges-for-mitigating-bias-in-algorithmic-hiring ( [n.,d.]).
- Robert Rosenthal and M Robin DiMatteo . 2002. Metaanalysis. Stevens' handbook of experimental psychology ( 2002 ). Robert Rosenthal and M Robin DiMatteo. 2002. Metaanalysis. Stevens' handbook of experimental psychology (2002).
- Yi Chern Tan and L Elisa Celis. 2019. Assessing social and intersectional biases in contextualized word representations. In Advances in Neural Information Processing Systems. 13209--13220. Yi Chern Tan and L Elisa Celis. 2019. Assessing social and intersectional biases in contextualized word representations. In Advances in Neural Information Processing Systems. 13209--13220.
- Autumn Toney and Aylin Caliskan . 2020. ValNorm: A New Word Embedding Intrinsic Evaluation Method Reveals Valence Biases are Consistent Across Languages and Over Decades. arXiv preprint arXiv:2006.03950 ( 2020 ). Autumn Toney and Aylin Caliskan. 2020. ValNorm: A New Word Embedding Intrinsic Evaluation Method Reveals Valence Biases are Consistent Across Languages and Over Decades. arXiv preprint arXiv:2006.03950 (2020). · arXiv
- Autumn Toney , Akshat Pandey , Wei Guo , David Broniatowski , and Aylin Caliskan . 2020. Pro-Russian Biases in Anti-Chinese Tweets about the Novel Coronavirus. arXiv preprint arXiv:2004.08726 ( 2020 ). Autumn Toney, Akshat Pandey, Wei Guo, David Broniatowski, and Aylin Caliskan. 2020. Pro-Russian Biases in Anti-Chinese Tweets about the Novel Coronavirus. arXiv preprint arXiv:2004.08726 (2020). · arXiv
- Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008. Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.
- Rob Voigt , David Jurgens , Vinodkumar Prabhakaran , Dan Jurafsky , and Yulia Tsvetkov . 2018 . RtGender: A corpus for studying differential responses to gender . In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). Rob Voigt, David Jurgens, Vinodkumar Prabhakaran, Dan Jurafsky, and Yulia Tsvetkov. 2018. RtGender: A corpus for studying differential responses to gender. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).
- Zhilin Yang , Zihang Dai , Yiming Yang , Jaime Carbonell , Russ R Salakhutdinov , and Quoc V Le . 2019 . Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems. 5754--5764. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems. 5754--5764.
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases
Steed, R., & Caliskan, A. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445932 · Cited by 189
Self-citationTowards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10896-y · Cited by 141
Garrido-Muñoz, I., Montejo-Ráez, A., Martínez-Santiago, F., & Ureña-López, L. A. (2021) · Applied Sciences · doi:10.3390/app11073184 · Cited by 123
Auto-Debias: Debiasing Masked Language Models with Automated Biased Prompts
Guo, Y., Yang, Y., & Abbasi, A. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.72 · Cited by 131
Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models
Cheng, M., Durmus, E., & Jurafsky, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.84 · Cited by 235
Algorithmic fairness datasets: the story so far
Fabris, A., Messina, S., Silvello, G., & Susto, G. A. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00854-z · Cited by 161
Ghosh, S., & Caliskan, A. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604672 · Cited by 143
Self-citationMcIntosh, T. R., Susnjak, T., Liu, T., Watters, P., Xu, D., Liu, D., & Halgamuge, M. N. (2025) · Technologies · doi:10.3390/technologies13020051 · Cited by 102
“A significant challenge in this area is dealing with intersectional biases [221], [222], [223] and understanding the causal interactions that may contribute to these biases [224], [225], [226], [227].”
BackgroundExploring gender biases in ML and AI academic research through systematic literature review
Shrestha, S., & Das, S. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.976838 · Cited by 105
The Security of Using Large Language Models: A Survey with Emphasis on ChatGPT
Zhou, W., Zhu, X., Han, Q., Li, L., Chen, X., Wen, S., & Xiang, Y. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124983 · Cited by 107
Explicitly unbiased large language models still form biased associations
Bai, X., Wang, A., Sucholutsky, I., & Griffiths, T. L. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2416228122 · Cited by 306
Wang, A., Ramaswamy, V., & Russakovsky, O. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533101 · Cited by 122
Navigating LLM ethics: advancements, challenges, and future directions
Jiao, J., Afroogh, S., Xu, Y., & Phillips, C. (2025) · AI and Ethics · doi:10.1007/s43681-025-00814-5 · Cited by 171
Gender Bias in Word Embeddings
Caliskan, A., Ajay, P. P., Charlesworth, T. E. S., Wolfe, R., & Banaji, M. R. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534162 · Cited by 84
Self-citationMoral consideration for AI systems by 2030
Sebo, J., & Long, R. (2025) · AI and Ethics · doi:10.1007/s43681-023-00379-1 · Cited by 80
“They might amplify biases within their training data or lead to disparate impacts [8–10], disproportionately impacting people with intersecting marginalized identities [11, 12].”
A survey on multilingual large language models: corpora, alignment, and bias
Xu, Y., Hu, L., Zhao, J., Qiu, Z., Xu, K., Ye, Y., & Gu, H. (2025) · Frontiers of Computer Science · doi:10.1007/s11704-024-40579-4 · Cited by 152
Lehr, S. A., Caliskan, A., Liyanage, S., & Banaji, M. R. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2404328121 · Cited by 66
Self-citationYou reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings
Talat, Z., Névéol, A., Biderman, S., Clinciu, M., Dey, M., Longpre, S., Luccioni, A. S., Masoud, M., Mitchell, M., Radev, D., et al. (2022) · doi:10.18653/v1/2022.bigscience-1.3 · Cited by 77
Probing Pre-Trained Language Models for Cross-Cultural Differences in Values
Arora, A., Kaffee, L.-A., & Augenstein, I. (2023) · Workshop on Cross-Cultural Considerations in NLP (C3NLP) · doi:10.18653/v1/2023.c3nlp-1.12 · Cited by 142
Delobelle, P., Tokpo, E., Calders, T., & Berendt, B. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.122 · Cited by 57
Fairness in Deep Learning: A Survey on Vision and Language Research
Parraga, O., Móre, M. D., Mattjie, C., Gavenski, N. S., Kupssinskü, L. S., Medronha, A., de Moura, L. V., Simões, G. S., & Barros, R. C. (2023) · ACM Computing Surveys · doi:10.1145/3637549 · Cited by 82
Investigating Gender and Racial Biases in DALL-E Mini Images
Cheong, M., Abedin, E., Ferreira, M., Reimann, R., Chalson, S., Robinson, P., Byrne, J., Ruppanner, L., Alfano, M., & Klein, C. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3649883 · Cited by 57
Mei, K., Fereidooni, S., & Caliskan, A. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594109 · Cited by 56
Self-citationCao, Y., Pruksachatkun, Y., Chang, K.-W., Gupta, R., Kumar, V., Dhamala, J., & Galstyan, A. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.62 · Cited by 70
Shen, T., Li, J., Bouadjenek, M. R., Mai, Z., & Sanner, S. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103139 · Cited by 57
A Survey on Intersectional Fairness in Machine Learning: Notions, Mitigation, and Challenges
Gohar, U., & Cheng, L. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/742 · Cited by 40
Wolfe, R., Yang, Y., Howe, B., & Caliskan, A. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594072 · Cited by 42
Self-citationSafeguarding human values: rethinking US law for generative AI’s societal impacts
Cheong, I., Caliskan, A., & Kohno, T. (2024) · AI and Ethics · doi:10.1007/s43681-024-00451-4 · Cited by 61
Self-citationHassan, S., Huenerfauth, M., & Alm, C. O. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.267 · Cited by 54
American == White in Multimodal Language-and-Image AI
Wolfe, R., & Caliskan, A. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534136 · Cited by 44
Self-citationShortcut Learning Explanations for Deep Natural Language Processing: A Survey on Dataset Biases
Dogra, V., Verma, S., Kavita, K., Woźniak, M., Shafi, J., & Ijaz, M. F. (2024) · IEEE Access · doi:10.1109/access.2024.3360306 · Cited by 36
Stereotypes and Smut: The (Mis)representation of Non-cisgender Identities by Text-to-Image Models
Ungless, E. L., Roß, B., & Lauscher, A. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.502 · Cited by 53
Kroon, A. C., Welbers, K., Trilling, D., & van Atteveldt, W. (2023) · Communication Methods and Measures · doi:10.1080/19312458.2023.2261372 · Cited by 39
Markedness in Visual Semantic AI
Wolfe, R., & Caliskan, A. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533183 · Cited by 37
Self-citationDisambiguating Algorithmic Bias: From Neutrality to Justice
Edenberg, E., & Wood, A. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604695 · Cited by 38
DeAR: Debiasing Vision-Language Models with Additive Residuals
Seth, A., Hemani, M., & Agarwal, C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00659 · Cited by 34
Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Models
Wolfe, R., & Caliskan, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.41 · Cited by 42
Self-citationJentzsch, S. F., & Turan, C. (2022) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.20 · Cited by 37
When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization
Ladhak, F., Durmus, E., Suzgun, M., Zhang, T., Jurafsky, D., McKeown, K., & Hashimoto, T. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.234 · Cited by 30
Monolingual and Multilingual Reduction of Gender Bias in Contextualized Representations
liang, S. M., Dufter, P., & Schütze, H. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.446 · Cited by 33
Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal
Gupta, U., Dhamala, J., Kumar, V., Verma, A., Pruksachatkun, Y., Krishna, S., Gupta, R., Chang, K.-W., Steeg, G. V., & Galstyan, A. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.55 · Cited by 38
Computational Understanding of Narratives: A Survey
Ranade, P., Dey, S., Joshi, A., & Finin, T. (2022) · IEEE Access · doi:10.1109/access.2022.3205314 · Cited by 35
Paraphrase Identification With Deep Learning: A Review of Datasets and Methods
Zhou, C., Qiu, C. J., Liang, L., & Acuna, D. E. (2025) · IEEE Access · doi:10.1109/access.2025.3556899 · Cited by 48
Deng, C., Duan, Y., Jin, X., Chang, H., Tian, Y., Liu, H., Wang, Y., Gao, K., Zou, H. P., Jin, Y., et al. (2025) · AI and Ethics · doi:10.1007/s43681-025-00797-3 · Cited by 73
Large scale analysis of gender bias and sexism in song lyrics
Betti, L., Abrate, C., & Kaltenbrunner, A. (2023) · EPJ Data Science · doi:10.1140/epjds/s13688-023-00384-8 · Cited by 45
T2IAT: Measuring Valence and Stereotypical Biases in Text-to-Image Generation
Wang, J., Liu, X., Di, Z., Liu, Y., & Wang, X. E. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.160 · Cited by 50
McIlroy-Young, R., Kleinberg, J. M., Sen, S., Barocas, S., & Anderson, A. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534177 · Cited by 31
AI Ethics: A Bibliometric Analysis, Critical Issues, and Key Gaps
Gao, D., Haverly, A., Mittal, S., Wu, J., & Chen, J.-D. (2024) · International Journal of Business Analytics · doi:10.4018/ijban.338367 · Cited by 14
Extensive study on the underlying gender bias in contextualized word embeddings
Basta, C., Costa‐jussà, M. R., & Casas, N. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-05211-z · Cited by 26
Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models
Cao, Y., Sotnikova, A., Daumé, H., Rudinger, R., & Zou, L. X. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.92 · Cited by 30
Ghai, B., Hoque, N., & Mueller, K. D. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3451587 · Cited by 22
Artificial Intelligence, Bias, and Ethics
Caliskan, A. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/799 · Cited by 26
Self-citationRaza, S. (2022) · Healthcare Analytics · doi:10.1016/j.health.2022.100100 · Cited by 24
Evaluating Biased Attitude Associations of Language Models in an Intersectional Context
Sabbaghi, S. O., Wolfe, R., & Caliskan, A. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604666 · Cited by 25
Self-citationMABEL: Attenuating Gender Bias using Textual Entailment Data
He, J., Xia, M., Fellbaum, C., & Chen, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.657 · Cited by 39
Fairness-Aware Structured Pruning in Transformers
Zayed, A., Mordido, G., Shabanian, S., Baldini, I., & Chandar, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30256 · Cited by 25
Nguyen, H., Nguyen, V., Ludovise, S., & Santagata, R. (2024) · Learning Media and Technology · doi:10.1080/17439884.2024.2435834 · Cited by 26
Câmara, A., Taneja, N., Azad, T., Allaway, E., & Zemel, R. S. (2022) · Workshop on Language Technology for Equality, Diversity, Inclusion (LT-EDI) · doi:10.18653/v1/2022.ltedi-1.11 · Cited by 22
Choose Your Lenses: Flaws in Gender Bias Evaluation
Orgad, H., & Belinkov, Y. (2022) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.17 · Cited by 24
A Copious Void: Rhetoric as Artificial Intelligence 1.0
Hallsby, A. (2024) · Rhetoric Society Quarterly · doi:10.1080/02773945.2024.2343265 · Cited by 19
ValNorm Quantifies Semantics to Reveal Consistent Valence Biases Across Languages and Over Centuries
Toney, A., & Caliskan, A. (2020) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.574 · Cited by 32
Self-citationEvidence for Hypodescent in Visual Semantic AI
Wolfe, R., Banaji, M. R., & Caliskan, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533185 · Cited by 19
Self-citationA Comparative Analysis to Evaluate Bias and Fairness Across Large Language Models with BenchmarksPreprint
陳文意, & 黃兆明 (2024) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/mc762 · Cited by 23
Janghorbani, S., & de Melo, G. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.126 · Cited by 32
Allan, K., Azcona, J., Sripada, S. G., Leontidis, G., Sutherland, C. A. M., Phillips, L. H., & Martin, D. S. (2025) · Royal Society Open Science · doi:10.1098/rsos.241472 · Cited by 19
Mapping the individual, social and biospheric impacts of Foundation Models
Hernández, A. D., Krishna, S., Perini, A., Katell, M. A., Bennett, S., Borda, A., Hashem, Y., Hadjiloizou, S., Mahomed, S., Jayadeva, S., et al. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658939 · Cited by 38
Gender Bias in Natural Language Processing and Computer Vision: A Comparative Survey
Bartl, M., Mandal, A., Leavy, S., & Little, S. (2024) · ACM Computing Surveys · doi:10.1145/3700438 · Cited by 23
How Gender Debiasing Affects Internal Model Representations, and Why It Matters
Orgad, H., Goldfarb-Tarrant, S., & Belinkov, Y. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.188 · Cited by 27
Anoop, K. R., Gangan, M. P., Deepak, P., & Lajish, V. L. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-4453-6_2 · Cited by 17
The gender agency gap in fiction writing (1850 to 2010)
Stuhler, O. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2319514121 · Cited by 15
Pugh, S. L., Chandler, C., Cohen, A. S., Diaz-Asper, C. M., Elvevåg, B., & Foltz, P. W. (2024) · Psychiatry Research · doi:10.1016/j.psychres.2024.116119 · Cited by 20
A Robust Bias Mitigation Procedure Based on the Stereotype Content Model
Ungless, E. L., Rafferty, A., Nag, H., & Roß, B. (2022) · Workshop on Natural Language Processing and Computational Social Science (NLP+CSS) · doi:10.18653/v1/2022.nlpcss-1.23 · Cited by 16
Understanding and evaluating harms of AI-generated image captions in political images
Sarhan, H., & Hegelich, S. (2023) · Frontiers in Political Science · doi:10.3389/fpos.2023.1245684 · Cited by 16
Foundation Models in Healthcare: Opportunities, Biases and Regulatory Prospects in Europe
Wójcik, M. A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-12673-4_3 · Cited by 14
SODAPOP: Open-Ended Discovery of Social Biases in Social Commonsense Reasoning Models
An, H., Li, Z., Zhao, J., & Rudinger, R. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.116 · Cited by 24
Investigating the role of large language models on questions about refractive surgery
Demir, S. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105787 · Cited by 13
Intersectional Bias in Causal Language ModelsPreprint
Magee, L., Ghahremanlou, L., Soldatić, K., & Robertson, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.07691 · Cited by 42
Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue modelsPreprint
Smith, E. M., & Williams, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.03300 · Cited by 33
Artificial intelligence in hybrid and information warfare
Moy, W. R., & Gradoń, K. (2023) · Routledge eBooks · doi:10.4324/9781003284093-4 · Cited by 13
Baumer, E. P. S., Taylor, A. S., Brubaker, J. R., & McGee, M. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3660344 · Cited by 19
Harikandeh, S. R. T., Aliakbary, S., & Taheri, S. (2023) · Scientometrics · doi:10.1007/s11192-023-04642-4 · Cited by 15
Bias Detection for Customer Interaction Data: A Survey on Datasets, Methods, and Tools
Donald, A., Galanopoulos, A., Curry, E., Muñoz, E., Ullah, I., Waskow, M. A., Dabrowski, M., & Kalra, M. (2023) · IEEE Access · doi:10.1109/access.2023.3276757 · Cited by 12
Lewis, A., Dangol, A., Suh, H., Olszewski, A., Fogarty, J. A., & Kientz, J. A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714131 · Cited by 18
A Survey on the Impact of Pre-Trained Language Models in Sentiment Classification Task
Gautam, H., Gaur, A., & Yadav, D. K. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00805-z · Cited by 14
On Political Theory and Large Language Models
Rodman, E. (2023) · Political Theory · doi:10.1177/00905917231200826 · Cited by 11
The 5th International Workshop on Narrative Extraction from Texts: Text2Story 2022
Campos, R., Jorge, A., Jatowt, A., Bhatia, S., & Litvak, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-99739-7_68 · Cited by 11
VAST: The Valence-Assessing Semantics Test for Contextualizing Language Models
Wolfe, R., & Caliskan, A. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21400 · Cited by 5
Self-citationAllen, J. W., Wilkinson, D. J. C., & Savulescu, J. (2025) · Bioethics · doi:10.1111/bioe.70032 · Cited by 12
Intersectional Inquiry, on the Ground and in the Algorithm
Robertson, S., Magee, L., & Soldatić, K. (2022) · Qualitative Inquiry · doi:10.1177/10778004221099560 · Cited by 14
Measuring Gender Bias in Contextualized Embeddings
Katsarou, S., Rodríguez-Gálvez, B., & Shanahan, J. (2022) · doi:10.3390/cmsf2022003003 · Cited by 11
Parolin, E. S., Hosseini, M., Hu, Y., Khan, L. R., Brandt, P. T., Osorio, J., & D'Orazio, V. J. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534178 · Cited by 10
Intersectional biases in narratives produced by open-ended prompting of generative language models
Shieh, E., Vassel, F.-M., Sugimoto, C. R., & Monroe‐White, T. (2026) · Nature Communications · doi:10.1038/s41467-025-68004-9 · Cited by 21
Intersectional Male-Centric and White-Centric Biases in Collective Concepts
Bailey, A. H., Williams, A., Poddar, A., & Cimpian, A. (2024) · Personality and Social Psychology Bulletin · doi:10.1177/01461672241232114 · Cited by 12
Prompting the Machine: Introducing an LLM Data Extraction Method for Social Scientists
Foisy, L.-O. M., Proulx, É., Cadieux, H., Gilbert, J., Rivest, J., Bouillon, A., & Dufresne, Y. (2025) · Social Science Computer Review · doi:10.1177/08944393251344865 · Cited by 11
Cantini, R., Orsino, A., Ruggiero, M., & Talia, D. (2025) · Machine Learning · doi:10.1007/s10994-025-06862-6 · Cited by 34
Metadata
- Authors
- Guo, W., & Caliskan, A.
- Affiliations
- George Washington University (US)
- Venue
- Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2021-07-21
- Pages
- 122-133
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W3034115845
- Cited by
- 180
- References
- 26
- Citations by year
- 2026: 16 · 2025: 27 · 2024: 34 · 2023: 44 · 2022: 45 · 2021: 11
- FWCI
- 30.34
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion, cc-by-nc-sa)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- intersectional bias, contextual word embeddings, social bias, language models, GPT-2, BERT, ELMo, random effects models, bias measurement, English language
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
