Semantics derived automatically from language corpora contain human-like biases
Open accessThe DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Abstract
Machines learn what people know implicitly: AlphaGo has demonstrated that a machine can learn how to do things that people spend many years of concentrated study learning, and it can rapidly learn how to do them better than any human can. Caliskan et al. now show that machines can learn word associations from written texts and that these associations mirror those learned by humans, as measured by the Implicit Association Test (IAT) (see the Perspective by Greenwald). Why does this matter? Because the IAT has predictive value in uncovering the association between concepts, such as pleasantness and flowers or unpleasantness and insects. It can also tease out attitudes and beliefs—for example, associations between female names and family or male names and career. Such biases may not be expressed explicitly, yet they can prove influential in behavior. Science , this issue p. 183 ; see also p. 133
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Training deep learning models on biased data can amplify human biases, replicate toxic language, or perpetuate stereotypes of gender, sexual orientation, political leaning, or religion (e.g., Caliskan et al. 2017; Hartmann et al. 2023b).”
From Generative AI (2024) · cites this paper as Caliskan et al. 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some studies (Caliskan et al. 2017;Greenwald et al. 1998) have found that names associated with European Americans are more likely to be linked with pleasantness, while non-European names tend to be associated with unpleasantness.”
From Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models (2024) · cites this paper as Caliskan et al. 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Machine learning models reproduce various content biases inherent to their training data, including gender bias, racial or ethnic bias, bias for negative and threat-related information, and socio-economic bias 69,[86][87][88][89][90] .”
From Machine culture (2023) · cites this paper as [87]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“If people type “He is a doctor. She is a nurse” into Google Translate and translate the phrases into Turkish, Google Translate provides: “O bir hemşire. O bir doktor”. Those Turkish sentences are gender-neutral; Turkish does not differentiate between the words “he” and “she”. When translating the Turkish text into English again, Google Translate provides: “She is a nurse. He is a doctor”.”
From Discrimination, artificial intelligence, and algorithmic decision-making (2025) · cites this paper as Caliskan, Bryson and Narayanan 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“That most models produced overgeneralizations reliably across time suggests that they may have incorporated an ‘algorithmic bias’ from the training data similar to the way models trained on linguistic data can acquire gender, racial, or political biases, by picking up human tendencies implicit in training text corpora [51–53].”
From Generalization bias in large language model summarization of scientific research (2025) · cites this paper as [51]
Experimental — accuracy not yet measured
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 3×
“For example, we repeated all the WEAT and WEFAT experiments presented above using a different pre-trained embedding: word2vec on a Google News corpus (Mikolov and Dean, 2013).”
BackgroundMethodMeasuring individual differences in implicit cognition: the implicit association test
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.74.6.1464 · Cited by 9,210
Bertrand, M., & Mullainathan, S. (2004) · American Economic Review · doi:10.1257/0002828042002561 · Cited by 4,504
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
A Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048
From Frequency to Meaning: Vector Space Models of Semantics
Turney, P. D., & Pantel, P. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2934 · Cited by 2,866
Pattern recognition and machine learning
(2007) · Choice Reviews Online · doi:10.5860/choice.44-5091 · Cited by 2,650
Research works printed but not linked: 1
- M. Stubbs Text and Corpus Analysis: Computer-Assisted Studies of Language and Culture (Blackwell Oxford 1996).
Cited by
Items 201–285 of 285
(Hans) Korteling, J. E., & Toet, A. (2022) · Encyclopedia of Behavioral Neuroscience, 2nd edition · doi:10.1016/b978-0-12-809324-5.24105-9 · Cited by 65
Mitigating Language-Dependent Ethnic Bias in BERT
Ahn, J., & Oh, A. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.42 · Cited by 64
Do neural ranking models intensify gender bias?
Rekabsaz, N., & Schedl, M. (2020) · Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3397271.3401280 · Cited by 64
Which politicians receive abuse? Four factors illuminated in the UK general election 2019
Gorrell, G., Bakir, M. E., Roberts, I., Greenwood, M. A., & Bontcheva, K. (2020) · EPJ Data Science · doi:10.1140/epjds/s13688-020-00236-9 · Cited by 63
Cotter, K., Medeiros, M., Pak, C., & Thorson, K. (2021) · Big Data & Society · doi:10.1177/2053951721996046 · Cited by 63
Fairness-Aware Machine Learning Engineering: How Far Are We?
Ferrara, C., Sellitto, G., Ferrucci, F., Palomba, F., & De Lucia, A. (2024) · Empirical Software Engineering · doi:10.1007/s10664-023-10402-y · Cited by 60
“Similar gender-related biases have been found in Google Translate, which used to hint at professions like nurse as being practiced by women, and occupations like an engineer as being held by men (Brun and Meliou 2018; Caliskan et al. 2017).”
BackgroundEquity Beyond Bias in Language Technologies for Education
Mayfield, E., Madaio, M., Prabhumoye, S., Gerritsen, D., McLaughlin, B., Dixon-Román, E., & Black, A. W. (2019) · Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w19-4446 · Cited by 62
“…results returning stereotypical images of minorities (Noble, 2018) or other algorithmic stereotyping (Abbasi et al., 2019); much of the work in word embeddings falls into this category (Caliskan et al., 2017), though research on downstream tasks and outcomes often have more allocational focus.”
BackgroundData and its (dis)contents: A survey of dataset development and use in machine learning research
Paullada, A., Raji, I. D., Bender, E. M., Denton, E., & Hanna, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100336 · Cited by 548
“She argues that, given that pre-trained representations like word2vec and other word embeddings suffer from gender and racial bias [Caliskan et al., 2017, Packer et al., 2018], and other public domain datasets are older or obtained through means likely to result in amplified representation of…”
BackgroundClinical judgement in the era of big data and predictive analytics
Chin‐Yee, B., & Upshur, R. (2018) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12852 · Cited by 59
An, J., Huang, D., Lin, C., & Tai, M. (2025) · PNAS Nexus · doi:10.1093/pnasnexus/pgaf089 · Cited by 58
“This aligns with broader findings in AI research, where language technologies have been shown to perpetuate — and in some cases, amplify — societal biases (8, 9, 15 – 18).”
Kaplan, D. M., Palitsky, R., Arconada Alvarez, S. J., Pozzo, N. S., Greenleaf, M. N., Atkinson, C. A., & Lam, W. A. (2024) · Journal of Medical Internet Research · doi:10.2196/51837 · Cited by 55
Felmingham, C. M., Adler, N. R., Ge, Z., Morton, R. L., Janda, M., & Mar, V. J. (2021) · American Journal of Clinical Dermatology · doi:10.1007/s40257-020-00574-4 · Cited by 57
Protein language models are biased by unequal sequence sampling across the tree of life
Ding, F., & Steinhardt, J. (2024) · doi:10.1101/2024.03.07.584001 · Cited by 57
Delobelle, P., Tokpo, E., Calders, T., & Berendt, B. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.122 · Cited by 57
Mei, K., Fereidooni, S., & Caliskan, A. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594109 · Cited by 56
Self-citationGulson, K. N., & Webb, P. T. (2017) · Research in Education · doi:10.1177/0034523717723385 · Cited by 55
Giraldo-Luque, S., & Fernández-Rovira, C. (2020) · El profesional de la información · doi:10.3145/epi.2020.sep.28 · Cited by 55
Curto, G., Jojoa Acosta, M. F., Comim, F., & Garcia-Zapirain, B. (2024) · AI & SOCIETY · doi:10.1007/s00146-022-01494-z · Cited by 53
“We have named this model AWEAT (Aporophobia Word Embedding Association Test), since it is inspired on the WEAT (Word Embedding Association Test) by Caliskan et al. (2017).”
Municipal surveillance regulation and algorithmic accountability
Young, M., Katell, M., & Krafft, P. M. (2019) · Big Data & Society · doi:10.1177/2053951719868492 · Cited by 51
“Discriminatory harms in algorithmic systems have been examined in technologies as varied as facial recognition (Buolamwini and Gebru, 2018), machine translation (Caliskan et al., 2017), and search engines (Noble, 2018; Sweeney, 2013).”
BackgroundSoftware support for environmental evidence synthesis
Westgate, M. J., Haddaway, N. R., Cheng, S. H., McIntosh, E. J., Marshall, C., & Lindenmayer, D. B. (2018) · Nature Ecology & Evolution · doi:10.1038/s41559-018-0502-x · Cited by 51
Müller, S. R., Chen, X., Peters, H., Chaintreau, A., & Matz, S. C. (2021) · Scientific Reports · doi:10.1038/s41598-021-93087-x · Cited by 51
A hunt for the Snark: Annotator Diversity in Data Practices
Kapania, S., Taylor, A. S., & Wang, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580645 · Cited by 50
“, age [24], gender [10, 97], sexual orientation [37], race/ethnicity [23, 54] and disability [1, 42]).”
BackgroundUnintended machine learning biases as social barriers for persons with disabilitiess
Hutchinson, B., Prabhakaran, V., Denton, R., Webster, K., Zhong, Y., & Denuyl, S. (2020) · ACM SIGACCESS Accessibility and Computing · doi:10.1145/3386296.3386305 · Cited by 49
Theories of ”gender” in NLP bias research
Devinney, H., Björklund, J., & Björklund, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3534627 · Cited by 49
Fairness metrics and bias mitigation strategies for rating predictions
Ashokan, A., & Haas, C. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102646 · Cited by 47
Cao, Y. T., & Daumé, H. (2021) · Computational Linguistics · doi:10.1162/coli_a_00413 · Cited by 47
Silva, A., Tambwekar, P., & Gombolay, M. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.189 · Cited by 46
“Caliskan et al. (2017) introduce the Word Embedding Associate Test (WEAT) to estimate implicit biases in word embeddings by measuring average cosine similarities of target and attribute sets.”
MethodFairness in representation: quantifying stereotyping as a representational harm
Abbasi, M., Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2019) · Proceedings of the 2019 SIAM International Conference on Data Mining · doi:10.1137/1.9781611975673.90 · Cited by 44
“There have been a few works that look at bias that emerges from the representation process, most notably when looking at learned representations that come from word-vector embeddings[4, 8, 33, 14].”
BackgroundChandra, M., Naik, S., Ford, D., Okoli, E., De Choudhury, M., Ershadi, M., Ramos, G., Hernandez, J., Bhattacharjee, A., Warreth, S., et al. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732063 · Cited by 44
“A well-known issue with AI conversational agents is their tendency to generate content that appears credible but is factually incorrect or biased [10, 12].”
Ovalle, A., Goyal, P., Dhamala, J., Jaggers, Z., Chang, K.-W., Galstyan, A., Zemel, R., & Gupta, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594078 · Cited by 43
A Comprehensive Survey of Bias in LLMs: Current Landscape and Future Directions
Ranjan, R., Gupta, S., & Singh, S. N. (2024) · arXiv · doi:10.48550/arxiv.2409.16430 · Cited by 42
Collecting a large-scale gender bias dataset for coreference resolution and machine translation
Levy, S., Lazar, K., & Stanovsky, G. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.211 · Cited by 41
Personalized chatbot trustworthiness ratings
Srivastava, B., Rossi, F., Usmani, S., & Bernagozzi, M. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.3023919 · Cited by 40
“The latter has been found to be a possible source of human bias (Caliskan, Bryson, and Narayanan 2017).”
BackgroundReinforcement Learning for Fair Dynamic Pricing
Maestre, R., Duque, J., Rubio, A., & Arevalo, J. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-030-01054-6_8 · Cited by 37
Ferrara, E. (2023) · doi:10.2139/ssrn.4614223 · Cited by 266
Hartmann, J., Schwenzow, J., & Witte, M. (2023) · arXiv · doi:10.48550/arxiv.2301.01768 · Cited by 37
“First, it could originate from the massive web-scraped data that ChatGPT was trained on, which is contaminated with human biases (9, 47).”
BackgroundYu, Z., Peng, C., Yang, X., Dang, C., Adekkanattu, P., Gopal Patra, B., Peng, Y., Pathak, J., Wilson, D. L., Chang, C.-Y., et al. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104642 · Cited by 37
Bandy, J., & Vincent, N. (2021) · arXiv · doi:10.48550/arxiv.2105.05241 · Cited by 36
“” On smashwords.com, the book is tagged with “alpha male,” and “submissive female,” and thus could contribute to well-documented gender discrimination in computational language technologies [4, 6, 37].”
BackgroundIntrinsic bias metrics do not correlate with application bias
Goldfarb-Tarrant, S., Marchant, R., Muñoz Sánchez, R., Pandya, M., & Lopez, A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.150 · Cited by 33
“The most commonly used metric is WEAT (Caliskan et al., 2017).”
BackgroundMethodInfluentialOberst, U., De Quintana, M., Del Cerro, S., & Chamarro, A. (2021) · Management Research Review · doi:10.1108/mrr-06-2020-0356 · Cited by 32
Mitigating Dataset Harms Requires Stewardship: Lessons from 1000 Papers
Peng, K., Mathur, A., & Narayanan, A. (2021) · arXiv · doi:10.48550/arxiv.2108.02922 · Cited by 31
“These concerns include the privacy of individuals included [45, 70], representational harms introduced by annotations [25, 44], effects of biases on downstream use [20, 21, 18], and use for ethically dubious purposes [45, 75, 66].”
BackgroundSelf-citationCHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models
Zhao, J., Fang, M., Shi, Z., Li, Y., Chen, L., & Pechenizkiy, M. (2023) · arXiv · doi:10.48550/arxiv.2305.11262 · Cited by 31
Multimodal large language models for inclusive collaboration learning tasks
Lewis, A. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop · doi:10.18653/v1/2022.naacl-srw.26 · Cited by 30
Directional Bias Amplification
Wang, A., & Russakovsky, O. (2021) · arXiv · doi:10.48550/arxiv.2102.12594 · Cited by 30
“However, with newer models like BERT and ELMo that have contextualized embeddings, WEAT does not work (May et al., 2019), so new techniques have been proposed incorporating context (Lu et al., 2019; Kuang & Davison, 2016).”
BackgroundWhen Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization
Ladhak, F., Durmus, E., Suzgun, M., Zhang, T., Jurafsky, D., McKeown, K., & Hashimoto, T. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.234 · Cited by 30
“Recent work shows that NLP models exhibit biases from their training datasets (Caliskan et al., 2017; Zhao et al., 2019; Kurita et al., 2019; Sun et al., 2019; Bartl et al., 2020; Rae et al., 2021; Hon-navalli et al., 2022).”
BackgroundZhou, F., Mao, Y., Yu, L., Yang, Y., & Zhong, T. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.232 · Cited by 29
Misztal-Radecka, J., & Indurkhya, B. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102519 · Cited by 29
“Existing recommendation systems suffer from different types of biases that stem from the cultural and demography stereotypes in he training data and the disparities in user activities (Baeza-Yates, 2018; Caliskan et al., 2017; Eskandanian, Sonboli, & Mobasher, 019; Graells-Garrido et al., 2015; Tsintzou, Pitoura, & Ts…”
InfluentialViewpoint: AI as Author – Bridging the Gap Between Machine Learning and Literary Theory
Van Heerden, I., & Bas, A. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12593 · Cited by 29
Public preferences for governing AI technology: Comparative evidence
Ehret, S. (2022) · Journal of European Public Policy · doi:10.1080/13501763.2022.2094988 · Cited by 28
Wingate, D., Shoeybi, M., & Sorensen, T. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.412 · Cited by 26
Wingate, D., Shoeybi, M., & Sorensen, T. (2022) · arXiv · doi:10.48550/arxiv.2210.03162 · Cited by 26
Automating Automaticity: How the Context of Human Choice Affects the Extent of Algorithmic Bias
Agan, A., Davenport, D., Ludwig, J., & Mullainathan, S. (2023) · doi:10.3386/w30981 · Cited by 25
Kraft, A., & Soulier, E. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658981 · Cited by 25
Do I Look Like a Criminal? Examining how Race Presentation Impacts Human Judgement of Recidivism
Mallari, K., Inkpen, K., Johns, P., Tan, S., Ramesh, D., & Kamar, E. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376257 · Cited by 24
“Developing solutions to mitigate bias is currently a major challenge in AI [6, 1, 38, 35, 4].”
BackgroundHybrid social learning in human-algorithm cultural transmission
Brinkmann, L., Gezerli, D., Kleist, K. V., Müller, T. F., Rahwan, I., & Pescetelli, N. (2022) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2020.0426 · Cited by 24
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
Liang, P. P., Lyu, Y., Fan, X., Wu, Z., Cheng, Y., Wu, J., Chen, L., Wu, P., Lee, M. A., Zhu, Y., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.07502 · Cited by 85
Using Large Language Models for Qualitative Analysis can Introduce Serious Bias
Ashwin, J., Chhabra, A., & Rao, V. (2026) · Sociological Methods & Research · doi:10.1177/00491241251338246 · Cited by 22
Data Statements: From Technical Concept to Community Practice
McMillan-Major, A., Bender, E. M., & Friedman, B. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3594737 · Cited by 21
“Data statements were initially proposed by Bender and Friedman [2] in response to growing awareness of the fact that machine learning (ML) approaches to language technology bring various risks of harm to both system users and others afected by system use [5, 25, 30].”
BackgroundReducing prejudice with counter‐stereotypical AI
Hermann, E., De Freitas, J., & Puntoni, S. (2025) · Consumer Psychology Review · doi:10.1002/arcp.1102 · Cited by 21
Noise induced hearing loss: Building an application using the ANGELIC methodology
Al-Abdulkarim, L., Atkinson, K., Bench-Capon, T., Whittle, S., Williams, R., & Wolfenden, C. (2018) · Argument & Computation · doi:10.3233/aac-181005 · Cited by 21
Davis, M. A., Lim, N., Jordan, J., Yee, J., Gichoya, J. W., & Lee, R. (2023) · American Journal of Roentgenology · doi:10.2214/ajr.22.28802 · Cited by 19
Automating intellectual freedom: Artificial intelligence, bias, and the information landscape
Smith, C. (2022) · IFLA Journal · doi:10.1177/03400352211057145 · Cited by 26
“In a replication study, researchers were able to replicate all reported instances of positive and negative biases associated with gender, race, and age identified by a wide range of research on semantic analysis of text (Caliskan et al., 2021).”
BackgroundGender-tuning: Empowering Fine-tuning for Debiasing Pre-trained Language Models
Ghanbarzadeh, S., Huang, Y., Palangi, H., Cruz Moreno, R., & Khanpour, H. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.336 · Cited by 17
What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus
Luccioni, A. S., & Viviano, J. D. (2021) · arXiv · doi:10.48550/arxiv.2105.02732 · Cited by 17
MarIA and BETO are sexist: evaluating gender bias in large language models for Spanish
(2023) · doi:10.1007/s10579-023-09670-3 · Cited by 16
Measuring and Mitigating Gender Bias in Legal Contextualized Language Models
Bozdag, M., Sevim, N., & Koç, A. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3628602 · Cited by 15
Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions
Borah, A., & Mihalcea, R. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.545 · Cited by 15
How Reliable are Model Diagnostics?
Aribandi, V., Tay, Y., & Metzler, D. (2021) · Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 · doi:10.18653/v1/2021.findings-acl.155 · Cited by 15
Debiasing pretrained text encoders by paying attention to paying attention
Gaci, Y., Benatallah, B., Casati, F., & Benabdeslem, K. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.651 · Cited by 14
On measuring and mitigating biased inferences of word embeddings
Dev, S., Li, T., Phillips, J. M., & Srikumar, V. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6267 · Cited by 14
Venkit, P. N., & Wilson, S. (2021) · arXiv · doi:10.48550/arxiv.2111.13259 · Cited by 14
On the independence of association bias and empirical fairness in language models
Cabello, L., Jørgensen, A. K., & Søgaard, A. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594004 · Cited by 14
CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models
Zhao, J., Fang, M., Shi, Z., Li, Y., Chen, L., & Pechenizkiy, M. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.757 · Cited by 14
Quantum leap in medical mentorship: exploring ChatGPT’s transition from textbooks to terabytes
Chokkakula, S., Chong, S., Yang, B., Jiang, H., Yu, J., Han, R., Attitalla, I. H., Yin, C., & Zhang, S. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1517981 · Cited by 12
Is Your Classifier Actually Biased? Measuring Fairness under Uncertainty with Bernstein Bounds
Ethayarajh, K. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.262 · Cited by 12
‘New Wilderness’ Requires Algorithmic Transparency: a Response to Cantrell et al
Galaz, V., & Mouazen, A. M. (2017) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2017.06.013 · Cited by 10
The tail wagging the dog: Dataset construction biases of social bias benchmarks
Selvam, N., Dev, S., Khashabi, D., Khot, T., & Chang, K.-W. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2023.acl-short.118 · Cited by 11
Identification, Interpretability, and
Lauretig, A. (2019) · Proceedings of the Third Workshop on Natural Language Processing and Computational Social Science · doi:10.18653/v1/w19-2102 · Cited by 9
“, 2016), cultural assumptions and biases (Caliskan et al., 2017; Kozlowski et al., 2018; Garg et al., 2018), and ideological affinity in international organizations (Pomeroy et al.”
BackgroundCognitive phantoms in large language models through the lens of latent variables
Peereboom, S., Schwabe, I., & Kleinberg, B. (2025) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2025.100161 · Cited by 8
“For example, social biases present in training data were found to become ingrained in word embeddings [4], and LLMs have occasionally been found to otherwise misalign with human values [5]: a challenge that is yet unsolved and carries implications for AI safety.”
BackgroundWhat changed? Investigating debiasing methods using causal mediation analysis
Jeoung, S., & Diesner, J. (2022) · Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.26 · Cited by 5
The Strong Pull of Prior Knowledge in Large Language Models and Its Impact on Emotion Recognition
Chochlakis, G., Potamianos, A., Lerman, K., & Narayanan, S. (2024) · arXiv · doi:10.48550/arxiv.2403.17125 · Cited by 5
Understanding Gender Bias in AI-Generated Product Descriptions
Kelly, M., Tahaei, M., Smyth, P., & Wilcox, L. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732169 · Cited by 4
Incorporating subjectivity into gendered ambiguous pronoun (GAP) resolution using style transfer
Pant, K., & Dadu, T. (2022) · Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.28 · Cited by 3
Who's Asking? Investigating Bias Through the Lens of Disability-Framed Queries in LLMs
Hari, V., Panda, K., Panda, S., Agarwal, A., & Patel, H. L. (2025) · 2025 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw69036.2025.00692 · Cited by 2
“The presence of specific contexts, such as disability-related language, may amplify or alter these inferences, leading to intersectional biases that disproportionately affect marginalized groups [4, 7, 9, 13, 17].”
Model Editing with Canonical Examples
Hewitt, J., Chen, S., Xie, L. L., Adams, E., Liang, P., & Manning, C. D. (2024) · arXiv · doi:10.48550/arxiv.2402.06155 · Cited by 1
Metadata
- Authors
- Caliskan, A., Bryson, J. J., & Narayanan, A.
- Affiliations
- Princeton University (US)
- Corresponding author
- Caliskan, Aylin
- Bryson, Joanna J.
- Narayanan, Arvind
- Venue
- Science
- Published
- 2017-04-14
- Volume
- 356
- Issue
- 6334
- Pages
- 183-186
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2893425640
- Cited by
- 2,887
- References
- 50
- Citations by year
- 2026: 299 · 2025: 368 · 2024: 343 · 2023: 411 · 2022: 326 · 2021: 399 · 2020: 326 · 2019: 224 · 2018: 134 · 2017: 47
- FWCI
- 459.41
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- The University of Bath Online Publications Store (The University of Bath) (submittedVersion)
- Pure (University of Bath) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- semantic bias, machine learning, text corpora, Implicit Association Test, algorithmic bias, natural language processing, gender bias, racial bias, cultural bias, bias detection
- MeSH terms
- Machine Learning, Association, Female, Humans, Language, Male, Names, Semantics, Sex Factors, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.