AI generates covertly racist decisions about people based on their dialect
Open accessThe DOI matches its Crossref record; 100 of the 100 references it cites were found in a registry or an index.
Abstract
Hundreds of millions of people now interact with language models, with uses ranging from help with writing 1,2 to informing hiring decisions 3 . However, these language models are known to perpetuate systematic racial prejudices, making their judgements biased in problematic ways about groups such as African Americans 4–7 . Although previous research has focused on overt racism in language models, social scientists have argued that racism with a more subtle character has developed over time, particularly in the United States after the civil rights movement 8,9 . It is unknown whether this covert racism manifests in language models. Here, we demonstrate that language models embody covert racism in the form of dialect prejudice, exhibiting raciolinguistic stereotypes about speakers of African American English (AAE) that are more negative than any human stereotypes about African Americans ever experimentally recorded. By contrast, the language models’ overt stereotypes about African Americans are more positive. Dialect prejudice has the potential for harmful consequences: language models are more likely to suggest that speakers of AAE be assigned less-prestigious jobs, be convicted of crimes and be sentenced to death. Finally, we show that current practices of alleviating racial bias in language models, such as human preference alignment, exacerbate the discrepancy between covert and overt stereotypes, by superficially obscuring the racism that language models maintain on a deeper level. Our findings have far-reaching implications for the fair and safe use of language technology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Hofmann et al. ( 2024 ) found that models trained with RLHF exhibited weaker overt prejudice (using terms to describe African Americans such as 'passionate' or 'imaginative') but this did not diminish covert prejudice.”
From AI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War (2025) · cites this paper as Hofmann et al., 2024
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“AI may also introduce bias, influenced by the data used in training [29].”
From Using artificial intelligence to semi-automate trustworthiness assessment of randomized controlled trials: a case study (2025) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“Notably, subtle biases, such as those related to linguistic connotations, regional dialects, or implicit associations, can be especially insidious and difficult to eliminate through technical safeguards144.”
From Current applications and challenges in large language models for patient care: a systematic review (2025) · cites this paper as [144]
The cited source says what the claim attributes to it.
In the citing paper
“Similarly, as LLMs are systems based on predictive text algo rithms it is perhaps not surprising that they have given sexist (Wan et al., 2023 ) and racist (Hofmann et al., 2024 ) outputs.”
From AI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War (2025) · cites this paper as Hofmann et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“The researchers found that, while these models overtly associate African Americans with positive attributes (such as 'brilliant') they covertly associated African Ameri -cans with exclusively negative attributes, (words such as 'lazy' and 'dirty').”
From AI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War (2025) · cites this paper as Hofmann et al., 2024
11 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“It is worth mentioning that the lack of awareness of covert racism also manifests during evaluation, where it is common to test language models for overt racism but not for covert racism 21,63,79,80 .”
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 2×
“The typical pipeline of training language models includes steps such as data filtering 48 and, more recently, HF training 62 that remove overt racial prejudice.”
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 3×
“In the Supplementary Information, we show that this deviation was caused by the base T5 version, and that the larger T5 versions follow the general pattern (Supplementary Table 10).”
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · cited 2×
“As a second potential way to resolve dialect prejudice in language models, we examined training with HF 49,62 .”
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · cited 11×
“2 and Supplementary Table 4).”
InfluentialPaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Unsettling race and language: Toward a raciolinguistic perspective
Rosa, J., & Flores, N. (2017) · Language in Society · doi:10.1017/s0047404517000562 · Cited by 1,675 · cited 1×
“These associations represent raciolinguistic ideologies, demonstrating how AAE is othered through the emphasis on its perceived deviance from standardized norms 44 .”
Research works printed but not linked: 15
- Zhao, W. et al. WildChat: 1M ChatGPT interaction logs in the wild. In Proc. Twelfth International Conference on Learning Representations (OpenReview.net, 2024).
- Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V. & Kalai, A. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. Adv. Neural Inf. Process. Syst. 30, 4356–4364 (2016).
- Li, L. & Bamman, D. Gender and representation bias in GPT-3 generated stories. In Proc. Third Workshop on Narrative Understanding (eds Akoury, N. et al.) 48–55 (Association for Computational Linguistics, 2021).
- Grogger, J. Speech patterns and racial wage inequality. J. Hum. Resour. 46, 1–25 (2011).
- Rodriguez, J. I., Cargile, A. C. & Rich, M. D. Reactions to African-American vernacular English: do more phonological features matter? West. J. Black Stud. 28, 407–414 (2004).
- Surden, H. Artificial intelligence and law: an overview. Ga State Univ. Law Rev. 35, 1305–1337 (2019).
- Schuman, H., Steeh, C., Bobo, L. D. & Krysan, M. (eds) Racial Attitudes in America: Trends and Interpretations (Harvard Univ. Press, 1998).
- Terkel, S. Race: How Blacks and Whites Think and Feel about the American Obsession (New Press, 1992).
- Bonilla-Silva, E. The New Racism: Racial Structure in the United States, 1960s–1990s. In Race, Ethnicity, and Nationality in the United States: Toward the Twenty-First Century 1st edn (ed. Wong, P.) Ch. 4 (Westview Press, 1999).
- Ethayarajh, K., Choi, Y. & Swayamdipta, S. Understanding dataset difficulty with V-usable information. In Proc. 39th International Conference on Machine Learning (eds Chaudhuri, K. et al.) 5988–6008 (Proceedings of Machine Learning Research, 2022).
- Eisenstein, J., O’Connor, B., Smith, N. A. & Xing, E. P. A latent variable model for geographic lexical variation. In Proc. 2010 Conference on Empirical Methods in Natural Language Processing (eds Li, H. & Màrquez, L.) 1277–1287 (Association for Computational Linguistics, 2010).
- Eisenstein, J. What to do about bad language on the internet. In Proc. 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (eds Vanderwende, L. et al.) 359–369 (Association for Computational Linguistics, 2013).
- Zhao, T. Z., Wallace, E., Feng, S., Klein, D. & Singh, S. Calibrate before use: Improving few-shot performance of language models. In Proc. 38th International Conference on Machine Learning (eds Meila, M. & Zhang, T.) 12697–12706 (Proceedings of Machine Learning Research, 2021).
- Jurafsky, D. & Martin, J. H. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition (Prentice Hall, 2000).
- Santurkar, S. et al. Whose opinions do language models reflect? In Proc. 40th International Conference on Machine Learning (eds Krause, A. et al.) 29971–30004 (Proceedings of Machine Learning Research, 2023).
Cited by
Current applications and challenges in large language models for patient care: a systematic review
Busch, F., Hoffmann, L., Rueger, C., van Dijk, E. H., Kader, R., Ortiz-Prado, E., Makowski, M. R., Saba, L., Hadamitzky, M., Kather, J. N., et al. (2025) · Communications Medicine · doi:10.1038/s43856-024-00717-2 · Cited by 358
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
Anwar, U., Saparov, A., Ramírez, J. R., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., et al. (2024) · arXiv (Cornell University) · doi:10.3929/ethz-b-000706962 · Cited by 17
Explicitly unbiased large language models still form biased associations
Bai, X., Wang, A., Sucholutsky, I., & Griffiths, T. L. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2416228122 · Cited by 442
Looking Beyond the Hype: Understanding the Effects of AI on Learning
Bauer, E., Greiff, S., Graesser, A. C., Scheiter, K., & Sailer, M. (2025) · Educational Psychology Review · doi:10.1007/s10648-025-10020-8 · Cited by 153
Using natural language processing to analyse text data in behavioural science
Feuerriegel, S., Maarouf, A., Bär, D., Geissler, D., Schweisthal, J., Pröllochs, N., Robertson, C. E., Rathje, S., Hartmann, J., Mohammad, S. M., et al. (2025) · Nature Reviews Psychology · doi:10.1038/s44159-024-00392-z · Cited by 140
“For example, LLMs have been found to generate racially biased outputs based on people’s dialect 104 .”
Generative AI in Marketing and Principles for Ethical Design and Deployment
Hermann, E., & Puntoni, S. (2025) · Journal of Public Policy & Marketing · doi:10.1177/07439156241309874 · Cited by 134
“Similarly, GenAI (re)produces race and gender biases when creating imagery or modeling demographic diversity (Sun et al. 2024; Zack et al. 2024) and even generates covertly racist and potentially harmful decisions about people just based on their dialect (Hofmann et al. 2024).”
The life cycle of large language models in education: A framework for understanding sources of bias
Lee, J., Hicke, Y., Yu, R., Brooks, C., & Kizilcec, R. F. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13505 · Cited by 119
Frenkenberg, A., & Hochman, G. (2025) · Systems · doi:10.3390/systems13020082 · Cited by 97
“AI has also been found to produce biased decisions based on individuals’ dialects and even fabricate fake references, further eroding trust in its outputs [20,21].”
Natural Language Processing for Dialects of a Language: A Survey
Joshi, A., Dabre, R., Kanojia, D., Li, Z., Zhan, H., Haffari, G., & Dippold, D. (2025) · ACM Computing Surveys · doi:10.1145/3712060 · Cited by 152
Generative language models exhibit social identity biases
Hu, T., Kyrychenko, Y., Rathje, S., Collier, N., van der Linden, S., & Roozenbeek, J. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00741-1 · Cited by 100
“GPT-2-Large Positive .5106 .9375 2 Results”
Cultural tendencies in generative AI
Lu, J. G., Song, L. L., & Zhang, L. D. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02242-1 · Cited by 136
“Developers have been called upon to examine potential biases in generative AI models 35,45 .”
Cui, Z., Qi, C., Zhou, T., Yu, Y., Wang, Y., Zhang, Z., Zhang, Y., Wang, W., & Liu, Y. (2025) · Comprehensive Reviews in Food Science and Food Safety · doi:10.1111/1541-4337.70068 · Cited by 106
AI-generated faces influence gender stereotypes and racial homogenization
AlDahoul, N., Rahwan, T., & Zaki, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-99623-3 · Cited by 107
“For example, as our analysis has shown, Stable Diffusion associates low-income jobs such as Cleaner, Janitor, and Security Guard with Black people, while associating higher-prestige jobs such as Doctor, Lawyer, and Professor with White people.”
InfluentialCulturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art
Liu, C. C., Gurevych, I., & Korhonen, A. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00760 · Cited by 158
Red-Teaming for Generative AI: Silver Bullet or Security Theater?
Feffer, M., Sinha, A., Deng, W. H., Lipton, Z. C., & Heidari, H. (2024) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v7i1.31647 · Cited by 75
The shrinking landscape of linguistic diversity in the age of large language models
Sourati, Z., Karimi-Malekabadi, F., Ozcan, M., McDaniel, C., Ziabari, A. S., Trager, J. P., Tak, A. N., Chen, M., Morstatter, F., & Dehghani, M. (2026) · Nature Human Behaviour · doi:10.1038/s41562-026-02550-0 · Cited by 78
“This rich variation allows researchers to trace the formation of cultural norms 18,19 , reveal moral values and social biases 20–22 , and even diagnose and track cognitive and mental health conditions 23–27 .”
Responsible AI in Marketing: AI Booing and AI Washing Cycle of AI Mistrust
Öztürkcan, S., & Bozdağ, A. A. (2025) · International Journal of Market Research · doi:10.1177/14707853251379285 · Cited by 68
“…models (LLM)) display co-vert racism by showing dialect prejudice against African American English speakers, often assigning them to less prestigious jobs and harsher legal outcomes, while current bias mitigation strategies tend to only conceal deeper levels of racism (Hofmann et al., 2024).”
Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination
Fleisig, E., Smith, G., Bossi, M., Rustagi, I., Yin, X., & Klein, D. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.750 · Cited by 64
Large Language Models Are Biased Because They Are Large Language Models
Resnik, P. (2025) · Computational Linguistics · doi:10.1162/coli_a_00558 · Cited by 95
Can AI Provide Useful Holistic Essay Scoring?
Tate, T. P., Steiss, J., Bailey, D., Graham, S., Moon, Y., Ritchie, D., Tseng, W., & Warschauer, M. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100255 · Cited by 60
Bouguettaya, A., Stuart, E. M., & Aboujaoude, E. N. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01746-4 · Cited by 85
“Therefore, there is concern that if LLMs are interpreting the transcript from a clinical interview where the patient uses AAVE, the LLM ’ s output might be biased.”
International Scientific Report on the Safety of Advanced AI (Interim Report)Preprint
Bengio, Y., Mindermann, S., Privitera, D., Besiroglu, T., Bommasani, R., Casper, S., Choi, Y., Goldfarb, D. A., Heidari, H., Khalatbari, L., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.05282 · Cited by 79
K-12 teachers’ ethical competencies for AI literacy: Insights from a systematic literature review
Tagare, D., Karki, T., & Yu, W. (2025) · Computers & Education · doi:10.1016/j.compedu.2025.105435 · Cited by 66
The sociolinguistic foundations of language modeling
Grieve, J., Bartl, S., Fuoli, M., Grafmiller, J., Huang, W., Jawerbaum, A., Murakami, A., Perlman, M., Rœmling, D., & Winter, B. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1472411 · Cited by 89
Why AI Is WEIRD and Shouldn't Be This Way: Towards AI for Everyone, with Everyone, by Everyone
Mihalcea, R. F., Ignat, O., Bai, L., Borah, A., Chiruzzo, L., Jin, Z., Kwizera, C., Nwatu, J. C., Poria, S., & Solorio, T. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i27.35092 · Cited by 105
Alvero, A. J., Lee, J., Regla-Vargas, A., Kizilcec, R. F., Joachims, T., & Antonio, A. L. (2024) · Journal of Big Data · doi:10.1186/s40537-024-00986-7 · Cited by 48
“Recent studies indicate that AI has more negative tendencies toward dialectal forms of language [38], suggesting that AI is likely to use higher and more formal registers used by people and communities with higher socioeconomic status.”
BackgroundFine, A., Berthelot, E. R., & Marsh, S. (2025) · Behavioral Sciences · doi:10.3390/bs15040476 · Cited by 67
“For instance, recent research has shown that AI models can covertly encode raciolinguistic stereotypes, leading to negative biases against African American English (AAE) speakers that surpass even the most negative stereotypes historically recorded about African Americans (Hofmann et al., 2024).”
Artificial intelligence bias auditing – current approaches, challenges and lessons from practice
Lacmanović, S., & Škare, M. (2025) · Review of Accounting and Finance · doi:10.1108/raf-01-2025-0006 · Cited by 59
de Roock, R. S. (2024) · Reading Research Quarterly · doi:10.1002/rrq.569 · Cited by 47
Stereotype or Personalization? User Identity Biases Chatbot Recommendations
Kantharuban, A., Milbauer, J., Sap, M., Strubell, E., & Neubig, G. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.1254 · Cited by 67
Biased AI writing assistants shift users’ attitudes on societal issues
Williams‐Ceci, S., Jakesch, M., Bhat, A., Kadoma, K., Zalmanson, L., & Naaman, M. (2026) · Science Advances · doi:10.1126/sciadv.adw5578 · Cited by 61
“…writing may steer the stance of AI suggestions to an extent ( 44 ), biases can also be deliberately introduced into AI suggestions in adversarial scenarios with people providing biased training data, prompts, or feedback during the training process when creating these applications ( 42 , 45 – 47 ).”
Generative Artificial Intelligence in Medicine
Shanmugam, D., Agrawal, M., Movva, R., Chen, I. Y., Ghassemi, M., Jacobs, M. L., & Pierson, E. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-095332 · Cited by 42
On the verge of a digital divide in the use of generative AI?
Suárez, D., & García-Mariñoso, B. (2025) · Telecommunications Policy · doi:10.1016/j.telpol.2025.102997 · Cited by 46
Kenthapadi, K., Sameki, M., & Taly, A. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671467 · Cited by 36
Evaluating Cultural Adaptability of a Large Language Model via Simulation of Synthetic PersonasPreprint
Kwok, L., Bravansky, M., & Griffin, L. D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.06929 · Cited by 48
Warr, M., & Heath, M. K. (2025) · Journal of Teacher Education · doi:10.1177/00224871251325073 · Cited by 35
“For example, recent studies have illustrated that as models “improve”—become more useful for common tasks—they also become more implicitly biased (Gupta et al., 2023; Hofmann et al., 2024; The international scientific report on the safety of advanced AI, 2025; Srivastava et al., 2022).”
InfluentialBeyond Chatbot-K: On Large Language Models, “Generative AI,” and Rise of Chatbots—An Introduction
Goodlad, L. M. E., & Stone, M. (2024) · Critical AI · doi:10.1215/2834703x-11205147 · Cited by 36
How do people react to political bias in generative artificial intelligence (AI)?
Messer, U. (2024) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2024.100108 · Cited by 43
AI–AI bias: Large language models favor communications generated by large language models
Laurito, W., Davis, B., Grietzer, P., Gavenčiak, T., Böhm, A., & Kulveit, J. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2415697122 · Cited by 32
MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMsPreprint
Neplenbroek, V., Bisazza, A., & Fernández, R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.07243 · Cited by 63
Training large language models on narrow tasks can lead to broad misalignment
Betley, J., Warncke, N., Sztyber-Betley, A., Tan, D., Bao, X., Soto, M., Srivastava, M., Labenz, N., & Evans, O. (2026) · Nature · doi:10.1038/s41586-025-09937-5 · Cited by 31
“Previous safety research has largely focused on isolated undesirable behaviours, such as reinforcing harmful stereotypes or providing dangerous information 2,3 .”
A new competitive edge: crafting a service climate that facilitates optimal human–AI collaboration
van Riel, A. C. R., Tabatabaei, F., Yang, X., Masłowska, E., Palanichamy, V., Clark, D., & Luongo, M. (2024) · Journal of service management · doi:10.1108/josm-05-2024-0220 · Cited by 36
Cheung, L. M. E., & Shi, H. (2024) · Australian Review of Applied Linguistics · doi:10.1075/aral.24101.che · Cited by 36
Assessing Dialect Fairness and Robustness of Large Language Models in Reasoning Tasks
Lin, F., Mao, S., Malfa, E. L., Hofmann, V., de Wynter, A., Wang, X., Chen, S., Wooldridge, M., Pierrehumbert, J. B., & Wei, F. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.317 · Cited by 61
Self-citationBeyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression
Xu, Z., Gupta, A., Li, T., Bentham, O., & Srikumar, V. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.901 · Cited by 55
Autoethnographic Insights from Neurodivergent GAI “Power Users”
Glazko, K. S., Cha, J., Lewis, A., Kosa, B., Wimer, B. L., Zheng, A., Zheng, Y., & Mankoff, J. C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713670 · Cited by 32
BRIDGE: benchmarking large language models for understanding real-world clinical practice texts
Wu, J., Gu, B., Zhou, R., Xie, K., Snyder, D., Jiang, Y., Carducci, V., Wyss, R., Desai, R. J., Alsentzer, E., et al. (2026) · Nature Biomedical Engineering · doi:10.1038/s41551-026-01719-2 · Cited by 51
AI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War
Humphreys, D. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00928-y · Cited by 26
“There have been several studies which have shown how, despite RLHF, different LLMs exhibit implicit bias against certain groups (Hofmann et al., 2024 ; Warr et al., 2024 ).”
InfluentialKambeitz, J. P., & Meyer‐Lindenberg, A. (2025) · npj Digital Medicine · doi:10.1038/s41746-024-01422-z · Cited by 26
“Oneoftheprimaryethicalconcernsinvolvestheinherentbiaseswithin LLMs,particularlytowardsminoritygroups 75 – 77 .”
Actions Speak Louder than Words: Agent Decisions Reveal Implicit Biases in Language Models
Li, Y., Shirado, H., & Das, S. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732212 · Cited by 47
Winder, P. D., Hildebrand, C., & Hartmann, J. (2025) · PLoS ONE · doi:10.1371/journal.pone.0325459 · Cited by 36
Pitfalls of Evidence-Based AI Policy
Casper, S., Krueger, D., & Hadfield-Menell, D. (2025) · SuperIntelligence - Robotics - Safety & Alignment · doi:10.70777/si.v2i2.14611 · Cited by 31
Embodied artificial intelligence in ophthalmology
Qiu, Y., Chen, X., Wu, X., Li, Y., Xu, P., Jin, K., Shang, X., Chotcomwongse, P., He, M., & Shi, D. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01754-4 · Cited by 31
Chen, L., Zhang, Y., Feng, J., Chai, H., Zhang, H., Fan, B., Ma, Y., Zhang, S., Li, N., Liu, T., et al. (2026) · Humanities and Social Sciences Communications · doi:10.1057/s41599-026-07316-7 · Cited by 37
“Trigger AI-human feedback loops [67] Individual agent Trigger Masked deception detection [10] Individual agent Trigger Implicit bias evaluation [85] Individual agent Trigger Causal fairness prompting [93] Individual agent Trigger Attention-inspired bias intervention [124] Individual agent Trigger…”
“I Want to Think Like an SLP”: A Design Exploration of AI-Supported Home Practice in Speech Therapy
Dangol, A., Lewis, A., Suh, H., Hong, X., Meadan, H., Fogarty, J. A., & Kientz, J. A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713986 · Cited by 29
“One significant concern is algorithmic bias, which can arise if AI systems are trained on datasets that fail to reflect the diversity of speech patterns, dialects, and cultural backgrounds [48, 64].”
The Medium is the Message: How Non-Clinical Information Shapes Clinical Decisions in LLMs
Gourabathina, A., Gerych, W., Pan, E., & Ghassemi, M. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732121 · Cited by 37
“LLMs have been previously shown to encode and perpetuate societal biases [37, 55] generally — most relatedly one study specifically examined how LLMs encode racial biases from African American English, hence promoting ‘dialect bias’ [30].”
Implicit Bias in LLMs: A SurveyPreprint
Lin, X., & Li, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.02776 · Cited by 34
Tanksley, T., Smith, A. D. R., Sharma, S., & Huff, E. W. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713510 · Cited by 19
Allan, K., Azcona, J., Sripada, S. G., Leontidis, G., Sutherland, C. A. M., Phillips, L. H., & Martin, D. S. (2025) · Royal Society Open Science · doi:10.1098/rsos.241472 · Cited by 28
Rejected Dialects: Biases Against African American Language in Reward Models
Mire, J., Aysola, Z. T., Chechelnitsky, D., Deas, N., Zerva, C., & Sap, M. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.417 · Cited by 30
Borinca, I. (2025) · Analyses of Social Issues and Public Policy · doi:10.1111/asap.70031 · Cited by 22
“They are uncultured”: Unveiling Covert Harms and Social Threats in LLM Generated Conversations
Dammu, P. P. S., Jung, H., Singh, A. A. K., Choudhury, M., & Mitra, T. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.1134 · Cited by 47
Zamudio, C., Grigsby, J. L., & Michelsen, M. (2025) · Journal of Advertising · doi:10.1080/00913367.2024.2438002 · Cited by 23
AI as a talent management tool: An organizational justice perspective
Bennett, N., & Martin, C. L. (2025) · Business Horizons · doi:10.1016/j.bushor.2025.03.005 · Cited by 21
Mitigating AI Bias in School Psychology: Toward Equitable and Ethical Implementation
Lockwood, A. B., & Brown, J. (2025) · School Psychology Review · doi:10.1080/2372966x.2025.2529148 · Cited by 21
“For instance, Hofmann et al. (2024) discussed AI algorithms that included prejudice against users of African American Vernacular English (AAVE).”
Algorithmic Fairness in Automatic Short Answer Scoring
Andersen, N., Mang, J., Goldhammer, F., & Zehner, F. (2025) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-025-00495-5 · Cited by 22
von der Heyde, L., Haensch, A., Wenz, A., & Ma, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.09045 · Cited by 20
“However, politically “neutral” fine-tuning may bring out biases created due to unbalanced training corpora, and even the active alignment against explicit biases might inadvertently exacerbate covert stereotypes (Hofmann et al., 2024; Li et al., 2024).”
International AI Safety Report
Bengio, Y., Mindermann, S., Privitera, D., Besiroglu, T., Bommasani, R., Casper, S., Choi, Y., Fox, P., Garfinkel, B., Goldfarb, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.17805 · Cited by 16
Basoah, J., Chechelnitsky, D., Long, T., Reinecke, K., Zerva, C., Zhou, K., Díaz, M., & Sap, M. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732045 · Cited by 30
Artificial Intelligence in migrant health: a critical perspective on opportunities and risks
Matlin, S. A., Claron, I. M. M., Merone, J., Netto, G., Takian, A., Zaman, M. H., & Saso, L. (2025) · The Lancet Regional Health - Europe · doi:10.1016/j.lanepe.2025.101421 · Cited by 21
“…in uses of AI in healthcare relate to unfairness linked to model design, training data, and AI interactions with clinicians and with patients, 15,71 as well as systematic methodological flaws in the ML literature for diagnosis and prognosis models using imaging data; and the fact that AI…”
Lee, J., Shang, T., Baik, J. Y., Duong-Tran, D., Yang, S., Li, L., & Shen, L. (2026) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1007/s41666-026-00241-z · Cited by 20
Bias and stereotyping: Human and artificial intelligence (AI)
Kang, O., & Hirschi, K. (2025) · Annual Review of Applied Linguistics · doi:10.1017/s026719052500008x · Cited by 25
“Hofmann et al. (2024) examined covert and overt racism in words associated with SAE and African American English (AAE) in several GPT models.”
AI, climate change and justice: elements for a normative framework centring the Global South
Anand, P., & Coeckelbergh, M. (2026) · AI and Ethics · doi:10.1007/s43681-025-00949-5 · Cited by 19
Xiang, A. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104554 · Cited by 15
ChatGPT Doesn’t Trust Chargers Fans: Guardrail Sensitivity in Context
Li, V. R., Chen, Y., & Saphra, N. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.363 · Cited by 29
Li, H. X., Appleby, G., & Suh, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.01425 · Cited by 16
Oğuz, E. (2025) · Assessing Writing · doi:10.1016/j.asw.2025.100981 · Cited by 13
Schneider, B., & de Oliveira, M. M. (2025) · AI & Society · doi:10.1007/s00146-025-02727-7 · Cited by 16
Algorithmic Monocultures in Hiring
Bommasani, R., Bana, S., Creel, K., Jurafsky, D., & Liang, P. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812400 · Cited by 15
Self-citationHedonic price models, social media data and AI – An application to the AIRBNB sector in us cities
Östh, J., Türk, U., Kourtit, K., & Nijkamp, P. J. (2025) · Computers Environment and Urban Systems · doi:10.1016/j.compenvurbsys.2025.102303 · Cited by 12
A Framework for Auditing Chatbots for Dialect-Based Quality-of-Service Harms
Harvey, E., Kizilcec, R. F., & Koenecke, A. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732137 · Cited by 22
What do librarians look like? Stereotyping of a profession by generative Ai
Spennemann, D. H. R., & Oddone, K. (2025) · Journal of Librarianship and Information Science · doi:10.1177/09610006251357286 · Cited by 13
Towards interactive evaluations for interaction harms in human-AI systems
Ibrahim, L., Huang, S., Bhatt, U., Ahmad, L., & Anderljung, M. (2024) · arXiv · doi:10.48550/arxiv.2405.10632 · Cited by 11
“Participation strategy : determining appropriate forms of human involvement to balance experimental control and costs with authentic interaction”
Adultification Bias in LLMs and Text-to-Image Models
Castleman, J., & Korolova, A. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732178 · Cited by 13
A Survey on Stereotype Detection in Natural Language Processing
Cignarella, A. T., Giachanou, A., & Lefever, E. (2025) · ACM Computing Surveys · doi:10.1145/3770754 · Cited by 22
“Similarly, studies on bias in language models have demonstrated how stereotypes are encoded and amplified by large-scale pre-trained models, with biases such as dialect prejudice or gendered associations persisting even after mitigation efforts [39].”
Aligning to What? Limits to RLHF Based Alignment
Barnhart, L., Bafghi, R. A., Becker, S., & Raissi, M. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.421 · Cited by 17
Large Language Models are Catalyzing Chemistry EducationPreprint
Du, Y., Duan, C., Bran, A. M., Sotnikova, A., Qu, Y., Kulik, H. J., Bosselut, A., Xu, J., & Schwaller, P. (2024) · ChemRxiv · doi:10.26434/chemrxiv-2024-h722v · Cited by 14
Au, L. S., Qu, L., Nielsen, J., Ge, Z., Gurrin, L. C., Mol, B. W., & Wang, R. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111672 · Cited by 9
Lessons from the Trenches on Reproducible Evaluation of Language Models
Biderman, S., Schoelkopf, H., Sutawika, L., Gao, L., Tow, J., Abbasi, B., Aji, A. F., Ammanamanchi, P. S., Black, S., Clive, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.14782 · Cited by 9
Characterizing Bias: Benchmarking Large Language Models in Simplified versus Traditional Chinese
Lyu, H., Luo, J., Kang, J. L., & Koenecke, A. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732182 · Cited by 21
Exploring LLMs for Automated Generation and Adaptation of Questionnaires
Adhikari, D. M., Hartland, A., Weber, I. G., & Cannanure, V. K. (2025) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3719160.3736606 · Cited by 18
Ugwuanyi, K. O., Mair, C., Dovchin, S., Erdocia, I., Kuteeva, M., & Airemionkhale, E. O. (2025) · Journal of English for Research Publication Purposes · doi:10.1075/jerpp.00035.ugw · Cited by 10
“As with efforts to address gender bias in AI, optimised models and multilingual language models could help mitigate the systemic linguistic bias towards Anglophone centres and accommodate linguistic variation (cf. Hofmann et al., 2024).”
Basgier, C., & Olejnik, M. (2026) · The WAC Clearinghouse; University Press of Colorado eBooks · doi:10.37514/per-b.2026.2777.2.26 · Cited by 9
Chapter 7. AI Use is a Natural Skillset ✨ AI is a Learned Literacy
Bell, L., & Marcum, J. H. (2026) · The WAC Clearinghouse; University Press of Colorado eBooks · doi:10.37514/per-b.2026.2777.2.07 · Cited by 9
GPT-3.5 altruistic advice is sensitive to reciprocal concerns but not to strategic risk
Schmidt, E., Bonati, S., Köbis, N., & Soraperra, I. (2024) · Scientific Reports · doi:10.1038/s41598-024-73306-x · Cited by 18
HumanStudy-Bench: Towards AI Agent Design for Participant SimulationPreprint
Liu, X., Shang, H., Liu, Z., Liu, X., Xiao, Y., Tu, Y., & Jin, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.00685 · Cited by 10
“…models (LLMs) exhibit human-like decision-making and social behavior across diverse domains, including economic experiments (Hor-ton, 2023), political science (Argyle et al., 2023; Hofmann et al., 2024), marketing (Li et al., 2024), and social psychology (Dillion et al., 2023; Aher et al., 2023).”
Gender and racial bias issues in a commercial “tone of voice” analysis system
Holliday, N. R., & Reed, P. E. (2025) · PLoS ONE · doi:10.1371/journal.pone.0314470 · Cited by 10
“Without careful attention to underlying issues in training data that lead to unequal evaluations by SFSTs, firms run the risk of embedding and amplifying social biases across domains [19].”
Natural and artificial intelligence – the psychotechnical agenda of the 21st century
Webster, C. S. (2025) · Journal of Psychology and AI · doi:10.1080/29974100.2025.2491445 · Cited by 11
From annotation to reflection: how participatory AI training enhances critical thinking
Gholiagha, S., Neyer, J., Sienknecht, M., Wolska, M. A., Kiesel, D., Riehmann, P., Voigt, J., Wiegmann, M., García, I. L., Girgensohn, K., et al. (2025) · AI & Society · doi:10.1007/s00146-025-02539-9 · Cited by 9
Not “just a language model”: AI’s ideological functions
Erfani, F. (2025) · AI & Society · doi:10.1007/s00146-025-02623-0 · Cited by 10
“Language models consistently associate African American English with negative traits, ranking AAE speakers as “aggressive,” “lazy,” or “dirty,” even when content is neutral (Hofmann et al. 2024).”
Metadata
- Authors
- Hofmann, V., Kalluri, P. R., Jurafsky, D., & King, S.
- Affiliations
- University of Oxford (GB)
- Stanford University (US)
- University of Chicago (US)
- Corresponding author
- Hofmann, Valentin
- King, Sharese
- Venue
- Nature
- Published
- 2024-08-28
- Volume
- 633
- Issue
- 8028
- Pages
- 147-154
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- PMC
- PMC11374696
- OpenAlex
- W4401947350
- Cited by
- 233
- References
- 89
- Citations by year
- 2026: 116 · 2025: 111 · 2024: 6
- FWCI
- 96.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Knowledge@UChicago (University of Chicago) (submittedVersion, cc-by-nc-nd)
- RePEc: Research Papers in Economics (submittedVersion)
- University of Chicago (cc-by)
- Funders
- Stanford Institute for Human-Centered Artificial Intelligence, Stanford University
- Studienstiftung des Deutschen Volkes
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Linguistics and Language › Linguistic Variation and Morphology
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- covert racism, African American English, language models, racial bias in AI, preference alignment, hiring decisions, algorithmic fairness
- MeSH terms
- Artificial Intelligence, Black or African American, Decision Making, Humans, Language, Natural Language Processing, Stereotyping, Racism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.