AI generates covertly racist decisions about people based on their dialect
Open accessThe DOI matches its Crossref record; 100 of the 100 references it cites were found in a registry or an index.
Abstract
Hundreds of millions of people now interact with language models, with uses ranging from help with writing 1,2 to informing hiring decisions 3 . However, these language models are known to perpetuate systematic racial prejudices, making their judgements biased in problematic ways about groups such as African Americans 4–7 . Although previous research has focused on overt racism in language models, social scientists have argued that racism with a more subtle character has developed over time, particularly in the United States after the civil rights movement 8,9 . It is unknown whether this covert racism manifests in language models. Here, we demonstrate that language models embody covert racism in the form of dialect prejudice, exhibiting raciolinguistic stereotypes about speakers of African American English (AAE) that are more negative than any human stereotypes about African Americans ever experimentally recorded. By contrast, the language models’ overt stereotypes about African Americans are more positive. Dialect prejudice has the potential for harmful consequences: language models are more likely to suggest that speakers of AAE be assigned less-prestigious jobs, be convicted of crimes and be sentenced to death. Finally, we show that current practices of alleviating racial bias in language models, such as human preference alignment, exacerbate the discrepancy between covert and overt stereotypes, by superficially obscuring the racism that language models maintain on a deeper level. Our findings have far-reaching implications for the fair and safe use of language technology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Hofmann et al. ( 2024 ) found that models trained with RLHF exhibited weaker overt prejudice (using terms to describe African Americans such as 'passionate' or 'imaginative') but this did not diminish covert prejudice.”
From AI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War (2025) · cites this paper as Hofmann et al., 2024
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“AI may also introduce bias, influenced by the data used in training [29].”
From Using artificial intelligence to semi-automate trustworthiness assessment of randomized controlled trials: a case study (2025) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“Notably, subtle biases, such as those related to linguistic connotations, regional dialects, or implicit associations, can be especially insidious and difficult to eliminate through technical safeguards144.”
From Current applications and challenges in large language models for patient care: a systematic review (2025) · cites this paper as [144]
The cited source says what the claim attributes to it.
In the citing paper
“Similarly, as LLMs are systems based on predictive text algo rithms it is perhaps not surprising that they have given sexist (Wan et al., 2023 ) and racist (Hofmann et al., 2024 ) outputs.”
From AI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War (2025) · cites this paper as Hofmann et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“The researchers found that, while these models overtly associate African Americans with positive attributes (such as 'brilliant') they covertly associated African Ameri -cans with exclusively negative attributes, (words such as 'lazy' and 'dirty').”
From AI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War (2025) · cites this paper as Hofmann et al., 2024
11 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“It is worth mentioning that the lack of awareness of covert racism also manifests during evaluation, where it is common to test language models for overt racism but not for covert racism 21,63,79,80 .”
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 2×
“The typical pipeline of training language models includes steps such as data filtering 48 and, more recently, HF training 62 that remove overt racial prejudice.”
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 3×
“In the Supplementary Information, we show that this deviation was caused by the base T5 version, and that the larger T5 versions follow the general pattern (Supplementary Table 10).”
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · cited 2×
“As a second potential way to resolve dialect prejudice in language models, we examined training with HF 49,62 .”
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · cited 11×
“2 and Supplementary Table 4).”
InfluentialPaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Unsettling race and language: Toward a raciolinguistic perspective
Rosa, J., & Flores, N. (2017) · Language in Society · doi:10.1017/s0047404517000562 · Cited by 1,675 · cited 1×
“These associations represent raciolinguistic ideologies, demonstrating how AAE is othered through the emphasis on its perceived deviance from standardized norms 44 .”
Research works printed but not linked: 15
- Zhao, W. et al. WildChat: 1M ChatGPT interaction logs in the wild. In Proc. Twelfth International Conference on Learning Representations (OpenReview.net, 2024).
- Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V. & Kalai, A. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. Adv. Neural Inf. Process. Syst. 30, 4356–4364 (2016).
- Li, L. & Bamman, D. Gender and representation bias in GPT-3 generated stories. In Proc. Third Workshop on Narrative Understanding (eds Akoury, N. et al.) 48–55 (Association for Computational Linguistics, 2021).
- Grogger, J. Speech patterns and racial wage inequality. J. Hum. Resour. 46, 1–25 (2011).
- Rodriguez, J. I., Cargile, A. C. & Rich, M. D. Reactions to African-American vernacular English: do more phonological features matter? West. J. Black Stud. 28, 407–414 (2004).
- Surden, H. Artificial intelligence and law: an overview. Ga State Univ. Law Rev. 35, 1305–1337 (2019).
- Schuman, H., Steeh, C., Bobo, L. D. & Krysan, M. (eds) Racial Attitudes in America: Trends and Interpretations (Harvard Univ. Press, 1998).
- Terkel, S. Race: How Blacks and Whites Think and Feel about the American Obsession (New Press, 1992).
- Bonilla-Silva, E. The New Racism: Racial Structure in the United States, 1960s–1990s. In Race, Ethnicity, and Nationality in the United States: Toward the Twenty-First Century 1st edn (ed. Wong, P.) Ch. 4 (Westview Press, 1999).
- Ethayarajh, K., Choi, Y. & Swayamdipta, S. Understanding dataset difficulty with V-usable information. In Proc. 39th International Conference on Machine Learning (eds Chaudhuri, K. et al.) 5988–6008 (Proceedings of Machine Learning Research, 2022).
- Eisenstein, J., O’Connor, B., Smith, N. A. & Xing, E. P. A latent variable model for geographic lexical variation. In Proc. 2010 Conference on Empirical Methods in Natural Language Processing (eds Li, H. & Màrquez, L.) 1277–1287 (Association for Computational Linguistics, 2010).
- Eisenstein, J. What to do about bad language on the internet. In Proc. 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (eds Vanderwende, L. et al.) 359–369 (Association for Computational Linguistics, 2013).
- Zhao, T. Z., Wallace, E., Feng, S., Klein, D. & Singh, S. Calibrate before use: Improving few-shot performance of language models. In Proc. 38th International Conference on Machine Learning (eds Meila, M. & Zhang, T.) 12697–12706 (Proceedings of Machine Learning Research, 2021).
- Jurafsky, D. & Martin, J. H. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition (Prentice Hall, 2000).
- Santurkar, S. et al. Whose opinions do language models reflect? In Proc. 40th International Conference on Machine Learning (eds Krause, A. et al.) 29971–30004 (Proceedings of Machine Learning Research, 2023).
Cited by
Items 201–300 of 476
Presentism as a digital language ideology in generative AI discourse
Heyd, T. (2025) · Internet Pragmatics · doi:10.1075/ip.00132.hey · Cited by 2
Mrowka, S., Beardsley, A. K., Sobo, E. J., & Goldberg, D. M. (2025) · Anthropological Forum · doi:10.1080/00664677.2025.2596669 · Cited by 3
A Framework for Responsible AI in Higher Education
Bilgin, H. (2025) · Higher Education Governance and Policy · doi:10.55993/hegp.1788343 · Cited by 4
Frank, D., Bruckdorfer, R. E., Jacobsen, L., Li, Y., & Otterbring, T. (2026) · Psychology and Marketing · doi:10.1002/mar.70111 · Cited by 4
“…AI can generate disempowering outcomes, including identity threat, objectification, and the perceived loss of agency, particularly when models reproduce societal stereotypes or when consumers feel reduced to data points (Dang and Liu 2025; D. A. Frank and Otterbring 2024; Hofmann et al. 2024).”
Refoua, E., Elyoseph, Z., Piterman, D., Geller, A., Meinlschmidt, G., & Hadar‐Shoval, D. (2026) · Scientific Reports · doi:10.1038/s41598-026-39292-y · Cited by 4
“Studies have shown that AI models can inadvertently mirror societal prejudices, including racial and ethnic biases, which may affect their judgment in social contexts 8,9 .”
A review of fairness challenges in natural language processing
Zadid, T. H., Jahin, S. M., Dhruba, A. R., Tanim, S. A., Hamja, A. M., Hasan, M. R., Miah, M. S. U., Islam, M. I., & Mansur, A. A. (2026) · Array · doi:10.1016/j.array.2026.100773 · Cited by 4
Satish, S., Teleki, M., Minixhofer, C., Klejch, O., Bell, P., & Székely, É. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798677 · Cited by 2
AI, Normality, and Oppressive Things
Lin, T.-A., & Huang, L. T.-L. (2026) · Minds and Machines · doi:10.1007/s11023-026-09778-9 · Cited by 2
“…effort has been paid to ensure that LLMs do not generate overtly oppressive content (e.g., racist slurs or negative racial stereo - types), they might still perpetuate racism in a more covert way by manifesting dialect prejudice against speakers of African American English (Hofmann et al., 2024 ).”
Evaluation of LLM Performance in Tool Selection: A Feasibility Study on AI-Based Tool Development
Han, H., Kim, A., Sim, J. Y., & Kim, Y. Y. (2026) · International Journal of Precision Engineering and Manufacturing · doi:10.1007/s12541-026-01477-w · Cited by 2
Xia, Z., & Khamboon, C. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-01428-4 · Cited by 2
“In criminal justice, predictive policing and facial recognition technologies have been criticized for unfairly targeting minority groups and reinforcing systemic discrimination [14].”
LLMs produce racist output when prompted in African American English
Blodgett, S. L., & Talat, Z. (2024) · Nature · doi:10.1038/d41586-024-02527-x · Cited by 3
“Eh? Aye!”: Categorisation bias for natural human vs AI-augmented voices is influenced by dialect
Kirk, N. W. (2025) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2025.100153 · Cited by 4
The Sociology of Technical Choices in Predictive AI
Zanger-Tishler, M., & Zhang, S. (2025) · Social Science Computer Review · doi:10.1177/08944393251367045 · Cited by 2
“While trained to avoid overt stereotyping and discrimination, language models can still covertly act on biases when evaluating people (Bai et al., 2025; Hofmann et al., 2024).”
LLM anthropomorphization: balancing ethics and business value
Reusens, M., & Baesens, B. (2025) · Journal of Business Analytics · doi:10.1080/2573234x.2025.2551951 · Cited by 4
“This results into discrimination towards other language groups (Hofmann et al., 2024; Reusens et al., 2024).”
Olivos, F., Kamelski, T., & Gac, S. A. (2025) · Teaching Sociology · doi:10.1177/0092055x251397371 · Cited by 4
“Although topics such as racial and political biases in LLMs are more commonly researched (e.g., Bang et al. 2024; Hofmann et al. 2024; Lee, Montgomery, and Lai 2024; Mei, Fereidooni, and Caliskan et al. 2023; Motoki, Neto, and Rodrigues 2024; Zhou et al., 2024), the effect of sociodemographic…”
Ejiyi, C. J., Obiora, S. C., Obiora, I., Wauk, G., Ejiako, M. O., Omotayo, T., & Bamisile, O. (2026) · Future Internet · doi:10.3390/fi18040196 · Cited by 2
Sentiment Analysis in Digital Spaces: An Overview of Reviews
Ayravainen, L. E. M., Hinds, J. M., & Davidson, B. I. (2026) · ACM Computing Surveys · doi:10.1145/3842380 · Cited by 2
Certified Safe: A Schematic for Approval Regulation of Frontier AIPreprint
Salvador, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.06210 · Cited by 2
Pardo, E., Joseph, G., Rojo, E. M., Beyene, S., Taylor, K., Charlebois, E. D., Gansky, S. A., & Rhoads, K. F. (2025) · American Journal of Public Health · doi:10.2105/ajph.2025.308177 · Cited by 3
Building Situated Capabilities: An Introduction to the GenAI ArcadePreprint
Witzenberger, K., Burgess, J., He, W., & Snoswell, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5326869 · Cited by 2
Kameda, Y., Kaneko, Y., & Takada, S. (2025) · New England Journal of Medicine · doi:10.1056/nejmc2510443 · Cited by 2
Hardy, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.04820 · Cited by 2
SocioEval: A Template-Based Framework for Evaluating Socioeconomic Status Bias in Foundation ModelsPreprint
Kumar, D., Gupta, I., Birur, N. A., Baswa, T., Agarwal, S., & Harshangi, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.02660 · Cited by 2
(2026) · Nature Methods · doi:10.1038/s41592-026-03104-y · Cited by 2
Public Opinion in the Age of AIPreprint
Clinton, J. D., Barari, S., Busby, E. C., Buskirk, T. D., Duch, R. M., Haensch, A., Hillygus, D. S., Kennedy, C., Munger, K., Rivers, D., et al. (2026) · doi:10.33774/apsa-2026-1rg21-v5 · Cited by 3
Linguistic traces of stochastic empathy in language models
Kleinberg, B., Zegers, J., Festor, J., Vida, S., Präsent, J., Loconte, R., & Peereboom, S. (2024) · arXiv · doi:10.48550/arxiv.2410.01675 · Cited by 1
“Using that approach led to the discovery of racial prejudices in LLMs [21], proneness to cognitive biases [22], illusory truth effects in LLMs [23] and early efforts on testing a theory of mind in AI models [24,25].”
Computational challenges arising in algorithmic fairness and health equity with generative AI
Suriyakumar, V. M., Zink, A., Hightower, M., Ghassemi, M., & Beaulieu‐Jones, B. K. (2025) · Nature Computational Science · doi:10.1038/s43588-025-00806-9 · Cited by 4
Algorithmic Price Discrimination
Bar‐Gill, O., & Sunstein, C. R. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780197778197.003.0002 · Cited by 1
Derechos Humanos frente a la vulnerabilidad
Ordaz, R. G., & Valverde, E. G. (2025) · Ediciones Universidad de Salamanca eBooks · doi:10.14201/0aq0382 · Cited by 1
Leveraging AI to advance psychological research for climate policy
Yu, D., Thompson, B. D., & Dubey, R. (2025) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2025.101547 · Cited by 3
“Without robust validation, the risk of misrepresentation could undermine AI’s utility in policy design [3,28].”
Wang, Z., Xia, H., Bao, H., Jing, Y., & Gu, R. (2025) · Advanced Science · doi:10.1002/advs.202508623 · Cited by 4
Evaluating an AI Tutor for Bias Across Different Foundation Models
Vinodh, A., Harvey, E., Almoubayyed, H., Yu, R., Brooks, C. A., Koenecke, A., & Kizilcec, R. F. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98465-5_43 · Cited by 3
Gansky, B. (2025) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v8i2.36610 · Cited by 3
External Intelligence: Oracles, Divination and Animism, and the Use of LLMs/Generative AI
Fischer, M., Applin, S. A., & Ravula, S. (2025) · Emerald Group Publishing Limited eBooks · doi:10.1108/s0163-239620250000061004 · Cited by 1
Of Humans and Machines: Evaluating the Efficacy of GPT-4 in Coding Discourse Data
Zahid, O., Hsiang-Pan, J., Irgens, G. A., Behboudi, A., & Lane, A. C. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-12229-2_8 · Cited by 1
AI-assisted causal mapping: a validation study
Powell, S., & Cabral, G. C. (2025) · International Journal of Social Research Methodology · doi:10.1080/13645579.2025.2591157 · Cited by 1
Eren, S. U. N. A., & Özer, M. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7579-3.ch008 · Cited by 1
Schaarup, J. R., Isaksen, A. A., Norman, K., Bjerg, L., & Hulmán, Á. (2025) · BMJ Digital Health & AI · doi:10.1136/bmjdhai-2025-000090 · Cited by 3
“45 46 As chatbots are not yet implemented for healthcare purposes in Denmark, there is an urgent need for medically specialised systems to prevent potential harm or concern arising from biases in currently available models.”
Casting criminals: a framework for evaluating demographic bias in AI-generated narratives
Sheikhi, G. (2025) · Journal of Intelligent Information Systems · doi:10.1007/s10844-025-01016-5 · Cited by 1
Generating the Past: How Artificial Intelligence Summaries of Historical Events Affect Knowledge
Karell, D., Shu, M., Davidson, T., & Okura, K. (2025) · Social Science Computer Review · doi:10.1177/08944393251409744 · Cited by 2
Eğitimde Yapay Zekâ: Biçimlendirici Değerlendirme ile Denge Arayışı
Arslan, K., & Arslan, K. (2025) · Batı anadolu eğitim bilimleri dergisi · doi:10.51460/baebd.1757476 · Cited by 2
Kamba, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-12801-0_10 · Cited by 1
Gao, L. (2026) · Journal of Language Media and Society · doi:10.66056/jlms.2026.v1i1.002 · Cited by 1
Red Teaming LLMs as Socio-Technical Practice: From Exploration and Data Creation to Evaluation
Garcia, A. A., Wan, R., Oguine, O. C., & Badillo-Urquiola, K. A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790792 · Cited by 3
“Hofmann et al. [34] for example, found that LLMs assigned more negative stereotypes, lower-prestige jobs, and harsher criminal judgments to speakers of African-American English compared to Standard American English prompts highlighting how dialect bias can shape harmful system behavior.”
God(bots) and Authority: Trust and Faith in the Age of AI
Davis, B. C., Cotter, K., Kanthawala, S., Liu, Y., Ritchart, A., & McAtee, H. (2026) · AoIR Selected Papers of Internet Research · doi:10.5210/spir.v2024i0.15109 · Cited by 1
MEDEQUALQA: Evaluating Biases in LLMs with Counterfactual Reasoning
Ghosh, R., Gupta, A., McBride, H., Vaidya, A. J., & Mahmood, F. (2025) · doi:10.18653/v1/2025.sciprodllm-1.4 · Cited by 5
Cha, I., & Wong, R. Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790666 · Cited by 2
From Expectation to Evaluation: Expectation Cues Systematically Bias LLM and Human Judgment
Sun, Y., Dillion, D., Gray, K. J., Lyu, M., Zhang, Z., & Li, F. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790492 · Cited by 2
McKee, H. A. (2026) · Computers & composition/Computers and composition · doi:10.1016/j.compcom.2026.103001 · Cited by 4
Ngigi, E. N. (2026) · International Journal of Research and Innovation in Applied Science · doi:10.51584/ijrias.2026.11050080 · Cited by 1
Side-by-side Comparison Amplifies Dialect Bias in Language Models
Kondapally, K., Smerdon, C. J., Patel, P. C., Akoni, O., Torres, J., Ranjit, J., Finlayson, M., & Swayamdipta, S. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812217 · Cited by 4
Hearn, J., Aavik, K., COLLINSON, D. L., Hall, M., Hultman, M., Shefer, T., & Thym, A. (2026) · Feminism and Organization · doi:10.1332/27556654y2026d000000005 · Cited by 1
STEM Education: Shaping Future Learning Practices in the Age of AI
Elmas, R., & Akarsu, M. (2026) · Maarif global education series · doi:10.1007/978-981-95-9023-0 · Cited by 1
Do Language Models Pass the Bechdel Test? Auditing Gender Biases in LLM-Generated Screenplays
Govindu, M. N., Wang, S., Friedler, S. A., & Metaxa, D. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812208 · Cited by 1
Ferrara, E. (2026) · Future Internet · doi:10.3390/fi18070339 · Cited by 2
“Sharma et al. [52] documented sycophancy as the well-recognized instance; Hofmann et al. [20] reported empirically that human-feedback training exacerbates a covert dialect-conditional discrepancy in some models, which is suggestive of the predicted dynamic.”
InfluentialCultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues
Kautsar, M. D. A., Almheiri, S., Ahsan, M., Elbouardi, B., Samih, Y., Ahmad, S., Keleg, A., Herraoui, O. E., Elzeky, K., Freihat, A. A., et al. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.963 · Cited by 5
DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation
Painter, J., Srirag, D., Kappiyath, A., Kanojia, D., Joshi, A., & Yin, L. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.2321 · Cited by 1
Gendered representations of occupations in large language model generated images and texts
Wen, X., & Zhou, X. (2026) · Social Semiotics · doi:10.1080/10350330.2026.2704561 · Cited by 1
AI systems and the reproduction of (standard) language ideologies in World Englishes
Ugwuanyi, K. O. (2026) · Linguistics Vanguard · doi:10.1515/lingvan-2025-0245 · Cited by 1
“…closer scrutiny from critical sociolinguists because “ dialect prejudice has the potential for harmful consequences: language models are more likely to suggest that speakers of AAE be assigned less-prestigious jobs, be convicted of crimes and be sentenced to death ” (Hofmann et al. 2024: 147).”
Handman, C. (2026) · Signs and Society · doi:10.1017/sas.2026.10072 · Cited by 1
The golden thread of AI subalternity: epistemic activism
Garcia, I. K. (2026) · AI and Ethics · doi:10.1007/s43681-026-01318-6 · Cited by 1
“And a great deal more research meets this criterion, including that on how common AI practices impact and marginalise trans people [83], queer people [80], women [99], Indigenous knowledge systems [67], African American voices [53], and disabled people [32].”
Social and Societal Issues in AAL
Lutz, C., Miguel, C., Mujirishvili, T., Perez‐Vega, R., & Fedosov, A. (2025) · Intelligent systems reference library · doi:10.1007/978-3-031-84158-3_13 · Cited by 2
The MERIT dataset: Modelling and efficiently rendering interpretable transcripts
de Rodrigo, I., Sanchez-Cuadrado, A., Boal, J., & López-López, Á. J. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112502 · Cited by 3
Allan, K., Azcona, J., Sripada, Y., Leontidis, G., Sutherland, C. A. M., Phillips, L. H., & Martin, D. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/r7vf5 · Cited by 3
Rzig, D. E., Paul, D. J., Pister, K., Henkel, J., & Hassan, F. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.12521 · Cited by 1
(2025) · Nature Reviews Psychology · doi:10.1038/s44159-025-00416-2 · Cited by 1
AI Investing and MisinformationPreprint
Kim, J. M., & Leone, A. J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5283695 · Cited by 1
MEDEQUALQA: Evaluating Biases in LLMs with Counterfactual ReasoningPreprint
Ghosh, R., Gupta, A., McBride, H., Vaidya, A. J., & Mahmood, F. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.12818 · Cited by 1
Kharchenko, J., Roosta, T. G., Chadha, A. R., & Shah, C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.04939 · Cited by 1
Liang, W. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.17467 · Cited by 2
Impacts of Racial Bias in Historical Training Data for News AIPreprint
Bhargava, R., Jespersen, M. H., Ndulue, E. B., & Dsouza, V. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.16901 · Cited by 1
“Many ML applications operate as “black boxes” [21] where the implications of misinterpretations can lead to miscommunication or misinformation [9].”
Large Language Models Reproduce Racial Stereotypes When Used for Text AnnotationPreprint
Törnberg, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.13891 · Cited by 3
“…work shows that language models encode anti-Muslim bias in text completion [1], that NLP classifiers disproportionately flag African American Vernacular English (AAVE) as toxic [5, 10, 24], and that contemporary LLMs produce racially disparate evaluations when dialect signals racial identity [16].”
Guey, W., Zhang, W., Bougault, P., Wang, Y., Ucar, B., Moura, V., & Gomes, J. O. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.01451 · Cited by 1
“Public-facing concerns about LLM racial bias have driven extensive RLHF refinement, particularly for Western-developed models (Ganguli et al. 2023), around outputs that mention Black suspects in criminal contexts (Hofmann et al. 2024; An et al. 2025).”
Negative Before Positive: Asymmetric Valence Processing in Large Language ModelsPreprint
Venkatesh, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.05653 · Cited by 1
Reflections and New Directions for Human-Centered Large Language ModelsPreprint
Ziems, C., Zhao, D., Wang, R., Jörke, M., Rushdi, A., Deepak, A., Yu, S., Agarwal, A., Agarwal, H., Aránguiz-Dias, G., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.06901 · Cited by 2
Validated Behavioral Hypotheses as a Lens for Evaluating Participant SimulationPreprint
Liu, X., Shang, H., Liu, Z., Feng, Y., Zhai, G., Xiao, Y., Tu, Y., & Jin, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.15473 · Cited by 1
“…et al., 2023; Gao et al., 2024), as testbeds for simulating user populations (Moon et al., 2024; Zhang et al., 2025; Mou et al., 2024), and as building blocks of multi-agent simulations of organizations and markets (Argyle et al., 2023; Hofmann et al., 2024; Li et al., 2024; Manning et al., 2024).”
Hamidavi, A. (2026) · Research Square · doi:10.21203/rs.3.rs-10369594/v1 · Cited by 1
Artificial Intelligence and the Wilderness & Environmental Medicine Community
Binder, W. D., Moretti, K., & Sheng, A. Y. (2026) · Wilderness and Environmental Medicine · doi:10.1177/10806032261467131 · Cited by 1
Lost in the Request: How Communication Variation Disrupts Retrieval and Action in Email AgentsPreprint
Chen, F., Dutt, R., Taheri, A., & Williams, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.02627 · Cited by 1
Emergence of Biased Consensus in Multi-Agent LLM Debates
Okawa, M. (2026) · arXiv · doi:10.48550/arxiv.2608.02827
“…tasks such as academic recommendation (Sorokovikova et al., 2025), re-sume screening (Wang et al., 2024b; Wilson & Caliskan, 2025; Armstrong et al., 2024), clinical trial matching (Ji et al., 2025), judicial decision in trials (Hofmann et al., 2024), and investment decisions (Winder et al., 2025).”
Addressing emerging issues in entomology: 2023 student debates
Pickens, V., Maille, J. M., Pitt, W. J., Ellis, J. T., Salgado, S., Tims, K. M., Edwards, C.-C., Peavy, M., Williamson, Z., Musgrove, T. R. T., et al. (2024) · Journal of Insect Science · doi:10.1093/jisesa/ieae080
Watson, J., Lee, S., Beekhuizen, B., & Stevenson, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.13852
Palmer, M. (2024) · Amicus Curiae · doi:10.14296/ac.v6i1.5742
Tackling DevOps and CI within ML projects
Rzig, D. E. (2024) · Deep Blue (University of Michigan) · doi:10.7302/25170 · Cited by 1
Tegon, R. (2025) · Advances in human and social aspects of technology book series · doi:10.4018/979-8-3693-9035-1.ch007
KI-basierte Literaturrecherche in der Labormedizin: Assistenz für die Wissenschaft
Adler, J. (2025) · Trillium Diagnostik · doi:10.47184/td.2025.01.01
Creating space for climate justice in library and information science
Oliphant, T., Youngman, T., Hackborn, D., Nathan, L. P., & Patin, B. J. H. (2025) · Information Research · doi:10.47989/ir30colis52330 · Cited by 1
Bar‐Gill, O., & Sunstein, C. R. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780197778197.003.0014
Algorithmically Enhanced Misperceptions
Bar‐Gill, O., & Sunstein, C. R. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780197778197.003.0005
Bar‐Gill, O., & Sunstein, C. R. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780197778197.002.0008
Algorithmic Price Discrimination: Extensions
Bar‐Gill, O., & Sunstein, C. R. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780197778197.003.0003
Bar‐Gill, O., & Sunstein, C. R. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780197778197.003.0001
Ex Post Policing and Ex Ante Regulation
Bar‐Gill, O., & Sunstein, C. R. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780197778197.003.0011
The Right to Algorithmic Transparency
Bar‐Gill, O., & Sunstein, C. R. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780197778197.003.0010
Bar‐Gill, O., & Sunstein, C. R. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780197778197.003.0013
Regulating Preconditions for Algorithmic Harm
Bar‐Gill, O., & Sunstein, C. R. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780197778197.003.0009
Applying the Reforms to the Different Harm Categories
Bar‐Gill, O., & Sunstein, C. R. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780197778197.003.0012
Bar‐Gill, O., & Sunstein, C. R. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780197778197.003.0007
Bar‐Gill, O., & Sunstein, C. R. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780197778197.003.0006 · Cited by 1
Metadata
- Authors
- Hofmann, V., Kalluri, P. R., Jurafsky, D., & King, S.
- Affiliations
- University of Oxford (GB)
- Stanford University (US)
- University of Chicago (US)
- Corresponding author
- Hofmann, Valentin
- King, Sharese
- Venue
- Nature
- Published
- 2024-08-28
- Volume
- 633
- Issue
- 8028
- Pages
- 147-154
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- PMC
- PMC11374696
- OpenAlex
- W4401947350
- Cited by
- 233
- References
- 89
- Citations by year
- 2026: 116 · 2025: 111 · 2024: 6
- FWCI
- 96.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Knowledge@UChicago (University of Chicago) (submittedVersion, cc-by-nc-nd)
- RePEc: Research Papers in Economics (submittedVersion)
- University of Chicago (cc-by)
- Funders
- Stanford Institute for Human-Centered Artificial Intelligence, Stanford University
- Studienstiftung des Deutschen Volkes
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Linguistics and Language › Linguistic Variation and Morphology
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- covert racism, African American English, language models, racial bias in AI, preference alignment, hiring decisions, algorithmic fairness
- MeSH terms
- Artificial Intelligence, Black or African American, Decision Making, Humans, Language, Natural Language Processing, Stereotyping, Racism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.