AI generates covertly racist decisions about people based on their dialect
Open accessThe DOI matches its Crossref record; 98 of the 98 references it cites were found in a registry or an index.
Abstract
Hundreds of millions of people now interact with language models, with uses ranging from help with writing 1,2 to informing hiring decisions 3 . However, these language models are known to perpetuate systematic racial prejudices, making their judgements biased in problematic ways about groups such as African Americans 4–7 . Although previous research has focused on overt racism in language models, social scientists have argued that racism with a more subtle character has developed over time, particularly in the United States after the civil rights movement 8,9 . It is unknown whether this covert racism manifests in language models. Here, we demonstrate that language models embody covert racism in the form of dialect prejudice, exhibiting raciolinguistic stereotypes about speakers of African American English (AAE) that are more negative than any human stereotypes about African Americans ever experimentally recorded. By contrast, the language models’ overt stereotypes about African Americans are more positive. Dialect prejudice has the potential for harmful consequences: language models are more likely to suggest that speakers of AAE be assigned less-prestigious jobs, be convicted of crimes and be sentenced to death. Finally, we show that current practices of alleviating racial bias in language models, such as human preference alignment, exacerbate the discrepancy between covert and overt stereotypes, by superficially obscuring the racism that language models maintain on a deeper level. Our findings have far-reaching implications for the fair and safe use of language technology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Hofmann et al. ( 2024 ) found that models trained with RLHF exhibited weaker overt prejudice (using terms to describe African Americans such as 'passionate' or 'imaginative') but this did not diminish covert prejudice.”
From AI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War (2025) · cites this paper as Hofmann et al., 2024
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“AI may also introduce bias, influenced by the data used in training [29].”
From Using artificial intelligence to semi-automate trustworthiness assessment of randomized controlled trials: a case study (2025) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“Notably, subtle biases, such as those related to linguistic connotations, regional dialects, or implicit associations, can be especially insidious and difficult to eliminate through technical safeguards144.”
From Current applications and challenges in large language models for patient care: a systematic review (2025) · cites this paper as [144]
The cited source says what the claim attributes to it.
In the citing paper
“Similarly, as LLMs are systems based on predictive text algo rithms it is perhaps not surprising that they have given sexist (Wan et al., 2023 ) and racist (Hofmann et al., 2024 ) outputs.”
From AI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War (2025) · cites this paper as Hofmann et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“The researchers found that, while these models overtly associate African Americans with positive attributes (such as 'brilliant') they covertly associated African Ameri -cans with exclusively negative attributes, (words such as 'lazy' and 'dirty').”
From AI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War (2025) · cites this paper as Hofmann et al., 2024
11 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“It is worth mentioning that the lack of awareness of covert racism also manifests during evaluation, where it is common to test language models for overt racism but not for covert racism 21,63,79,80 .”
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 3×
“In the Supplementary Information, we show that this deviation was caused by the base T5 version, and that the larger T5 versions follow the general pattern (Supplementary Table 10).”
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Unsettling race and language: Toward a raciolinguistic perspective
Rosa, J., & Flores, N. (2017) · Language in Society · doi:10.1017/s0047404517000562 · Cited by 1,675
Evaluational reactions to spoken languages.
Lambert, W. E., Hodgson, R. C., Gardner, R. C., & Fillenbaum, S. (1960) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0044430 · Cited by 1,563
Racial stereotypes of one hundred college students.
Katz, D., & Braly, K. (1933) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0074049 · Cited by 1,476
Word embeddings quantify 100 years of gender and ethnic stereotypes
Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720347115 · Cited by 1,065 · cited 2×
“However, the web also abounds with overt racism against African Americans 76,77 , so we wondered why the language models exhibit much less overt than covert racial prejudice.”
Self-citationDeep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058 · cited 1×
“5 and GPT4 are the only language models examined in this paper that were trained with HF, specifically reinforcement learning from human feedback 103 .”
Language (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050
Training Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2176 · Cited by 932 · cited 1×
“It is worth mentioning that the lack of awareness of covert racism also manifests during evaluation, where it is common to test language models for overt racism but not for covert racism 21,63,79,80 .”
Training Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.15556 · Cited by 929
Are Racial Stereotypes Really Fading? The Princeton Trilogy Revisited
Devine, P. G., & Elliot, A. J. (1995) · Personality and Social Psychology Bulletin · doi:10.1177/01461672952111002 · Cited by 859
The Risk of Racial Bias in Hate Speech Detection
Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1163 · Cited by 817
Aletras, N., Tsarapatsanis, D., Preoţiuc-Pietro, D., & Lampos, V. (2016) · PeerJ Computer Science · doi:10.7717/peerj-cs.93 · Cited by 810
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · cited 1×
“Rather than advocating these use cases of AI, which are inherently problematic 59 , the sole objective of this analysis is to examine the extent to which the decisions of language models, when they are used in such contexts, are impacted by dialect.”
African American English: A Linguistic Introduction
Green, L. J. (2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511800306 · Cited by 724
Gender bias in coreference resolution: Evaluation and debiasing methods
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2003 · Cited by 684
Recent unobtrusive studies of Black and White discrimination and prejudice: A literature review.
Crosby, F., Bromley, S., & Saxe, L. (1980) · Psychological Bulletin · doi:10.1037//0033-2909.87.3.546 · Cited by 654
Dovidio, J. F., & Gaertner, S. L. (2004) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(04)36001-6 · Cited by 650
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.416 · Cited by 643
StereoSet: Measuring stereotypical bias in pretrained language models
Nadeem, M., Bethke, A., & Reddy, S. (2020) · arXiv · doi:10.48550/arxiv.2004.09456 · Cited by 643 · cited 9×
“14 and 15 and Supplementary Tables 17–19).”
InfluentialPerceptual and Phonetic Experiments on American English Dialect Identification
Purnell, T., Idsardi, W., & Baugh, J. (1999) · Journal of Language and Social Psychology · doi:10.1177/0261927x99018001002 · Cited by 617
Jackman, M. R., & Muha, M. J. (1984) · American Sociological Review · doi:10.2307/2095528 · Cited by 577
The Woman Worked as a Babysitter: On Biases in Language Generation
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1339 · Cited by 537
On the fading of social stereotypes: Studies in three generations of college students.
Karlins, M., Coffman, T. L., & Walters, G. (1969) · Journal of Personality and Social Psychology · doi:10.1037/h0027994 · Cited by 534
An Intersectional Analysis of Gender and Ethnic Stereotypes: Testing Three Hypotheses
Ghavami, N., & Peplau, L. A. (2013) · Psychology of Women Quarterly · doi:10.1177/0361684312464203 · Cited by 505
The Pile: An 800GB Dataset of Diverse Text for Language Modeling
Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al. (2021) · arXiv · doi:10.48550/arxiv.2101.00027 · Cited by 492
Persistent Anti-Muslim Bias in Large Language Models
Abid, A., Farooqi, M., & Zou, J. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462624 · Cited by 476 · cited 5×
“14 and 15 and Supplementary Tables 17–19).”
InfluentialHealth system-scale language models are all-purpose prediction engines
Jiang, L. Y., Liu, X. C., Nejatian, N. P., Nasir-Moin, M., Wang, D., Abidin, A., Eaton, K., Riina, H. A., Laufer, I., Punjabi, P., et al. (2023) · Nature · doi:10.1038/s41586-023-06160-y · Cited by 460
Ethics of AI-enabled recruiting and selection: a review and research agenda
Hunkenschroer, A. L., & Luetge, C. (2022) · Journal of Business Ethics · doi:10.1007/s10551-022-05049-6 · Cited by 447
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Nangia, N., Vania, C., Bhalerao, R., & Bowman, S. R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.154 · Cited by 446
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.05862 · Cited by 442 · cited 3×
“As a second potential way to resolve dialect prejudice in language models, we examined training with HF 49,62 .”
Hudson, R. A. (1996) · Cambridge University Press eBooks · doi:10.1017/cbo9781139166843 · Cited by 433
Using machine learning to predict decisions of the European Court of Human Rights
Medvedeva, M., Vols, M., & Wieling, M. (2020) · Artificial Intelligence and Law · doi:10.1007/s10506-019-09255-y · Cited by 429
Racial Bias in Hate Speech and Abusive Language Detection Datasets
Davidson, T., Bhattacharya, D., & Weber, I. (2019) · Proceedings of the Third Workshop on Abusive Language Online · doi:10.18653/v1/w19-3504 · Cited by 414 · cited 1×
“…of resources to AAE speakers 81 (Barocas et al., unpublished observations), and adds to known cases of language technology putting speakers of AAE at a disadvantage by performing worse on AAE 82–88 , misclassifying AAE as hate speech 81,89–91 or treating AAE as incorrect English 83,85,92 .”
Stereotype persistence and change among college students.
Gilbert, G. M. (1951) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0053696 · Cited by 412
AI-enabled recruiting: What is it and how should a manager use it?
Black, J. S., & van Esch, P. (2020) · Business Horizons · doi:10.1016/j.bushor.2019.12.001 · Cited by 403 · cited 1×
“Indeed, AI systems are already being used for personnel selection 52,53 , including automated analyses of applicants’ social-media posts 54,55 , and technologies for predicting legal outcomes are under active development 56–58 .”
Applying artificial intelligence: implications for recruitment
Upadhyay, A. K., & Khandelwal, K. (2018) · Strategic HR Review · doi:10.1108/shr-07-2018-0051 · Cited by 380 · cited 1×
“Indeed, AI systems are already being used for personnel selection 52,53 , including automated analyses of applicants’ social-media posts 54,55 , and technologies for predicting legal outcomes are under active development 56–58 .”
Rickford, J. R., & King, S. (2016) · Language · doi:10.1353/lan.2016.0078 · Cited by 374
Self-citationMeasuring Bias in Contextualized Word Representations
Kurita, K., Vyas, N., Pareek, A., Black, A. W., & Tsvetkov, Y. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3823 · Cited by 348 · cited 1×
“12 and 13 and Supplementary Tables 15 and 16).”
A Critical and Comprehensive Sociological Theory of Race and Racism
Golash-Boza, T. (2016) · Sociology of Race and Ethnicity · doi:10.1177/2332649216632242 · Cited by 321
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus
Dodge, J., Sap, M., Marasović, A., Agnew, W., Ilharco, G., Groeneveld, D., Mitchell, M., & Gardner, M. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.98 · Cited by 306 · cited 1×
“Crucially, a growing body of evidence indicates that language models pick up prejudices present in the pretraining corpus 72–75 , which would explain how they become prejudiced against speakers of AAE, and why they show varying levels of dialect prejudice as a function of the pretraining corpus.”
Demographic dialectal variation in social media: a case study of African-American English
Blodgett, S. L., Green, L., & O'Connor, B. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1120 · Cited by 291 · cited 6×
“To evaluate the familiarity of the models with AAE, we measured their perplexity on the datasets used for the two evaluation settings 83,87 .”
InfluentialEthnic and National Stereotypes: The Princeton Trilogy Revisited and Revised
Madon, S., Guyll, M., Aboufadel, K., Montiel, E., Smith, A., Palumbo, P., & Jussim, L. (2001) · Personality and Social Psychology Bulletin · doi:10.1177/0146167201278007 · Cited by 276
Use of Black English and Racial Discrimination in Urban Housing Markets: New Methods and Findings
Massey, D. S., & Lundy, G. (2001) · Urban Affairs Review · doi:10.1177/10780870122184957 · Cited by 271
Holistic Evaluation of Language Models
Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.09110 · Cited by 255
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.11446 · Cited by 235 · cited 4×
“It is worth mentioning that the lack of awareness of covert racism also manifests during evaluation, where it is common to test language models for overt racism but not for covert racism 21,63,79,80 .”
InfluentialSalazar, J., Liang, D., Nguyen, T. Q., & Kirchhoff, K. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.240 · Cited by 191
Stereotyping by omission: Eliminate the negative, accentuate the positive.
Bergsieker, H. B., Leslie, L. M., Constantine, V. S., & Fiske, S. T. (2012) · Journal of Personality and Social Psychology · doi:10.1037/a0027717 · Cited by 190
Evaluating the Underlying Gender Bias in Contextualized Word Embeddings
Basta, C., Costa-jussà, M. R., & Casas, N. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3805 · Cited by 189 · cited 5×
“14 and 15 and Supplementary Tables 17–19).”
InfluentialMock Ebonics: Linguistic racism in parodies of Ebonics on the Internet
Ronkin, M., & Karn, H. E. (1999) · Journal of Sociolinguistics · doi:10.1111/1467-9481.00083 · Cited by 178
Understanding U.S. regional linguistic variation with Twitter data analysis
Huang, Y., Guo, D., Kasakoff, A., & Grieve, J. (2016) · Computers, Environment and Urban Systems · doi:10.1016/j.compenvurbsys.2015.12.003 · Cited by 166
Blodgett, S. L., & O'Connor, B. (2017) · arXiv · doi:10.48550/arxiv.1707.00061 · Cited by 153
Feng, S., Park, C. Y., Liu, Y., & Tsvetkov, Y. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.656 · Cited by 149
Toward a Description of African American Vernacular English Dialect Regions Using “Black Twitter”
Jones, T. (2015) · American Speech · doi:10.1215/00031283-3442117 · Cited by 144 · cited 1×
“Tweets are well suited for matched guise probing because they are a rich source of dialectal variation 97–99 , especially for AAE 100–102 , but matched guise probing can be applied to any type of text.”
Scientific, Legal, and Ethical Concerns About AI-Based Personnel Selection Tools: A Call to Action
Tippins, N., Oswald, F., & McPhail, S. M. (2021) · Personnel Assessment and Decisions · doi:10.25035/pad.2021.02.001 · Cited by 127
Thomas, E. R., & Reaser, J. (2004) · Journal of Sociolinguistics · doi:10.1111/j.1467-9841.2004.00251.x · Cited by 126
Systematic patterning in phonologically‐motivated orthographic variation
Eisenstein, J. (2015) · Journal of Sociolinguistics · doi:10.1111/josl.12119 · Cited by 124
Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models
Cheng, M., Durmus, E., & Jurafsky, D. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.84 · Cited by 115
Self-citationThinking Well of African Americans: Measuring Complimentary Stereotypes and Negative Prejudice
Czopp, A. M., & Monteith, M. J. (2006) · Basic and Applied Social Psychology · doi:10.1207/s15324834basp2803_3 · Cited by 108
Do Employment Recruiters Discriminate on the Basis of Nonstandard Dialect?
Atkins, C. P. (1993) · Journal of Employment Counseling · doi:10.1002/j.2161-1920.1993.tb00168.x · Cited by 98
Röttger, P., Hofmann, V., Pyatkin, V., Hinck, M., Kirk, H., Schuetze, H., & Hovy, D. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.816 · Cited by 89
Self-citationVALUE: Understanding Dialect Disparity in NLU
Ziems, C., Chen, J., Harris, C., Anderson, J., & Yang, D. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.258 · Cited by 86
King, S. (2020) · Annual Review of Linguistics · doi:10.1146/annurev-linguistics-011619-030556 · Cited by 83
Self-citationMulti-VALUE: A Framework for Cross-Dialectal English NLP
Ziems, C., Held, W., Yang, J., Dhamala, J., Gupta, R., & Yang, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.44 · Cited by 82
Zhang, E., & Zhang, Y. (2009) · Springer eBooks · doi:10.1007/978-0-387-39940-9_482 · Cited by 82
Mapping Dialectal Variation by Querying Social Media
Doyle, G. (2014) · Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/v1/e14-1011 · Cited by 73
Ball, P. D. (1983) · Language Sciences · doi:10.1016/s0388-0001(83)80021-7 · Cited by 71
Poulain, R., Bin Tarek, M. F., & Beheshti, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594102 · Cited by 63
Challenges of studying and processing dialects in social media
Jørgensen, A., Hovy, D., & Søgaard, A. (2015) · Proceedings of the Workshop on Noisy User-generated Text · doi:10.18653/v1/w15-4302 · Cited by 62
Delobelle, P., Tokpo, E., Calders, T., & Berendt, B. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.122 · Cited by 57
Twitter Universal Dependency Parsing for African-American and Mainstream American English
Blodgett, S. L., Wei, J., & O’Connor, B. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1131 · Cited by 49
Investigating African-American Vernacular English in Transformer-Based Text Generation
Groenwold, S., Ou, L., Parekh, A., Honnavalli, S., Levy, S., Mirza, D., & Wang, W. Y. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.473 · Cited by 46
Auditing the Use of Language Models to Guide Hiring DecisionsPreprint
Gaebler, J., Goel, S., Huq, A., & Tambe, P. B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.03086 · Cited by 45
Occupational Prestige: The Status Component of Socioeconomic Status
Hughes, B. T., Srivastava, S., Leszko, M., & Condon, D. M. (2024) · Collabra: Psychology · doi:10.1525/collabra.92882 · Cited by 44
Steed, R., Panda, S., Kobren, A., & Wick, M. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.247 · Cited by 42
Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection
Gururangan, S., Card, D., Dreier, S., Gade, E., Wang, L., Wang, Z., Zettlemoyer, L., & Smith, N. A. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.165 · Cited by 42
Large language models as tax attorneys: a case study in legal capabilities emergence
Nay, J. J., Karamardian, D., Lawsky, S. B., Tao, W., Bhat, M., Jain, R., Lee, A. T., Choi, J. H., & Kasai, J. (2024) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2023.0159 · Cited by 40
Kurinec, C. A., & Weaver, C. A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.785283 · Cited by 37
Harris, C., Halevy, M., Howard, A., Bruckman, A., & Yang, D. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533144 · Cited by 35
Discovering and Categorising Language Biases in Reddit
Ferrer, X., Van Nuenen, T., Such, J. M., & Criado, N. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18048 · Cited by 34
Learning a POS tagger for AAVE-like language
Jørgensen, A., Hovy, D., & Søgaard, A. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1130 · Cited by 34
LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
Zheng, L., Chiang, W.-L., Sheng, Y., Li, T., Zhuang, S., Wu, Z., Zhuang, Y., Li, Z., Lin, Z., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.11998 · Cited by 27
Payne, K. E., Downing, J. R., & Fleming, J. C. (2000) · Journal of Technical Writing and Communication · doi:10.2190/93u1-0859-0vc3-f5lk · Cited by 23
Billings, A. C. (2005) · Journal of Black Studies · doi:10.1177/0021934704271448 · Cited by 22
Language-Agnostic Bias Detection in Language Models with Bias Probing
Köksal, A., Yalcin, O. F., Akbiyik, A., Kılavuz, M. T., Korhonen, A., & Schuetze, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.848 · Cited by 15
Huntsville, hospitals, and hockey teams: Names can reveal your location
Salehi, B., Hovy, D., Hovy, E. H., & Søgaard, A. (2017) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/w17-4415 · Cited by 13
Evaluating and Mitigating Discrimination in Language Model Decisions
Tamkin, A., Askell, A., Lovitt, L., Durmus, E., Joseph, N., Kravec, S., Nguyen, K., Kaplan, J., & Ganguli, D. (2023) · arXiv · doi:10.48550/arxiv.2312.03689 · Cited by 11
Causally Testing Gender Bias in LLMs: A Case Study on Occupational BiasPreprint
Chen, Y., Raghuram, V. C., Mattern, J., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.10678 · Cited by 9
Röttger, P., Hofmann, V., Pyatkin, V., Hinck, M., Kirk, H. R., Schütze, H., & Hovy, D. (2024) · arXiv · doi:10.48550/arxiv.2402.16786 · Cited by 7
Self-citationDialect on Trial: An Experimental Examination of Raciolinguistic Ideologies and Character Judgments
Dunbar, A., King, S., & Vaughn, C. R. (2024) · Race and Justice · doi:10.1177/21533687241258772 · Cited by 4
Self-citationOpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
Research works printed but not linked: 15
- Zhao, W. et al. WildChat: 1M ChatGPT interaction logs in the wild. In Proc. Twelfth International Conference on Learning Representations (OpenReview.net, 2024).
- Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V. & Kalai, A. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. Adv. Neural Inf. Process. Syst. 30, 4356–4364 (2016).
- Li, L. & Bamman, D. Gender and representation bias in GPT-3 generated stories. In Proc. Third Workshop on Narrative Understanding (eds Akoury, N. et al.) 48–55 (Association for Computational Linguistics, 2021).
- Grogger, J. Speech patterns and racial wage inequality. J. Hum. Resour. 46, 1–25 (2011).
- Rodriguez, J. I., Cargile, A. C. & Rich, M. D. Reactions to African-American vernacular English: do more phonological features matter? West. J. Black Stud. 28, 407–414 (2004).
- Surden, H. Artificial intelligence and law: an overview. Ga State Univ. Law Rev. 35, 1305–1337 (2019).
- Schuman, H., Steeh, C., Bobo, L. D. & Krysan, M. (eds) Racial Attitudes in America: Trends and Interpretations (Harvard Univ. Press, 1998).
- Terkel, S. Race: How Blacks and Whites Think and Feel about the American Obsession (New Press, 1992).
- Bonilla-Silva, E. The New Racism: Racial Structure in the United States, 1960s–1990s. In Race, Ethnicity, and Nationality in the United States: Toward the Twenty-First Century 1st edn (ed. Wong, P.) Ch. 4 (Westview Press, 1999).
- Ethayarajh, K., Choi, Y. & Swayamdipta, S. Understanding dataset difficulty with V-usable information. In Proc. 39th International Conference on Machine Learning (eds Chaudhuri, K. et al.) 5988–6008 (Proceedings of Machine Learning Research, 2022).
- Eisenstein, J., O’Connor, B., Smith, N. A. & Xing, E. P. A latent variable model for geographic lexical variation. In Proc. 2010 Conference on Empirical Methods in Natural Language Processing (eds Li, H. & Màrquez, L.) 1277–1287 (Association for Computational Linguistics, 2010).
- Eisenstein, J. What to do about bad language on the internet. In Proc. 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (eds Vanderwende, L. et al.) 359–369 (Association for Computational Linguistics, 2013).
- Zhao, T. Z., Wallace, E., Feng, S., Klein, D. & Singh, S. Calibrate before use: Improving few-shot performance of language models. In Proc. 38th International Conference on Machine Learning (eds Meila, M. & Zhang, T.) 12697–12706 (Proceedings of Machine Learning Research, 2021).
- Jurafsky, D. & Martin, J. H. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition (Prentice Hall, 2000).
- Santurkar, S. et al. Whose opinions do language models reflect? In Proc. 40th International Conference on Machine Learning (eds Krause, A. et al.) 29971–30004 (Proceedings of Machine Learning Research, 2023).
Cited by
Current applications and challenges in large language models for patient care: a systematic review
Busch, F., Hoffmann, L., Rueger, C., van Dijk, E. H., Kader, R., Ortiz-Prado, E., Makowski, M. R., Saba, L., Hadamitzky, M., Kather, J. N., et al. (2025) · Communications Medicine · doi:10.1038/s43856-024-00717-2 · Cited by 358
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
Anwar, U., Saparov, A., Ramírez, J. R., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., et al. (2024) · arXiv (Cornell University) · doi:10.3929/ethz-b-000706962 · Cited by 17
Explicitly unbiased large language models still form biased associations
Bai, X., Wang, A., Sucholutsky, I., & Griffiths, T. L. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2416228122 · Cited by 432
Looking Beyond the Hype: Understanding the Effects of AI on Learning
Bauer, E., Greiff, S., Graesser, A. C., Scheiter, K., & Sailer, M. (2025) · Educational Psychology Review · doi:10.1007/s10648-025-10020-8 · Cited by 153
Using natural language processing to analyse text data in behavioural science
Feuerriegel, S., Maarouf, A., Bär, D., Geissler, D., Schweisthal, J., Pröllochs, N., Robertson, C. E., Rathje, S., Hartmann, J., Mohammad, S. M., et al. (2025) · Nature Reviews Psychology · doi:10.1038/s44159-024-00392-z · Cited by 140
Metadata
- Authors
- Hofmann, V., Kalluri, P. R., Jurafsky, D., & King, S.
- Affiliations
- University of Oxford (GB)
- Stanford University (US)
- University of Chicago (US)
- Corresponding author
- Hofmann, Valentin
- King, Sharese
- Venue
- Nature
- Published
- 2024-08-28
- Volume
- 633
- Issue
- 8028
- Pages
- 147-154
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- PMC
- PMC11374696
- OpenAlex
- W4401947350
- Cited by
- 233
- References
- 89
- Citations by year
- 2026: 116 · 2025: 111 · 2024: 6
- FWCI
- 96.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Knowledge@UChicago (University of Chicago) (submittedVersion, cc-by-nc-nd)
- RePEc: Research Papers in Economics (submittedVersion)
- University of Chicago (cc-by)
- Funders
- Stanford Institute for Human-Centered Artificial Intelligence, Stanford University
- Studienstiftung des Deutschen Volkes
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Linguistics and Language › Linguistic Variation and Morphology
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- covert racism, African American English, language models, racial bias in AI, preference alignment, hiring decisions, algorithmic fairness
- MeSH terms
- Artificial Intelligence, Black or African American, Decision Making, Humans, Language, Natural Language Processing, Stereotyping, Racism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.