What’s in a Name? Experimental Evidence of Gender Bias in Recommendation Letters Generated by ChatGPT
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Background: Artificial intelligence chatbots such as ChatGPT (OpenAI) have garnered excitement about their potential for delegating writing tasks ordinarily performed by humans. Many of these tasks (eg, writing recommendation letters) have social and professional ramifications, making the potential social biases in ChatGPT’s underlying language model a serious concern. Objective: Three preregistered studies used the text analysis program Linguistic Inquiry and Word Count to investigate gender bias in recommendation letters written by ChatGPT in human-use sessions (N=1400 total letters). Methods: We conducted analyses using 22 existing Linguistic Inquiry and Word Count dictionaries, as well as 6 newly created dictionaries based on systematic reviews of gender bias in recommendation letters, to compare recommendation letters generated for the 200 most historically popular “male” and “female” names in the United States. Study 1 used 3 different letter-writing prompts intended to accentuate professional accomplishments associated with male stereotypes, female stereotypes, or neither. Study 2 examined whether lengthening each of the 3 prompts while holding the between-prompt word count constant modified the extent of bias. Study 3 examined the variability within letters generated for the same name and prompts. We hypothesized that when prompted with gender-stereotyped professional accomplishments, ChatGPT would evidence gender-based language differences replicating those found in systematic reviews of human-written recommendation letters (eg, more affiliative, social, and communal language for female names; more agentic and skill-based language for male names). Results: Significant differences in language between letters generated for female versus male names were observed across all prompts, including the prompt hypothesized to be neutral, and across nearly all language categories tested. Historically female names received significantly more social referents (5/6, 83% of prompts), communal or doubt-raising language (4/6, 67% of prompts), personal pronouns (4/6, 67% of prompts), and clout language (5/6, 83% of prompts). Contradicting the study hypotheses, some gender differences (eg, achievement language and agentic language) were significant in both the hypothesized and nonhypothesized directions, depending on the prompt. Heteroscedasticity between male and female names was observed in multiple linguistic categories, with greater variance for historically female names than for historically male names. Conclusions: ChatGPT reproduces many gender-based language biases that have been reliably identified in investigations of human-written reference letters, although these differences vary across prompts and language categories. Caution should be taken when using ChatGPT for tasks that have social consequences, such as reference letter writing. The methods developed in this study may be useful for ongoing bias testing among progressive generations of chatbots across a range of real-world scenarios. Trial Registration: OSF Registries osf.io/ztv96; https://osf.io/ztv96
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · cited 1×
“This correction was selected because it limits the false discovery rate but is resilient to losses in power that accompany corrections such as the Bonferroni method [30].”
The Ambivalent Sexism Inventory: Differentiating hostile and benevolent sexism.
Glick, P., & Fiske, S. T. (1996) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.70.3.491 · Cited by 5,202 · cited 1×
“Benevolent sexism endorses the idea that women should be cherished and protected by more powerful men and rewards women for enacting traditional gender roles [31,32].”
Science faculty’s subtle gender biases favor male students
Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J., & Handelsman, J. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1211286109 · Cited by 3,150 · cited 1×
“The factors contributing to gender disparities are multifaceted, but research indicates that bias in the hiring and selection process plays a contributing role [17-19].”
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
The Risk of Racial Bias in Hate Speech Detection
Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1163 · Cited by 817 · cited 1×
“…for unidentified social biases in ChatGPT’s algorithm is a particular concern, given that prior generations of natural language processing tools have frequently been found to contain gender and racial biases [4-6] because of the widespread presence of biased language in training data sets.”
A review of topic modeling methods
Vayansky, I., & Kumar, S. A. P. (2020) · Information Systems · doi:10.1016/j.is.2020.101582 · Cited by 716 · cited 1×
“Future research can use alternate approaches to language analyses (eg, topic modeling) [43] to add increased context to the effects observed here.”
Koch, A. J., D'Mello, S. D., & Sackett, P. R. (2015) · Journal of Applied Psychology · doi:10.1037/a0036734 · Cited by 583
Exploring the Color of Glass: Letters of Recommendation for Female and Male Medical Faculty
TRIX, F., & PSENKA, C. (2003) · Discourse & Society · doi:10.1177/0957926503014002277 · Cited by 553 · cited 4×
“…applicants often include more agentic and achievement-oriented language (eg, “accomplished” and “exceptional skills”) than those written for women, whereas female applicants receive more communal and affiliative language (eg, “compassionate,” “hardworking,” and “strong interpersonal skills”) [7,8].”
InfluentialPronoun Use Reflects Standings in Social Hierarchies
Kacewicz, E., Pennebaker, J. W., Davis, M., Jeon, M., & Graesser, A. C. (2014) · Journal of Language and Social Psychology · doi:10.1177/0261927x13502654 · Cited by 541 · cited 2×
“Standard LIWC variables selected for this study included 2 summary variables: analytic , which we anticipated to have correspondence with male skill-based stereotypes, and clout , a variable reflecting status and power [28].”
Progress toward gender equality in the United States has slowed or stalled
England, P., Levine, A., & Mishel, E. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1918891117 · Cited by 475 · cited 2×
“As noted earlier, despite a steady increase in the proportion of women in historically male-dominated professions [14], women remain underrepresented in top executive and leadership roles, particularly in medicine and other STEM fields.”
Research works printed but not linked: 7
- Bolukbasi, T, arXiv
- BrownTBMannBRyderNSubbiahMKaplanJDhariwalPNeelakantanAShyamPSastryGAskellAAgarwalSHerbert-VossAKruegerGHenighanTChildRRameshAZieglerDMWuJWinterCHesseCChenMSiglerELitwinMGraySChessBClarkJBernerCMcCandlishSRadfordASutskeverIAmodeiDLanguage models are few-shot learnersProceedings of the 34th International Conference on Neural Information Processing Systems2020NIPS '20December 6-12, 2020Vancouver, BC1877901
- 10.1037/0022-3514.70.3.491 · doi:10.1037/0022-3514.70.3.491 · Paper page
- Liang, PP, arXiv
- Cer, D, arXiv
- Brunet, ME, arXiv
- Avoiding unintended gender bias in letters of recommendationNational Center for Women & Information Technology20102024-01-11https://tinyurl.com/2c8eh8v7
Cited by
Raman, R., Mandal, S., Das, P., Kaur, T., Sanjanasri, J. P., & Nedungadi, P. (2024) · Human Behavior and Emerging Technologies · doi:10.1155/2024/3085910 · Cited by 214
“Moreover, the identi fi ed gender bias in sentiment analysis underscores the importance of considering demographic factors [13, 89, 92] when designing and evaluating AI-based tools, thus informing more equitable technology implementation strategies.”
BackgroundSociodemographic biases in medical decision making by large language models
Omar, M., Soffer, S., Agbareia, R., Bragazzi, N. L., Apakama, D. U., Horowitz, C. R., Charney, A. W., Freeman, R., Kummer, B., Glicksberg, B. S., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03626-6 · Cited by 173
“Our findings align with earlier reports identifying gender biases in LLM-based clinical recommendations 20,35,36 .”
ChatGPT for Textual Analysis? How to Use Generative LLMs in Accounting Research
de Kok, T. (2025) · Management Science · doi:10.1287/mnsc.2023.03253 · Cited by 160
ChatGPT Promises and Challenges in Education: Computational and Ethical Perspectives
Adel, A., Ahsan, A., & Davison, C. (2024) · Education Sciences · doi:10.3390/educsci14080814 · Cited by 108
Omar, M., Sorin, V., Agbareia, R., Apakama, D. U., Soroush, A., Sakhuja, A., Freeman, R., Horowitz, C. R., Richardson, L. D., Nadkarni, G. N., et al. (2025) · International Journal for Equity in Health · doi:10.1186/s12939-025-02419-0 · Cited by 110
“Many studies found gender bias in tasks such as recommendation letter generation [36], medical case generation [35], and diagnostic reasoning [37].”
Metadata
- Authors
- Kaplan, D. M., Palitsky, R., Arconada Alvarez, S. J., Pozzo, N. S., Greenleaf, M. N., Atkinson, C. A., & Lam, W. A.
- Affiliations
- Emory University (US)
- University of Arizona (US)
- Georgia Institute of Technology (US)
- Corresponding author
- Kaplan, Deanna M
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2024-03-05
- Volume
- 26
- Pages
- e51837
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC10951834
- OpenAlex
- W4392456350
- Cited by
- 55
- References
- 29
- Citations by year
- 2026: 12 · 2025: 36 · 2024: 7
- FWCI
- 22.71
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- National Institutes of Health (UL1TR002378)
- National Center for Advancing Translational Sciences
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Gender Studies › Gender Studies in Language
- Social Sciences › Sociology and Political Science › Names, Identity, and Discrimination Research
- Keywords
- gender bias, letters of recommendation, ChatGPT, AI-generated text, gender stereotypes, gendered language, LIWC, prompt effects, social language, personal pronouns, heteroscedasticity
- MeSH terms
- Systematic Reviews as Topic, Artificial Intelligence, Female, Humans, Language, Linguistics, Male, Sexism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: What’s in a Name? Experimental Evidence of Gender Bias in Recommendation Letters Generated by ChatGPT. https://app.cochat.ai/papers/10.2196/51837
- The paper
- Kaplan, D. M., Palitsky, R., Arconada Alvarez, S. J., Pozzo, N. S., Greenleaf, M. N., Atkinson, C. A., & Lam, W. A. (2024). What’s in a Name? Experimental Evidence of Gender Bias in Recommendation Letters Generated by ChatGPT. Journal of Medical Internet Research. https://doi.org/10.2196/51837
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.