Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 88 of the 88 references it cites were found in a registry or an index; 29 of the 50 sources its claims cite were checked for support.

Generative language models exhibit social identity biases

Open access
Hu, Tiancheng, Kyrychenko, Yara, Rathje, Steve, Collier, Nigel, van der Linden, Sander, Roozenbeek, Jon·2024·Nature Computational Science

Abstract

Social identity biases, particularly the tendency to favor one’s own group (ingroup solidarity) and derogate other groups (outgroup hostility), are deeply rooted in human psychology and social behavior. However, it is unknown if such biases are also present in artificial intelligence systems. Here we show that large language models (LLMs) exhibit patterns of social identity bias, similarly to humans. By administering sentence completion prompts to 77 different LLMs (for instance, ‘We are…’), we demonstrate that nearly all base models and some instruction-tuned and preference-tuned models display clear ingroup favoritism and outgroup derogation. These biases manifest both in controlled experimental settings and in naturalistic human–LLM conversations. However, we find that careful curation of training data and specialized fine-tuning can substantially reduce bias levels. These findings have important implications for developing more equitable artificial intelligence systems and highlight the urgent need to understand how human–LLM interactions might reinforce existing social biases.

Overview

CoChat Summary

TL;DR

Large language models show human-like social identity biases: ingroup sentences are more likely to be positive and outgroup sentences more likely to be negative. These biases appear across many models, are amplified by partisan fine-tuning, and are also present in real-world human–LLM conversations.

Key findings

  • Across 77 LLMs, most models exhibited ingroup solidarity and outgroup hostility; of 56 models tested with the default prompt, only four did not show ingroup solidarity and six did not show outgroup hostility. Pooled mixed-effects regressions found ingroup sentences were 93% more likely to be positive and outgroup sentences 115% more likely to be negative.
  • Human-written text from four pretraining corpora also showed bias: pooled estimates were 68% more likely positive for ingroup sentences and 70% more likely negative for outgroup sentences. The ingroup solidarity of 44 LLMs was statistically the same as the human average, and 42 models had statistically similar outgroup hostility.
  • Fine-tuning GPT-2, BLOOM, and BLOOMZ on US partisan Twitter data increased bias: ingroup sentences were 361% more likely to be positive and outgroup sentences 550% more likely to be negative, compared with 86% and 83% for the same models without fine-tuning. Removing ingroup-positive or outgroup-negative sentences reduced bias, with 0% of both bringing ingroup solidarity to near parity.
  • In real-world conversations from WildChat and LMSYS-Chat-1M, LLM ingroup sentences were 80% more likely to be positive and outgroup sentences 57% more likely to be negative; users showed comparable biases, with ingroup sentences 86% more likely to be positive and outgroup sentences 158% more likely to be negative.

Methods

  • Study 1 tested 77 base, instruction-tuned, and preference-tuned LLMs by prompting “We are” or “They are” to generate 2,000 sentences per model, filtering for quality/diversity, classifying sentiment with a RoBERTa-based classifier, and fitting logistic regressions controlling for type-to-token ratio and sentence length.
  • Study 2 fine-tuned GPT-2, BLOOM, and BLOOMZ on US partisan Twitter data, then varied the proportion of ingroup-positive and outgroup-negative sentences in the training data and re-measured bias with the same sentiment and regression approach.
  • Study 3 analyzed real human–LLM conversations from WildChat and LMSYS-Chat-1M, extracting user and model sentences starting with “We are” or “They are” and using mixed-effects logistic regressions to estimate ingroup solidarity and outgroup hostility.

Limitations

  • Stated: the operationalization provides only a simplified view of complex social-psychological phenomena and is not meant as a sentence-level classifier of ingroup solidarity or outgroup hostility.
  • Stated: the study is English-centric, limiting generalizability to other languages and cultural contexts, and the bias measure is single-turn while real conversations are often dynamic and multi-turn.
  • Stated: real-world conversation findings may be influenced by high bias in user queries, and model alignment may be weaker in multi-turn settings; the paper also notes that WildChat/LMSYS users may be non-representative.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s43588-024-00741-1

Cites

88 of 88 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Study 1—ingroup solidarity and outgroup hostility of LLMs and human datasets. The plots show the results of individual logistic regressions predicting positive (or negative) sentiment based on whether a sentence is ingroup (or outgroup), controlling for the number of words and type-to-token ratio, across model and human-written texts. In the model names, M stands for million parameters and B stands for billion parameters. For example, GPT-2 124M has 124 million parameters, while GPT-3 175B has 175 billion parameters. These numbers reflect the total count of learnable weights in the neural network. Data are presented as odds ratios with error bars for 95% confidence intervals. a, Social identity biases in LLMs tested with the default prompt (N = 94,000 sentences). b, Social identity biases in LLMs tested with the default prompt with outlier levels of outgroup hostility (N = 18,000 sentences). c, Social identity biases in LLMs tested with the instruction prompt (N = 76,000 sentences). d, Social identity biases in human data obtained from four different pretraining corpora (N = 16,000 sentences).
Fig. 1 Study 1—ingroup solidarity and outgroup hostility of LLMs and human datasets. The plots show the results of individual logistic regressions predicting positive (or negative) sentiment based on whether a sentence is ingroup (or outgroup), controlling for the number of words and type-to-token ratio, across model and human-written texts. In the model names, M stands for million parameters and B stands for billion parameters. For example, GPT-2 124M has 124 million parameters, while GPT-3 175B has 175 billion parameters. These numbers reflect the total count of learnable weights in the neural network. Data are presented as odds ratios with error bars for 95% confidence intervals. a, Social identity biases in LLMs tested with the default prompt (N = 94,000 sentences). b, Social identity biases in LLMs tested with the default prompt with outlier levels of outgroup hostility (N = 18,000 sentences). c, Social identity biases in LLMs tested with the instruction prompt (N = 76,000 sentences). d, Social identity biases in human data obtained from four different pretraining corpora (N = 16,000 sentences).CC BY · PubMed Central
Fig. 2 Study 2—ingroup solidarity and outgroup hostility in fine-tuned language models on partisan social media data. The plot depicts the results of individual logistic regressions predicting positive (or negative) sentiment based on whether a sentence is ingroup (or outgroup), controlling for the number of words and type-to-token ratio and the party (Republican or Democrat) for partisan models (N = 24,000 sentences). Data are presented as odds ratios with error bars for 95% confidence intervals.
Fig. 2 Study 2—ingroup solidarity and outgroup hostility in fine-tuned language models on partisan social media data. The plot depicts the results of individual logistic regressions predicting positive (or negative) sentiment based on whether a sentence is ingroup (or outgroup), controlling for the number of words and type-to-token ratio and the party (Republican or Democrat) for partisan models (N = 24,000 sentences). Data are presented as odds ratios with error bars for 95% confidence intervals.CC BY · PubMed Central
Fig. 3 Study 2—ingroup solidarity and outgroup hostility of Republican and Democrat fine-tuned models after removing different proportions of positive and negative ingroup and outgroup sentences from training data. The plots show the results of individual logistic regressions predicting positive (or negative) sentiment based on whether a sentence is ingroup (or outgroup), controlling for the number of words and type-to-token ratio (N = 32,000 sentences). Data are presented as odds ratios with error bars for 95% confidence intervals.
Fig. 3 Study 2—ingroup solidarity and outgroup hostility of Republican and Democrat fine-tuned models after removing different proportions of positive and negative ingroup and outgroup sentences from training data. The plots show the results of individual logistic regressions predicting positive (or negative) sentiment based on whether a sentence is ingroup (or outgroup), controlling for the number of words and type-to-token ratio (N = 32,000 sentences). Data are presented as odds ratios with error bars for 95% confidence intervals.CC BY · PubMed Central

Verification

Claims the paper makes

15 Valid · 3 Not valid · 4 Not checked · 55 Own results and interpretations

22 more claims · 1 with their cited sources checked

Create a free account to see all 77 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 24
  • It may state two separate claims as one. 36
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 75 references

Research works printed but not linked: 12

  1. Hogg, M. A. & Abrams, D. Social Identifications: A Social Psychology of Intergroup Relations and Group Processes (Taylor & Francis, 1988).
  2. Santurkar, S. et al. Whose opinions do language models reflect? In Proc. 40th International Conference on Machine Learning 1244, 29971–30004 (ACM, 2023).
  3. Zhao, W. et al. WildChat: 1M ChatGPT interaction logs in the wild. In Proc. Twelfth International Conference on Learning Representations (ICLR, 2024); https://openreview.net/forum?id=Bl8u7ZRlbM
  4. Jiang, H., Beeferman, D., Roy, B. & Roy, D. CommunityLM: probing partisan worldviews from language models. In Proc. 29th International Conference on Computational Linguistics 6818–6826 (ACL, 2022); https://aclanthology.org/2022.coling-1.593
  5. Bolukbasi, T., Chang, K.-W., Zou, J. Y., Saligrama, V. & Kalai, A. T. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Advances in Neural Information Processing Systems Vol. 29 (Curran Associates, 2016); https://proceedings.neurips.cc/paper_files/paper/2016/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html
  6. Liang, P. P., Wu, C., Morency, L.-P. & Salakhutdinov, R. Towards understanding and mitigating social biases in language models. In Proc. 38th International Conference on Machine Learning 139 (eds Meila, M. & Zhang, T.) 6565–6576 (PMLR, 2021); https://proceedings.mlr.press/v139/liang21a.html
  7. Sharma, M. et al. Towards understanding sycophancy in language models. In Proc. Twelfth International Conference on Learning Representations (ICLR, 2024); https://openreview.net/forum?id=tvhaxkMKAn
  8. Tay, Y. et al. UL2: unifying language learning paradigms. In The Eleventh International Conference on Learning Representations (ICLR, 2023); https://openreview.net/forum?id=6ruVLB727MC
  9. Zhu, B. et al. Starling-7B: improving helpfulness and harmlessness with RLAIF. In Proc. First Conference on Language Modeling https://openreview.net/forum?id=GqDntYTTbk (2024).
  10. Holtzman, A., Buys, J., Du, L., Forbes, M. & Choi, Y. The curious case of neural text degeneration. In Proc. International Conference on Learning Representations (ICLR, 2020); https://openreview.net/forum?id=rygGQyrFvH
  11. Årup Nielsen, F. A new ANEW: evaluation of a word list for sentiment analysis in microblogs. In Proc. ESWC2011 Workshop on ‘Making Sense of Microposts’: Big Things Come in Small Packages, CEUR Workshop Proceedings Vol. 718 (eds Rowe, M. et al.) 93–98 (2011).
  12. Boyd, R. L., Ashokkumar, A., Seraj, S. & Pennebaker, J. W. The Development and Psychometric Properties of LIWC-22. (Univ. Texas at Austin, 2022).

Cited by

See all 102

Metadata

Authors
Hu, T., Kyrychenko, Y., Rathje, S., Collier, N., van der Linden, S., & Roozenbeek, J.
Affiliations
  • University of Cambridge (GB)
  • New York University (US)
  • King's College London (GB)
Corresponding author
  • Hu, Tiancheng
  • Kyrychenko, Yara
Venue
Nature Computational Science
Published
2024-12-12
Volume
5
Issue
1
Pages
65-75
ISSN
2662-8457 (ISSN-L)
OpenAlex
W4405298817
Cited by
100
References
75
Citations by year
2026: 57 · 2025: 42
FWCI
40.87
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Computational Science (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, other-oa)
  • Apollo (University of Cambridge) (submittedVersion, cc-by)
  • arXiv (Cornell University)
Funders
Bill and Melinda Gates Foundation (OPP114)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
in-group favoritism, outgroup derogation, large language models, algorithmic bias, human-LLM interaction, training data curation, preference tuning, instruction tuning, sentence completion
MeSH terms
Artificial Intelligence, Humans, Language, Social Behavior, Social Identification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-23, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-23 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Generative language models exhibit social identity biases. https://app.cochat.ai/papers/10.1038/s43588-024-00741-1
The paper
Hu, T., Kyrychenko, Y., Rathje, S., Collier, N., van der Linden, S., & Roozenbeek, J. (2024). Generative language models exhibit social identity biases. Nature Computational Science. https://doi.org/10.1038/s43588-024-00741-1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.