Large language models show human-like content biases in transmission chain experiments
Open accessThe DOI matches its Crossref record; 28 of the 31 references it cites were found in a registry or an index.
Abstract
As the use of large language models (LLMs) grows, it is important to examine whether they exhibit biases in their output. Research in cultural evolution, using transmission chain experiments, demonstrates that humans have biases to attend to, remember, and transmit some types of content over others. Here, in five preregistered experiments using material from previous studies with human participants, we use the same, transmission chain-like methodology, and find that the LLM ChatGPT-3 shows biases analogous to humans for content that is gender-stereotype-consistent, social, negative, threat-related, and biologically counterintuitive, over other content. The presence of these biases in LLM output suggests that such content is widespread in its training data and could have consequential downstream effects, by magnifying preexisting human tendencies for cognitively appealing and not necessarily informative, or valuable, content.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In the end, LLMs are trained on large datasets that inevitably contain biases—such as gender, racial, or cultural biases—embedded in the text140.”
From Current applications and challenges in large language models for patient care: a systematic review (2025) · cites this paper as [140]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fitting Linear Mixed-Effects Models Using lme4
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Advances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 8,890
Culture and the Evolutionary Process
Griesemer, J. R., Boyd, R., & Richerson, P. J. (1986) · The Condor · doi:10.2307/1367778 · Cited by 7,034
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526
Algorithmic Bias in Autonomous Systems
Danks, D., & London, A. J. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/654 · Cited by 472
Dale, R. (2021) · Natural Language Engineering · doi:10.1017/s1351324920000601 · Cited by 432
The multiple roles of cultural transmission experiments in understanding human cultural evolution
Mesoudi, A., & Whiten, A. (2008) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0129 · Cited by 394
A bias for social information in human cultural transmission
Mesoudi, A., Whiten, A., & Dunbar, R. (2006) · British Journal of Psychology · doi:10.1348/000712605x85871 · Cited by 349
Research works printed but not linked: 2
- F. C. Bartlett, Remembering (Cambridge University Press, 1932).
- 10.26613/esic.1.1.20 · doi:10.26613/esic.1.1.20
Cited by
Bedi, S., Liu, Y., Orr-Ewing, L., Dash, D., Koyejo, S., Callahan, A., Fries, J. A., Wornow, M., Swaminathan, A., Lehmann, L. S., et al. (2024) · doi:10.1101/2024.04.15.24305869 · Cited by 738
“This phenomenon can be attributed to the tendency of LLMs to reproduce information from their training data, which may contain human biases 60 .”
Testing and Evaluation of Health Care Applications of Large Language Models: A Systematic Review
Bedi, S., Liu, Y., Orr-Ewing, L., Dash, D., Koyejo, S., Callahan, A., Fries, J. A., Wornow, M., Swaminathan, A., Lehmann, L. S., et al. (2025) · JAMA · doi:10.1001/jama.2024.21700 · Cited by 662
Current applications and challenges in large language models for patient care: a systematic review
Busch, F., Hoffmann, L., Rueger, C., van Dijk, E. H., Kader, R., Ortiz-Prado, E., Makowski, M. R., Saba, L., Hadamitzky, M., Kather, J. N., et al. (2025) · Communications Medicine · doi:10.1038/s43856-024-00717-2 · Cited by 358
Large language models empowered agent-based modeling and simulation: a survey and perspectives
Gao, C., Lan, X., Li, N., Yuan, Y., Ding, J., Zhou, Z., Xu, F., & Li, Y. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03611-3 · Cited by 294
“Acerbi and Stubbers fi eld (2023) demonstrate that the information transmitted by large language models mirrors the biases inherent in human social communication.”
Using LLMs for Market Research
Brand, J., Israeli, A., & Ngwe, D. (2023) · doi:10.2139/ssrn.4395751 · Cited by 190
Metadata
- Authors
- Acerbi, A., & Stubbersfield, J. M.
- Affiliations
- University of Trento (IT)
- University of Winchester (GB)
- Corresponding author
- Acerbi, Alberto
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2023-10-26
- Volume
- 120
- Issue
- 44
- Pages
- e2313790120-e2313790120
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC10622889
- OpenAlex
- W4387952963
- Cited by
- 137
- References
- 31
- Citations by year
- 2026: 45 · 2025: 49 · 2024: 38 · 2023: 5
- FWCI
- 34.28
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Iris (University of Trento) (submittedVersion, cc-by-nc-nd)
- OSF Preprints (OSF Preprints) (submittedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion, cc-by)
- Funders
- University of Winchester
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Cultural Studies › Language and cultural evolution
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models, ChatGPT-3, content bias, cultural evolution, gender stereotypes, training data bias, social content
- MeSH terms
- Cultural Evolution, Humans, Language, Mental Recall, Bias, Ethical Theory
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Large language models show human-like content biases in transmission chain experiments. https://app.cochat.ai/papers/10.1073/pnas.2313790120
- The paper
- Acerbi, A., & Stubbersfield, J. M. (2023). Large language models show human-like content biases in transmission chain experiments. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.2313790120
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.