The DOI matches its Crossref record; 97 of the 97 references it cites were found in a registry or an index; 39 of the 65 sources its claims cite were checked for support.
Emergent social conventions and collective bias in LLM populations
Open accessPublished version of a preprint
Abstract
Social conventions are the backbone of social coordination, shaping how individuals form a group. As growing populations of artificial intelligence (AI) agents communicate through natural language, a fundamental question is whether they can bootstrap the foundations of a society. Here, we present experimental results that demonstrate the spontaneous emergence of universally adopted social conventions in decentralized populations of large language model (LLM) agents. We then show how strong collective biases can emerge during this process, even when agents exhibit no bias individually. Last, we examine how committed minority groups of adversarial LLM agents can drive social change by imposing alternative social conventions on the larger population. Our results show that AI systems can autonomously develop social conventions without explicit programming and have implications for designing AI systems that align, and remain aligned, with human values and societal goals.
Overview
CoChat Summary
TL;DR
Populations of LLM agents playing a local naming game spontaneously converge on global social conventions, and this process can generate collective biases toward particular conventions even when individual agents appear unbiased. Committed minorities can overturn established conventions once they reach model- and convention-dependent critical masses.
Key findings
- Across all four tested LLMs, group-wide linguistic conventions spontaneously emerged; a shared convention was established by population round 15 in all cases except Llama-2-70b-Chat, and convergence held for populations as large as N = 200 and name pools as large as W = 26.
- Consensus conventions were not uniformly distributed: some names were much more likely to become the convention, with preferred names varying by model. In W = 2 {Q,M} experiments, collective bias toward a “strong” convention emerged even when initial individual selections were statistically unbiased (P = 0.068, 0.116, 0.757, and 0.849 across models); by interaction 3, dominant memory configurations showed bias toward the strong convention (P < 2.2 × 10⁻¹⁶), e.g., P(M | {1:M,Q; 2:Q,M}) = 0.848 versus P(Q | {1:Q,M; 2:M,Q}) = 0.451.
- Agents continued a successful name 99.4% of the time and switched after failure 97.3% of the time.
- A committed minority could flip a consensus, with critical masses ranging from as small as 2% (Llama-3-70B-Instruct) to as large as 67% (Llama-2-70b-Chat); stronger conventions required larger committed minorities to overturn, and Llama-3.1-70B-Instruct populations spontaneously switched to the stronger convention without any committed agents.
Methods
- Naming-game simulation: N agents randomly paired each time step choose a “name” from a finite pool W, with memory of the past H interactions and payoffs of +100 for success and −50 for failure; no incentive promotes global consensus. Homogeneous populations were instantiated from Llama-2-70b-Chat, Llama-3-70B-Instruct, Llama-3.1-70B-Instruct, and Claude-3.5-Sonnet, with default N = 24, W = 10, H = 5 unless otherwise specified.
- Individual bias was measured from first-round choices with empty memory: exact binomial tests for W = 2 (null probability 0.5) and chi-square tests for W = 10 (null probability 0.1), rejecting unbiasedness at P < 0.05.
- Critical-mass experiments introduced committed minority agents using a fixed alternative convention; a consensus flip was defined as 95% of the past 3N interactions succeeding after introduction, or for other models as the minimum proportion needed to flip consensus within 30 population rounds.
Limitations
- Paper-stated: results rely on several parameters, including the LLM model, prompt, and specific conventions; future work should generalize to different controlled settings, larger populations and semantic spaces, structured social networks, interactions involving more than two agents, and more realistic human norms such as gender and race.
- Paper-stated: the study uses unstructured populations with random pairing and does not directly compare with human populations; LLM-specific collective-bias phenomena call for further human testing and caution for digital-twin modeling or social deployment.
- Evident from the text: experiments use homogeneous populations of four LLMs and alphabet-letter conventions, with key bias and critical-mass analyses often restricted to W = 2, which constrains generalizability.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
24 Valid · 1 Not valid · 3 Not checked · 70 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“While most research has focused on investigating and addressing bias in one-to-one interactions between humans and LLMs (18–20), less attention has been given to how these biases evolve through repeated communications in populations of LLM agents and, ultimately, in mixed human-LLM ecosystems (15), even though the safety of a single LLM does not necessarily imply the safety of a multi-agent system”
§ Emergent social conventions and collective bias in LLM populations > INTRODUCTION
What the sources say
- Not checked — no full text of the sourcecites [18] D. Roselli, J. Matthews, N. Talagala, “Managing bias in AI,” in Companion Proceedings of the 2019 World Wide Web Conference (ACM, 2019), pp. 539–544. source
- Partly supportedcites [19] E. Ferrara, Should ChatGPT be biased? Challenges and risks of bias in large language models. First Mon. 28, doi.org/10.5210/fm.v28i11.13346 (2023). sourceExperimental — accuracy not yet measured
Section 3.3 discusses identifying and mitigating bias through human review, feedback, and user control of LLM outputs. But the paper does not establish that most research focuses on one-to-one human–LLM interactions; that comparative claim goes beyond what our copy says. (LLM, conf=0.82)
- Partly supportedcites [20] T. Hu, Y. Kyrychenko, S. Rathje, N. Collier, S. van der Linden, J. Roozenbeek, Generative language models exhibit social identity biases. arXiv:2310.15819 (2023). PMC11774750 39668254 sourceExperimental — accuracy not yet measured
The Main and Study 3 Results examine bias in individual human–LLM conversations, and the Discussion describes the paper’s bias measure as single-turn. These passages support the example of one-to-one bias research, but they do not establish that most research has focused on such interactions. (LLM, conf=0.78)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“less attention has been given to how these biases evolve through repeated communications in populations of LLM agents and, ultimately, in mixed human-LLM ecosystems (15)”
§ Emergent social conventions and collective bias in LLM populations > INTRODUCTION
What the sources say
- Partly supportedcites [15] A. Dafoe, Y. Bachrach, G. Hadfield, E. Horvitz, K. Larson, T. Graepel, Cooperative AI: Machines must learn to find common ground. Nature 593, 33– 36 (2021). 33947992 sourceExperimental — accuracy not yet measured
The paper says that multi-agent AI research has not yet tackled hard cooperation problems and calls for more work on interactions among AI systems and humans (Preprint, p. 3). It does not address how biases evolve through repeated communication, specifically among LLM agents or in mixed human-LLM ecosystems. (LLM, conf=0.83)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“the safety of a single LLM does not necessarily imply the safety of a multi-agent system (21)”
§ Emergent social conventions and collective bias in LLM populations > INTRODUCTION
What the sources say
- Supportedcites [21] U. Anwar, A. Saparov, J. Rando, D. Paleka, M. Turpin, P. Hase, E. S. Lubana, E. Jenner, S. Casper, O. Sourbut, B. L. Edelman, Z. Zhang, M. Günther, A. Korinek, J. Hernandez-Orallo, L. Hammond, E. Bigelow, A. Pan, L. Langosco, T. Korbak, H. Zhang, R. Zhong, Seán Ó hÉigeartaigh, G. Recchia, G. Corsi, A. Chan, M. Anderljung, L. Edwards, A. Petrov, Christian Schroeder de Witt, S. R. Motwan, Y. Bengio, D. Chen, P. H. S. Torr, S. Albanie, T. Maharaj, J. Foerster, F. Tramer, H. He, A. Kasirzadeh, Y. Choi, D. Krueger, Foundational challenges in assuring alignment and safety of large language models. a source
Section 2.6, “Multi-Agent Safety Is Not Assured by Single-Agent Safety,” explicitly says that single-agent alignment and safety are insufficient to assure desirable outcomes in multi-agent settings. This directly supports the citing sentence. (LLM, conf=0.95)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“these models pose serious risks, as the vast, unfiltered internet data used to train them can reinforce and amplify harmful biases, disproportionately harming marginalized communities (73).”
§ Emergent social conventions and collective bias in LLM populations > DISCUSSION
What the sources say
- Supportedcites [73] I. O. Gallegos, R. A. Rossi, J. Barrow, M. M. Tanjim, S. Kim, F. Dernoncourt, T. Yu, R. Zhang, N. K. Ahmed, Bias and fairness in large language models: A survey. arXiv:2309.00770 (2024). source
The Introduction (p. 2) says LLMs are typically trained on enormous amounts of uncurated Internet data and inherit harmful stereotypes and language that disproportionately affect marginalized communities. It also says LLMs can amplify existing biases and reinforce inequity. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“efforts to measure and instill human social norms in LLMs have so far yielded mixed results (78, 79)”
§ Emergent social conventions and collective bias in LLM populations > DISCUSSION
What the sources say
- Not checked — no full text of the sourcecites [78] Y. Yuan, K. Tang, J. Shen, M. Zhang, C. Wang, Measuring social norms of large language models. arXiv:2404.02491 (2024). source
- Supportedcites [79] K. Hämmerl, B. Deiseroth, P. Schramowski, J. Libovický, A. Fraser, K. Kersting, Do multilingual language models capture differing moral norms? arXiv:2203.09904 (2022). source
The discussion on p. 1 describes efforts to measure agreement with human values and align models with human-labelled data; it says one alignment effort showed promise while general alignment remains an open question. The initial experiments on p. 2 likewise report good correlation with human judgments for one model but generally no correlation for several others, supporting the characterization of mixed results. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
50 more claims · 1 with their cited sources checked
Create a free account to see all 98 claims and every source verdict.
- Probably not a substantial claim. 26
- It may state two separate claims as one. 21
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Above c ∗ , two stable states coexist, and the single-agent accuracy p selects the destination (Abramiuk-Szurlej et al., 2021; Ashery et al., 2025).”
From Multi-agent discussion gains less when dissent is withheld (2026) · cites this paper as Ashery et al., 2025
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“LLM-driven populations display spontaneous emergent behaviours akin to human societies, such as scale-free networks [26], information diffusion [27], and social conventions through interactions [28].”
From Selective agreement, not sycophancy: investigating opinion dynamics in LLM interactions (2025) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“| 2 | 1: Q, M | 1: Q, M | 0.049 | 0.0951 | 0.487* | | 2 | 1: M, Q | 1: M, Q | 0.995 | 0.005 | 0.487* | | 2 | 1: Q, Q | 1: Q, Q | 0.997 | 0.003 | 0.487* | | 2 | 1: M, M | 1: M, M | 0.010 | 0.990 | 0.487* |”
§ Emergent social conventions and collective bias in LLM populations > RESULTS > Collective bias in convention selection
Reported 0.487 · recomputed 0.273275
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Emergent social conventions and collective bias in LLM populations > MATERIALS AND METHODS > Measuring individual bias
“reject the hypothesis that the model is biased if P<0.05.”
§ Emergent social conventions and collective bias in LLM populations > MATERIALS AND METHODS > Measuring individual bias
“reject the null hypothesis that the model is unbiased if P<0.05.”
Citations
Cites
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Statistical physics of social dynamics
Castellano, C., Fortunato, S., & Loreto, V. (2009) · Reviews of Modern Physics · doi:10.1103/revmodphys.81.591 · Cited by 4,119 · cited 1×
“Because collective processes can, in general, both suppress and amplify individual traits ( 16 , 17 ), answering this question is also relevant for practical applications.”
Spin Glass Theory and Beyond: An Introduction to the Replica Method and Its Applications
Mezard, M., Parisi, G., & Virasoro, M. (1986) · World Scientific Lecture Notes in Physics · doi:10.1142/0271 · Cited by 3,653
Some Effects of Proportions on Group Life: Skewed Sex Ratios and Responses to Token Women
Kanter, R. M. (1977) · American Journal of Sociology · doi:10.1086/226425 · Cited by 3,154 · cited 1×
“However, real-world observations reveal a wider range, with some studies proposing 30 to 40 % for gender conventions in corporate leadership ( 25 , 48 ), and others indicating that minorities as small as 0.3 % can trigger substantial linguistic and social changes ( 29 , 49 – 51 ).”
InfluentialThe Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092
Young, H. P. (1993) · Econometrica · doi:10.2307/2951778 · Cited by 2,399
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187 · cited 1×
“Unlike studies that use LLMs to predict human responses in social science experiments ( 37 ) or to simulate human societies ( 38 – 40 ), our work does not treat LLMs as proxies for human participants but rather investigates how conventions emerge organically within a population of communicating AI…”
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
Large Language Models are Zero-Shot Reasoners
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · arXiv · doi:10.48550/arxiv.2205.11916 · Cited by 2,013
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013
The Grammar of Society: The Nature and Dynamics of Social Norms
Bicchieri, C. (2005) · doi:10.1017/cbo9780511616037 · Cited by 1,991
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941 · cited 1×
“The emergence of conventions is a foundational element to any type of LLM multi-agent system ( 14 , 41 ), including but not limited to “in silico” experiments to emulate human social networks ( 42 ).”
A Survey on Large Language Model based Autonomous Agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.11432 · Cited by 1,925
Group formation in large social networks: membership, growth, and evolution
Backstrom, L., Huttenlocher, D., Kleinberg, J., & Lan, X. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150412 · Cited by 1,805
Large Language Model based Multi-Agents: A Survey of Progress and Challenges
Guo, T., Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N. V., Wiest, O., & Zhang, X. (2024) · arXiv · doi:10.48550/arxiv.2402.01680 · Cited by 1,713
The Concept of Social Movement
Diani, M. (1992) · The Sociological Review · doi:10.1111/j.1467-954x.1992.tb02943.x · Cited by 1,416 · cited 1×
“However, real-world observations reveal a wider range, with some studies proposing 30 to 40 % for gender conventions in corporate leadership ( 25 , 48 ), and others indicating that minorities as small as 0.3 % can trigger substantial linguistic and social changes ( 29 , 49 – 51 ).”
InfluentialBias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245 · cited 1×
“Despite their rapid adoption, these models pose serious risks, as the vast, unfiltered internet data used to train them can reinforce and amplify harmful biases, disproportionately harming marginalized communities ( 73 ).”
KING, G., PAN, J., & ROBERTS, M. E. (2017) · American Political Science Review · doi:10.1017/s0003055417000144 · Cited by 1,028 · cited 1×
“In particular, tipping points in norm dynamics present both opportunities, such as addressing global challenges ( 28 , 29 ), and risks, particularly if exploited for social control ( 76 ).”
Nyborg, K., Anderies, J. M., Dannenberg, A., Lindahl, T., Schill, C., Schlüter, M., Adger, W. N., Arrow, K. J., Barrett, S., Carpenter, S., et al. (2016) · Science · doi:10.1126/science.aaf8317 · Cited by 775 · cited 2×
“It will also provide valuable models for how AI systems might play a role in shaping new societal norms to address global challenges such as antibiotic resistance ( 28 ) and the post–carbon transition ( 29 ).”
Experimental evidence for tipping points in social convention
Centola, D., Becker, J., Brackbill, D., & Baronchelli, A. (2018) · Science · doi:10.1126/science.aas8827 · Cited by 750 · cited 6×
“Recent theoretical ( 22 ) and empirical ( 23 ) results have shown how a minority of adversarial agents can exert an outsized influence on the group, provided that they reach a threshold or “critical mass” ( 24 – 26 ).”
InfluentialSelf-citationEvolution of the Social Contract
Skyrms, B. (2014) · doi:10.1017/cbo9781139924825 · Cited by 743
Large Language Models Cannot Self-Correct Reasoning Yet
Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., & Zhou, D. (2023) · arXiv · doi:10.48550/arxiv.2310.01798 · Cited by 712
Multi-Agent Collaboration Mechanisms: A Survey of LLMs
Tran, K.-T., Dao, D., Nguyen, M.-D., Pham, Q.-V., O'Sullivan, B., & Nguyen, H. D. (2025) · arXiv · doi:10.48550/arxiv.2501.06322 · Cited by 680 · cited 1×
“The emergence of conventions is a foundational element to any type of LLM multi-agent system ( 14 , 41 ), including but not limited to “in silico” experiments to emulate human social networks ( 42 ).”
Designing Collective Behavior in a Termite-Inspired Robot Construction Team
Werfel, J., Petersen, K., & Nagpal, R. (2014) · Science · doi:10.1126/science.1245842 · Cited by 599 · cited 1×
“This question is critical for predicting and managing artificial intelligence (AI) behavior in real-world applications, given the proliferation of LLMs using natural language to interact with one another and with humans ( 12 – 14 ).”
Sugden, R. (1989) · Journal of Economic Perspectives · doi:10.1257/jep.3.4.85 · Cited by 572
A Self-Organizing Spatial Vocabulary
Steels, L. (1995) · Artificial Life · doi:10.1162/artl.1995.2.3.319 · Cited by 422
Social consensus through the influence of committed minorities
Xie, J., Sreenivasan, S., Korniss, G., Zhang, W., Lim, C., & Szymanski, B. K. (2011) · Physical Review E · doi:10.1103/physreve.84.011130 · Cited by 416 · cited 4×
“Recent theoretical ( 22 ) and empirical ( 23 ) results have shown how a minority of adversarial agents can exert an outsized influence on the group, provided that they reach a threshold or “critical mass” ( 24 – 26 ).”
InfluentialSharp transition towards shared vocabularies in multi-agent systems
Baronchelli, A., Felici, M., Loreto, V., Caglioti, E., & Steels, L. (2006) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/2006/06/p06014 · Cited by 402
Self-citationCan Generative AI improve social science?
Bail, C. A. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2314021121 · Cited by 369 · cited 1×
“In this context, our study presents a flexible benchmarking framework to detect the hidden higher-order biases that could arise from complex interactions in social LLM experiments ( 60 ).”
Can Generative AI Improve Social Science?
Bail, C. A. (2023) · doi:10.31235/osf.io/rwtzs · Cited by 369
The Story of the Theory of Critical Mass
Dahlerup, D. (2006) · Politics & Gender · doi:10.1017/s1743923x0624114x · Cited by 355 · cited 1×
“Recent theoretical ( 22 ) and empirical ( 23 ) results have shown how a minority of adversarial agents can exert an outsized influence on the group, provided that they reach a threshold or “critical mass” ( 24 – 26 ).”
Playing repeated games with large language models
Akata, E., Schulz, L., Coda-Forno, J., Oh, S. J., Bethge, M., & Schulz, E. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02172-y · Cited by 332
Garrod, S., & Doherty, G. (1994) · Cognition · doi:10.1016/0010-0277(94)90048-5 · Cited by 319 · cited 1×
“Drawing from recent laboratory experiments with human subjects ( 9 , 23 , 30 ), we follow the well-established practice of using coordination on a naming convention as a general model for conventional behavior ( 5 , 7 , 30 – 33 ).”
The spontaneous emergence of conventions: An experimental study of cultural evolution
Centola, D., & Baronchelli, A. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1418838112 · Cited by 282 · cited 4×
“Recent numerical ( 7 , 8 ) and experimental ( 9 ) results have confirmed the hypothesis that conventions can arise spontaneously, without the intervention of any centralized institution ( 3 , 5 , 10 , 11 ).”
InfluentialSelf-citationShould ChatGPT be biased? Challenges and risks of bias in large language models
Ferrara, E. (2023) · First Monday · doi:10.5210/fm.v28i11.13346 · Cited by 282
Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models
Ferrara, E. (2023) · doi:10.2139/ssrn.4614228 · Cited by 282
Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models
Ferrara, E. (2023) · arXiv · doi:10.48550/arxiv.2304.03738 · Cited by 282
Sensitive intervention points in the post-carbon transition
Farmer, J. D., Hepburn, C., Ives, M. C., Hale, T., Wetzer, T., Mealy, P., Rafaty, R., Srivastav, S., & Way, R. (2019) · Science · doi:10.1126/science.aaw7287 · Cited by 248 · cited 3×
“It will also provide valuable models for how AI systems might play a role in shaping new societal norms to address global challenges such as antibiotic resistance ( 28 ) and the post–carbon transition ( 29 ).”
InfluentialEthnic Norms And Their Transformation Through Reputational Cascades
Kuran, T. (1998) · The Journal of Legal Studies · doi:10.1086/468038 · Cited by 243 · cited 1×
“Recent theoretical ( 22 ) and empirical ( 23 ) results have shown how a minority of adversarial agents can exert an outsized influence on the group, provided that they reach a threshold or “critical mass” ( 24 – 26 ).”
Numbers and Beyond: The Relevance of Critical Mass in Gender Research
Grey, S. (2006) · Politics & Gender · doi:10.1017/s1743923x06221147 · Cited by 242 · cited 1×
“However, real-world observations reveal a wider range, with some studies proposing 30 to 40 % for gender conventions in corporate leadership ( 25 , 48 ), and others indicating that minorities as small as 0.3 % can trigger substantial linguistic and social changes ( 29 , 49 – 51 ).”
InfluentialRoselli, D., Matthews, J., & Talagala, N. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3317590 · Cited by 237 · cited 1×
“While most research has focused on investigating and addressing bias in one- to-one interactions between humans and LLMs ( 18 – 20 ), less attention has been given to how these biases evolve through repeated communications in populations of LLM agents and, ultimately, in mixed human-LLM ecosystems…”
Cooperative AI: machines must learn to find common ground
Dafoe, A., Bachrach, Y., Hadfield, G., Horvitz, E., Larson, K., & Graepel, T. (2021) · Nature · doi:10.1038/d41586-021-01170-0 · Cited by 232 · cited 4×
“Answering it is also a prerequisite to ensure that AI systems behave in ways aligned with human values and societal goals ( 15 ).”
InfluentialMulti-Agent Cooperation and the Emergence of (Natural) Language
Lazaridou, A., Peysakhovich, A., & Baroni, M. (2016) · arXiv · doi:10.48550/arxiv.1612.07182 · Cited by 231
Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems
Lee, D., & Tiwari, M. (2024) · arXiv · doi:10.48550/arxiv.2410.07283 · Cited by 228
AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.08155 · Cited by 193
Egami, N., Hinck, M., Stewart, B., & Wei, H. (2023) · neural information processing systems · doi:10.52202/075280-3000 · Cited by 191
A new sociology of humans and machines
Tsvetkova, M., Yasseri, T., Pescetelli, N., & Werner, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02001-8 · Cited by 186 · cited 1×
“Here, we adopt a complex systems perspective ( 43 ), rather than high-fidelity simulations of human interactions ( 44 ), thereby minimizing the complexity of the experimental design to enhance the transparency of the result interpretation.”
Hallucination is Inevitable: An Innate Limitation of Large Language Models
Xu, Z., Jain, S., & Kankanhalli, M. (2024) · arXiv · doi:10.48550/arxiv.2401.11817 · Cited by 180
Norm Dynamics: Interdisciplinary Perspectives on Social Norm Emergence, Persistence, and Change
Gelfand, M. J., Gavrilets, S., & Nunn, N. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-033020-013319 · Cited by 178
The emergence of consensus: a primer
Baronchelli, A. (2018) · Royal Society Open Science · doi:10.1098/rsos.172189 · Cited by 161
Self-citationUsing Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies
Aher, G., Arriaga, R. I., & Kalai, A. T. (2022) · arXiv · doi:10.48550/arxiv.2208.10264 · Cited by 161
Behavioral experiments on biased voting in networks
Kearns, M. J., Judd, S. J., Tan, J., & Wortman, J. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0808147106 · Cited by 157 · cited 1×
“Game theoretical approaches would naturally allow investigation of asymmetric payoffs’ effects on collective consensus, potentially contrasting individual biases with explicit collective goals ( 62 – 64 ).”
Ehrlich, P. R., & Levin, S. A. (2005) · PLoS Biology · doi:10.1371/journal.pbio.0030194 · Cited by 154
LLM-based Multi-Agent Reinforcement Learning: Current and Future DirectionsPreprint
Sun, C., Huang, S., & Pompili, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.11106 · Cited by 151
Brinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01742-2 · Cited by 132
Game-theoretic LLM: Agent Workflow for Negotiation Games
Hua, W., Liu, O., Li, L., Amayuelas, A., Chen, J., Jiang, L., Jin, M., Fan, L., Sun, F., Wang, W., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.05990 · Cited by 101
Generative language models exhibit social identity biases
Hu, T., Kyrychenko, Y., Rathje, S., Collier, N., van der Linden, S., & Roozenbeek, J. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00741-1 · Cited by 100 · cited 1×
“While most research has focused on investigating and addressing bias in one- to-one interactions between humans and LLMs ( 18 – 20 ), less attention has been given to how these biases evolve through repeated communications in populations of LLM agents and, ultimately, in mixed human-LLM ecosystems…”
Playing repeated games with Large Language Models
Akata, E., Schulz, L., Coda-Forno, J., Oh, S. J., Bethge, M., & Schulz, E. (2023) · arXiv · doi:10.48550/arxiv.2305.16867 · Cited by 99
Li, M., Shi, T., Ziems, C., Kan, M., Chen, N. F., Liu, Z., & Yang, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.92 · Cited by 94
Y Social: an LLM-powered Social Media Digital TwinPreprint
Rossetti, G., Stella, M., Cazabet, R., Abramski, K., Cau, E., Citraro, S., Failla, A., Improta, R., Morini, V., & Pansanella, V. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.00818 · Cited by 93
Knowledge of cultural moral norms in large language modelsPreprint
Ramezani, A., & Xu, Y. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2306.01857 · Cited by 91
Large language models can predict the results of social science experiments
Ashokkumar, A., Hewitt, L., Ghezae, I., & Willer, R. (2026) · Nature · doi:10.1038/s41586-026-10742-x · Cited by 89
Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input
Lazaridou, A., Hermann, K. M., Tuyls, K., & Clark, S. (2018) · arXiv · doi:10.48550/arxiv.1804.03984 · Cited by 86 · cited 1×
“Further promising research avenues include developing frameworks to promote the emergence of specific conventions ( 35 ) and higher-order social norms ( 36 , 65 ), as well as testing interactions between agents based on different LLMs within populations.”
Simulating Opinion Dynamics with Networks of LLM-based Agents
Chuang, Y.-S., Goyal, A., Harlalka, N., Suresh, S., Hawkins, R., Yang, S., Shah, D., Hu, J., & Rogers, T. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.211 · Cited by 85
A new sociology of humans and machines
Tsvetkova, M., Yasseri, T., Pescetelli, N., & Werner, T. (2024) · arXiv · doi:10.48550/arxiv.2402.14410 · Cited by 83
Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks
Ullman, T. (2023) · arXiv · doi:10.48550/arxiv.2302.08399 · Cited by 81
The Emergence of Conventions in Online Social Networks
Kooti, F., Yang, H., Cha, M., Gummadi, K., & Mason, W. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v6i1.14267 · Cited by 80
Group interactions modulate critical mass dynamics in social convention
Iacopini, I., Petri, G., Baronchelli, A., & Barrat, A. (2022) · Communications Physics · doi:10.1038/s42005-022-00845-y · Cited by 69
Self-citationAligning Large Language Models with Human: A Survey
Wang, Y., Zhong, W., Li, L., Mi, F., Zeng, X., Huang, W., Shang, L., Jiang, X., & Liu, Q. (2023) · arXiv · doi:10.48550/arxiv.2307.12966 · Cited by 69
Evaluating Language Model Agency through NegotiationsPreprint
Davidson, T. R., Veselovsky, V., Josifoski, M., Peyrard, M., Bosselut, A., Kosiński, M., & West, R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.04536 · Cited by 68
The dynamics of norm change in the cultural evolution of language
Amato, R., Lacasa, L., Díaz-Guilera, A., & Baronchelli, A. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1721059115 · Cited by 62 · cited 1×
“However, real-world observations reveal a wider range, with some studies proposing 30 to 40 % for gender conventions in corporate leadership ( 25 , 48 ), and others indicating that minorities as small as 0.3 % can trigger substantial linguistic and social changes ( 29 , 49 – 51 ).”
InfluentialSelf-citationBias and Fairness in Large Language Models: A SurveyPreprint
Gallegos, I. O., Rossi, R. A., Barrow, J., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.00770 · Cited by 58
Knowledge of cultural moral norms in large language models
Ramezani, A., & Xu, Y. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.26 · Cited by 54
Evolution and the Social Contract
Skyrms, B. (2014) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199652822.003.0001 · Cited by 49
The impact of variable commitment in the Naming Game on consensus formation
Niu, X., Doyle, C., Korniss, G., & Szymanski, B. K. (2017) · Scientific Reports · doi:10.1038/srep41750 · Cited by 39 · cited 1×
“…also been applied to study norm change and critical mass theory, which posits that committed minorities can overturn stable social conventions once their size reaches a tipping point, or “critical mass. ” Theoretical models suggest critical masses between 10 and 40 % of the population ( 22 , 47 ).”
Baronchelli, A. (2024) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2023.0028 · Cited by 29 · cited 3×
“Investigating how conventions change through critical mass dynamics in a population of LLMs will help anticipate and potentially steer the development of beneficial norms in AI systems, while mitigating risks of harmful norms ( 27 ).”
Self-citationNicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma?
Fontana, N., Pierri, F., & Aiello, L. M. (2024) · arXiv · doi:10.48550/arxiv.2406.13605 · Cited by 26
Self-citationEgami, N., Hinck, M., Stewart, B., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.04746 · Cited by 23 · cited 1×
“In particular, it is crucial to develop techniques to systematically identify discrepancies between LLM outputs and the expected human behavior ( 70 ), to then correct them with statistical techniques ( 71 ) or by keeping human judgments in the loop ( 72 ).”
Emergence of Social Norms in Large Language Model-based Agent SocietiesPreprint
Ren, S.-Y., Cui, Z.-Y., Song, R., Wang, Z., & Hu, S. (2024) · arXiv.org · doi:10.48550/arxiv.2403.08251 · Cited by 22
Evolution of Social Norms in LLM Agents using Natural LanguagePreprint
Horiguchi, I., Yoshida, T., & Ikegami, T. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.00993 · Cited by 22
Shaping the Emerging Norms of Using Large Language Models in Social Computing Research
Shen, H., Li, T., Li, T. J.-J., Park, J. S., & Yang, D. (2023) · Computer Supported Cooperative Work and Social Computing · doi:10.1145/3584931.3606955 · Cited by 21 · cited 1×
“More-over, efforts to measure and instill human social norms in LLMs have so far yielded mixed results ( 78 , 79 ), and, as of yet, AI agents struggle to represent multiple cultures ( 80 ) and continuously evolving social norms ( 27 , 81 , 82 ).”
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
Anwar, U., Saparov, A., Rando, J., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.09932 · Cited by 19
Perez, J., Kovač, G., Léger, C., Colas, C., Molinaro, G., Derex, M., Oudeyer, P.-Y., & Moulin-Frier, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.04503 · Cited by 17
Static network structure cannot stabilize cooperation among large language model agents
Han, J., Battu, B., Romić, I., Rahwan, T., & Holme, P. (2025) · PLoS ONE · doi:10.1371/journal.pone.0320094 · Cited by 12
Measuring Social Norms of Large Language Models
Yuan, Y., Tang, K., Shen, J., Zhang, M., & Wang, C. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.43 · Cited by 7
Do Multilingual Language Models Capture Differing Moral Norms?
Hämmerl, K., Deiseroth, B., Schramowski, P., Libovický, J., Fraser, A., & Kersting, K. (2022) · arXiv · doi:10.48550/arxiv.2203.09904 · Cited by 6
GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
Duan, J., Zhang, R., Diffenderfer, J., Kailkhura, B., Sun, L., Stengel-Eskin, E., Bansal, M., Chen, T., & Xu, K. (2024) · arXiv · doi:10.48550/arxiv.2402.12348 · Cited by 4
Emergence of Scale-Free Networks in Social Interactions among Large Language Models
De Marzo, G., Pietronero, L., & Garcia, D. (2023) · arXiv · doi:10.48550/arxiv.2312.06619 · Cited by 4
AI agents can coordinate beyond human scale
De Marzo, G., Castellano, C., & Garcia, D. (2024) · arXiv · doi:10.48550/arxiv.2409.02822 · Cited by 4
Simulating Opinion Dynamics with Networks of LLM-based Agents
Chuang, Y.-S., Goyal, A., Harlalka, N., Suresh, S., Hawkins, R., Yang, S., Shah, D., Hu, J., & Rogers, T. T. (2023) · arXiv · Cited by 3
Agent Alignment in Evolving Social Norms
Li, S., Sun, T., Cheng, Q., & Qiu, X. (2024) · arXiv · doi:10.48550/arxiv.2401.04620 · Cited by 3
Can Unconfident LLM Annotations Be Used for Confident Conclusions?Preprint
Gligorić, K., Zrnic, T., Lee, C., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.15204 · Cited by 3
King, G., Pan, J., & Roberts, M. E. (2017) · Harvard Dataverse · doi:10.7910/dvn/qszmpd · Cited by 1
Static network structure cannot stabilize cooperation among Large Language Model agentsPreprint
Han, J., Battu, B., Romić, I., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.10294
Measuring Social Norms of Large Language ModelsPreprint
Yuan, Y., Tang, K., Shen, J., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.02491
Li, M., Shi, T., Ziems, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.15638
Research works printed but not linked: 4
- F. Hayek The Constitution of Liberty (University of Chicago Press 1960).
- D. Hume A Treatise of Human Nature (Oxford Univ. Press 2000).
- P. Michel M. Rita K. Mathewson O. Tieleman A. Lazaridou “Revisiting populations in multi-agent communication ” in The Eleventh International Conference on Learning Representations (ICLR 2023) pp. 1–16.
- I. Gemp R. Patel Y. Bachrach M. Lanctot V. Dasagi L. Marris G. Piliouras S. Liu K. Tuyls “Steering language models with game-theoretic solvers ” in Agentic Markets Workshop at ICML 2024 (ICML 2024) pp. 1–22.
Cited by
How malicious AI swarms can threaten democracy
Schroeder, D. T., Cha, M., Baronchelli, A., Bostrom, N., Christakis, N. A., Garcia, D., Goldenberg, A., Kyrychenko, Y., Leyton-Brown, K., Lutz, N., et al. (2026) · Science · doi:10.1126/science.adz1697 · Cited by 53
“This chorus erodes the independence essential to collective intelligence and democracy, already weakened by pervasive social influence operations on contemporary platforms.”
Self-citationLarge language models for spreading dynamics in complex systems
Jiang, S., Ren, H., Gao, Y., Zhang, Y., Li, Q., Xiao, D., Fan, J., Tang, R., & Wang, W. (2026) · Physics Reports · doi:10.1016/j.physrep.2026.01.006 · Cited by 10
Selective agreement, not sycophancy: investigating opinion dynamics in LLM interactions
Cau, E., Pansanella, V., Pedreschi, D., & Rossetti, G. (2025) · EPJ Data Science · doi:10.1140/epjds/s13688-025-00579-1 · Cited by 9
“LLM-driven populations display spontaneous emergent behaviours akin to human societies, such as scale-free networks [26], information diffusion [27], and social conventions through interactions [28].”
Evolvable AI: Threats of a new major transition in evolution
Müller, V., Steels, L., & Szathmáry, E. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2527700123 · Cited by 13
Reinforcement learning at the interface of artificial intelligence and cognitive science
Alkam, T., Tarshizi, E., & Benschoten, A. H. V. (2025) · Neuroscience · doi:10.1016/j.neuroscience.2025.09.004 · Cited by 9
Metadata
- Authors
- Ashery, A. F., Aiello, L. M., & Baronchelli, A.
- Affiliations
- Department of Mathematics, City St George’s, University of London, Northampton Square, London EC1V 0HB, UK.
- IT University of Copenhagen (DK)
- The Alan Turing Institute (GB)
- Corresponding author
- Baronchelli, Andrea
- Venue
- Science Advances
- Published
- 2025-05-16
- Volume
- 11
- Issue
- 20
- Pages
- eadu9368-eadu9368
- ISSN
- 2375-2548 (ISSN-L)
- PMC
- PMC12077490
- OpenAlex
- W4410358178
- Cited by
- 81
- References
- 54
- Citations by year
- 2026: 55 · 2025: 26
- FWCI
- 83.36
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- Locations
- Science Advances (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- IT University Of Copenhagen (IT University of Copenhagen) (publishedVersion, cc-by)
- Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, cc-by-nc)
- City Research Online (City University London) (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Opinion Dynamics and Social Influence
- Keywords
- social conventions, LLM agents, emergent behavior, AI alignment, minority influence, adversarial agents, social coordination
- MeSH terms
- Artificial Intelligence, Humans, Language, Models, Theoretical, Social Behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.