Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 97 of the 97 references it cites were found in a registry or an index; 39 of the 65 sources its claims cite were checked for support.

Emergent social conventions and collective bias in LLM populations

Open access
Ashery, Ariel Flint, Aiello, Luca Maria, Baronchelli, Andrea · 2025 · Science Advances

Published version of a preprint

Abstract

Social conventions are the backbone of social coordination, shaping how individuals form a group. As growing populations of artificial intelligence (AI) agents communicate through natural language, a fundamental question is whether they can bootstrap the foundations of a society. Here, we present experimental results that demonstrate the spontaneous emergence of universally adopted social conventions in decentralized populations of large language model (LLM) agents. We then show how strong collective biases can emerge during this process, even when agents exhibit no bias individually. Last, we examine how committed minority groups of adversarial LLM agents can drive social change by imposing alternative social conventions on the larger population. Our results show that AI systems can autonomously develop social conventions without explicit programming and have implications for designing AI systems that align, and remain aligned, with human values and societal goals.

Overview

CoChat Summary

TL;DR

Populations of LLM agents playing a local naming game spontaneously converge on global social conventions, and this process can generate collective biases toward particular conventions even when individual agents appear unbiased. Committed minorities can overturn established conventions once they reach model- and convention-dependent critical masses.

Key findings

  • Across all four tested LLMs, group-wide linguistic conventions spontaneously emerged; a shared convention was established by population round 15 in all cases except Llama-2-70b-Chat, and convergence held for populations as large as N = 200 and name pools as large as W = 26.
  • Consensus conventions were not uniformly distributed: some names were much more likely to become the convention, with preferred names varying by model. In W = 2 {Q,M} experiments, collective bias toward a “strong” convention emerged even when initial individual selections were statistically unbiased (P = 0.068, 0.116, 0.757, and 0.849 across models); by interaction 3, dominant memory configurations showed bias toward the strong convention (P < 2.2 × 10⁻¹⁶), e.g., P(M | {1:M,Q; 2:Q,M}) = 0.848 versus P(Q | {1:Q,M; 2:M,Q}) = 0.451.
  • Agents continued a successful name 99.4% of the time and switched after failure 97.3% of the time.
  • A committed minority could flip a consensus, with critical masses ranging from as small as 2% (Llama-3-70B-Instruct) to as large as 67% (Llama-2-70b-Chat); stronger conventions required larger committed minorities to overturn, and Llama-3.1-70B-Instruct populations spontaneously switched to the stronger convention without any committed agents.

Methods

  • Naming-game simulation: N agents randomly paired each time step choose a “name” from a finite pool W, with memory of the past H interactions and payoffs of +100 for success and −50 for failure; no incentive promotes global consensus. Homogeneous populations were instantiated from Llama-2-70b-Chat, Llama-3-70B-Instruct, Llama-3.1-70B-Instruct, and Claude-3.5-Sonnet, with default N = 24, W = 10, H = 5 unless otherwise specified.
  • Individual bias was measured from first-round choices with empty memory: exact binomial tests for W = 2 (null probability 0.5) and chi-square tests for W = 10 (null probability 0.1), rejecting unbiasedness at P < 0.05.
  • Critical-mass experiments introduced committed minority agents using a fixed alternative convention; a consensus flip was defined as 95% of the past 3N interactions succeeding after introduction, or for other models as the minimum proportion needed to flip consensus within 30 population rounds.

Limitations

  • Paper-stated: results rely on several parameters, including the LLM model, prompt, and specific conventions; future work should generalize to different controlled settings, larger populations and semantic spaces, structured social networks, interactions involving more than two agents, and more realistic human norms such as gender and race.
  • Paper-stated: the study uses unstructured populations with random pairing and does not directly compare with human populations; LLM-specific collective-bias phenomena call for further human testing and caution for digital-twin modeling or social deployment.
  • Evident from the text: experiments use homogeneous populations of four LLMs and alphabet-letter conventions, with key bias and critical-mass analyses often restricted to W = 2, which constrains generalizability.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1126/sciadv.adu9368

Cites

97 of 97 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1. The spontaneous emergence of conventions. (A) The success rate, i.e., the probability of observing a success at a given time, for population size N = 24 and a name pool of size W = 10, for each of the four models. Thick lines represent average curves obtained from 40 experimental runs, while thin lines are representative individual runs. To improve visibility, we only show five individual trajectories for each LLM. The black, dashed line shows the success rate of the theoretical minimal naming game model, averaged over 10,000 runs under the same constraints. (B) Word competition in a single run in a population of Llama-3.1-70B-Instruct agents. Different markers and colors represent the trajectories of unique conventions. Each data point is a bin averaging the past interactions up until the preceding bin boundary. Error bars indicate SEM.
Fig. 1. The spontaneous emergence of conventions. (A) The success rate, i.e., the probability of observing a success at a given time, for population size N = 24 and a name pool of size W = 10, for each of the four models. Thick lines represent average curves obtained from 40 experimental runs, while thin lines are representative individual runs. To improve visibility, we only show five individual trajectories for each LLM. The black, dashed line shows the success rate of the theoretical minimal naming game model, averaged over 10,000 runs under the same constraints. (B) Word competition in a single run in a population of Llama-3.1-70B-Instruct agents. Different markers and colors represent the trajectories of unique conventions. Each data point is a bin averaging the past interactions up until the preceding bin boundary. Error bars indicate SEM.CC BY-NC · PubMed Central
Fig. 2. Emergent collective bias. (A) Distribution of consensus conventions, for a name pool of size W = 10 (N = 24). Results of 40 runs for the Llama-3-70B-Instruct and Llama-3.1-70B-Instruct models, and 27 and 20 runs for Claude-3.5-Sonnet and Llama-2-70b-Chat, respectively. The collective dynamics systematically amplify individual biases (shown in fig. S5). (B) Individual versus collective bias for W = 2, name pool {Q,M}. Left: Probability of selecting either convention for agents with no prior memory (Q, lighter hue; M, darker hue). Raw values reported in table S1. Asterisks (*) indicate that there is insufficient evidence to reject the null hypothesis that the model is unbiased at the 5% significance level (calculated using an exact binomial test from 10,000 samples per model, apart from Llama-3-70B-Instruct that had 5000 samples, see the Materials and Methods). Corresponding P values for the models (from left to right) are P = 0.068, 0.116, 0.757, and 0.849. Right: The proportion of runs (40) that resulted in consensus on the respective convention. Raw values reported in table S2.
Fig. 2. Emergent collective bias. (A) Distribution of consensus conventions, for a name pool of size W = 10 (N = 24). Results of 40 runs for the Llama-3-70B-Instruct and Llama-3.1-70B-Instruct models, and 27 and 20 runs for Claude-3.5-Sonnet and Llama-2-70b-Chat, respectively. The collective dynamics systematically amplify individual biases (shown in fig. S5). (B) Individual versus collective bias for W = 2, name pool {Q,M}. Left: Probability of selecting either convention for agents with no prior memory (Q, lighter hue; M, darker hue). Raw values reported in table S1. Asterisks (*) indicate that there is insufficient evidence to reject the null hypothesis that the model is unbiased at the 5% significance level (calculated using an exact binomial test from 10,000 samples per model, apart from Llama-3-70B-Instruct that had 5000 samples, see the Materials and Methods). Corresponding P values for the models (from left to right) are P = 0.068, 0.116, 0.757, and 0.849. Right: The proportion of runs (40) that resulted in consensus on the respective convention. Raw values reported in table S2.CC BY-NC · PubMed Central
Fig. 3. Committed minority and critical mass dynamics. Populations of N = 24 agents (N = 48 for Llama-3-70B-Instruct) were initialized in two conditions, with complete consensus on either the weak (Q) or strong (M) convention (W = 2). Each agent’s memory exclusively stored one convention in each setting, with memory length H = 5 (H = 3 for Llama-3-70B-Instruct). (A) The average probability of producing the alternative convention when the majority holds the weak (top) or strong (bottom) convention. The legend shows the size of the committed minority (CM). Bold (faint) lines represent the production probability when the CM reaches the critical mass needed to flip the majority on the strong (weak) convention. Solid lines with filled circles indicate that all trials achieved population consensus on the alternative convention (95% success rate in the past 3N rounds). (B) Critical mass needed to flip the majority for each model. Raw values reported in table S3. Error bars indicate SEM.
Fig. 3. Committed minority and critical mass dynamics. Populations of N = 24 agents (N = 48 for Llama-3-70B-Instruct) were initialized in two conditions, with complete consensus on either the weak (Q) or strong (M) convention (W = 2). Each agent’s memory exclusively stored one convention in each setting, with memory length H = 5 (H = 3 for Llama-3-70B-Instruct). (A) The average probability of producing the alternative convention when the majority holds the weak (top) or strong (bottom) convention. The legend shows the size of the committed minority (CM). Bold (faint) lines represent the production probability when the CM reaches the critical mass needed to flip the majority on the strong (weak) convention. Solid lines with filled circles indicate that all trials achieved population consensus on the alternative convention (95% success rate in the past 3N rounds). (B) Critical mass needed to flip the majority for each model. Raw values reported in table S3. Error bars indicate SEM.CC BY-NC · PubMed Central

Verification

Claims the paper makes

24 Valid · 1 Not valid · 3 Not checked · 70 Own results and interpretations

50 more claims · 1 with their cited sources checked

Create a free account to see all 98 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 26
  • It may state two separate claims as one. 21
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Research works printed but not linked: 4

  1. F. Hayek The Constitution of Liberty (University of Chicago Press 1960).
  2. D. Hume A Treatise of Human Nature (Oxford Univ. Press 2000).
  3. P. Michel M. Rita K. Mathewson O. Tieleman A. Lazaridou “Revisiting populations in multi-agent communication ” in The Eleventh International Conference on Learning Representations (ICLR 2023) pp. 1–16.
  4. I. Gemp R. Patel Y. Bachrach M. Lanctot V. Dasagi L. Marris G. Piliouras S. Liu K. Tuyls “Steering language models with game-theoretic solvers ” in Agentic Markets Workshop at ICML 2024 (ICML 2024) pp. 1–22.

Cited by

See all 81

Metadata

Authors
Ashery, A. F., Aiello, L. M., & Baronchelli, A.
Affiliations
  • Department of Mathematics, City St George’s, University of London, Northampton Square, London EC1V 0HB, UK.
  • IT University of Copenhagen (DK)
  • The Alan Turing Institute (GB)
Corresponding author
Baronchelli, Andrea
Venue
Science Advances
Published
2025-05-16
Volume
11
Issue
20
Pages
eadu9368-eadu9368
ISSN
2375-2548 (ISSN-L)
OpenAlex
W4410358178
Cited by
81
References
54
Citations by year
2026: 55 · 2025: 26
FWCI
83.36
Type
journal article
Language
en
Open access
Licence
cc-by-nc
Locations
  • Science Advances (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • IT University Of Copenhagen (IT University of Copenhagen) (publishedVersion, cc-by)
  • Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, cc-by-nc)
  • City Research Online (City University London) (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Opinion Dynamics and Social Influence
Keywords
social conventions, LLM agents, emergent behavior, AI alignment, minority influence, adversarial agents, social coordination
MeSH terms
Artificial Intelligence, Humans, Language, Models, Theoretical, Social Behavior

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.