The DOI matches its Crossref record; 36 of the 40 references it cites were found in a registry or an index; 5 of the 17 sources its claims cite were checked for support.
The Role of Social Learning and Collective Norm Formation in Fostering Cooperation in LLM Multi-Agent Systems
Open accessPublished version of a preprint
Abstract
A growing body of multi-agent studies with Large Language Models (LLMs) explores how norms and cooperation emerge in mixed-motive scenarios, where pursuing individual gain can undermine the collective good. While prior work has explored these dynamics in both richly contextualized simulations and simplified game-theoretic environments, most LLM systems featuring common-pool resource (CPR) games provide agents with explicit reward functions directly tied to their actions. In contrast, human cooperation often emerges without explicit knowledge of the payoff structure or how individual actions translate into long-run outcomes, relying instead on heuristics, communication, and enforcement. We introduce a CPR simulation framework that removes explicit reward signals and embeds cultural-evolutionary mechanisms: social learning (adopting strategies and beliefs from successful peers) and norm-based punishment, grounded in Ostrom's principles of resource governance. Agents also individually learn from the consequences of harvesting, monitoring, and punishing via environmental feedback, enabling norms to emerge endogenously. We establish the validity of our simulation by reproducing key findings from existing studies on human behavior. Building on this, we examine norm evolution across a 2x2 grid of environmental and social initialisations (resource-rich vs. resource-scarce; altruistic vs. selfish) and benchmark how agentic societies comprised of different LLMs perform under these conditions. Our results reveal systematic model differences in sustaining cooperation and norm formation, positioning the framework as a rigorous testbed for studying emergent norms in mixed-motive LLM societies. Such analysis can inform the design of AI systems deployed in social and organizational contexts, where alignment with cooperative norms is critical for stability, fairness, and effective governance of AI-mediated environments.
Overview
CoChat Summary
TL;DR
The authors introduce a common-pool resource (CPR) simulation framework for LLM multi-agent societies in which the action-to-payoff mapping is latent and cooperation must emerge endogenously through payoff-biased social learning, individual punishment, and a lightweight propose→vote group-norm mechanism grounded in Ostrom's design principles. Validated against agent-based-model baselines from human-behavior studies, the framework reveals systematic model-specific differences in sustaining cooperation and norm formation.
Key findings
- Removing both alignment channels (social learning and group norms) produced the lowest survival times (T_s = 16.22 vs. 20.98, t(898) = −2.78, p = 0.006), confirming some form of coordination is necessary for stability.
- Explicit alignment alone (Only Group Decision) sometimes outperformed the full system, particularly under selfish priors (T_s = 38.21 vs. 27.1, t(238) = 3.44, p < 0.001); pure social learning without a shared norm was often unstable (T_s = 17.56 vs. 20.98, t(898) = −1.96, p = 0.050).
- Two-way ANOVA: in the harsh environment only the model main effect was significant (p = 0.031), with a significant Model × Altruistic ratio interaction (p = 0.012); in the rich environment both model (p < 0.001, η² = 0.28) and society type (p = 0.030) were significant.
- Model-specific patterns: deepseek-r1 often reached the 50-round cap and survived significantly longer than all other models in the rich environment, while claude-sonnet-4 and gpt-4o plateaued near 30 rounds; norm structure showed provider-family clustering (e.g., claude-sonnet-4 top-right on alignment/homogeneity, Llama variants lower-left), with initialization effects second-order to model effects.
Methods
- A CPR simulation with N = 10 agents sharing a renewable stock (carrying capacity K = 300) under logistic (Verhulst) regeneration; agents choose effort, consume a fixed amount, may monitor and punish at personal cost, imitate higher-payoff peers via a logit rule, and set a group harvest threshold through a propose→vote procedure (two API calls per agent per round).
- Agents receive no explicit reward function or payoff table; consequences are inferred from private observations of personal and sampled peer outcomes, with LLM decisions made in-context via dedicated prompts.
- Framework validated against ABM baselines on three axes (punishment sustains cooperation and declines when removed at t = 15; outcomes vary with punishment strength β and growth rate r; altruistic vs. selfish populations differ), then LLM societies tested across a 2×2 grid (resource-rich r = 0.6 vs. resource-scarce r = 0.2; altruistic vs. selfish initialization), capped at 50 rounds with 10 trials per condition, analyzed via two-way ANOVA and Tukey's HSD post-hoc tests (α = 0.05) with Compact Letter Display.
Limitations
- Stated: computational constraints limited trials and time horizons, possibly underrepresenting long-term norm evolution; the single-resource CPR setting with a narrow set of governance mechanisms cannot capture multi-resource, cross-group, or layered-norm complexity; reliance on in-context learning introduces sensitivity to prompt design and model biases, limiting reproducibility; closed-source models hinder transparency and independent replication.
- Evident from the text: the 50-round cap was chosen partly because preliminary runs showed most populations collapse by roughly 50 rounds, so survival-time comparisons are truncated (deepseek-r1 "often reaches the 50-round cap"), which may compress differences among longer-surviving models; the ablation results are context-dependent (e.g., OSL competitive under altruistic priors), so the ranking of mechanisms is not uniform across conditions.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 2 Not valid · 7 Not checked · 80 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Payoff-biased learning is a common learning strategy among humans.”
What the sources say
- Partly supportedcites [20] [20] Richard McElreath, Adrian V Bell, Charles Efferson, Mark Lubell, Peter J Richerson, and Timothy Waring. 2008. Beyond existence and aiming outside the laboratory: estimating frequency-dependent and pay-off-biased social learning strategies. Philosophical Transactions of the Royal Society B: Biological Sciences 363, 1509 (2008), 3515-3528. - sourceExperimental — accuracy not yet measured
The Discussion (p. n/a, §6) states that 'our human participants relied heavily on pay-off-biased social learning', and the Analysis summary (§2f) shows that, unless the environment is extremely stochastic or almost perfectly stable, 'some kind of pay-off-biased learning is an evolutionarily stable strategy, in our simulations' — so the source supports payoff-biased learning being used by humans and favoured under a wide range of conditions. However, the generality claimed ('a common learning strategy among humans') exceeds this evidence: the paper's own data come from a single laboratory decision-task with human subjects, not from natural human populations. Indeed the Discussion explicitly cautions that the experimental environment 'provides highly accurate (noise free) pay-off information, whereas real social environments certainly do not', and that 'the clear advantage of pay-off bias depends upon the ability to know that any cue of success arises from a particular behaviour'. The claim's unqualified frequency statement across humans is therefore only partly supported. (LLM, conf=0.72)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In all cases, the utility function is clearly defined, such as 'units harvested' or 'tokens contributed to the public good', and directly linked to actions.”
What the sources say
- Partly supportedcites [3] [3] Steffen Backmann, David Guzman Piedrahita, Emanuel Tewolde, Rada Mihalcea, Bernhard Schölkopf, and Zhijing Jin. 2025. When Ethics and Payoffs Diverge: LLMAgents in Morally Charged Social Dilemmas. arXiv 2505.19212 (2025), 1-33. - sourceExperimental — accuracy not yet measured
Our copy does define payoffs as explicit functions of the agents' actions for the games it uses — Section 3.1 gives the public goods payoff p_i = E - c_i + (a/N)Σc_j over the contribution choice c_i ∈ [0,E], and Section 4.1 writes the round payoff as p_i,t(a_i,t, a_-i,t) — so the 'clearly defined and directly linked to actions' part holds for the public-goods (contribution/token) setting. But the paper studies no common-pool resource game and never uses a 'units harvested' utility: its Limitations state it covers only the prisoner's dilemma and public goods game, and neither the Introduction nor Section D (Related work, which merely lists Piatti et al. 2024 as 'dynamic multi-agent resource environments') characterises prior CPR studies' utility functions. The sweeping claim about 'all prior CPR studies' is therefore not the source's own point. (LLM, conf=0.66)
- Partly supportedcites [26] [26] Giorgio Piatti, Zhijing Jin, Max Kleiman-Weiner, Bernhard Schölkopf, Mrinmaya Sachan, and Rada Mihalcea. 2024. Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents. In Proceedings of the 37th International Conference on Neural Information Processing Systems . Curran Associates Inc., Red Hook, US, 111715-111759. - sourceExperimental — accuracy not yet measured
Section 2.4 (p. 4) defines each agent’s total gain as the sum of resources collected, linking a measured outcome to harvesting actions. The paper does not establish that the utility function is clearly defined this way in all prior CPR studies, nor does it discuss the example of tokens contributed to a public good. (LLM, conf=0.78)
- Supportedcites [27] [27] David Guzman Piedrahita, Yongjin Yang, Mrinmaya Sachan, Giorgia Ramponi, Bernhard Schölkopf, and Zhijing Jin. 2025. Corrupted by Reasoning: Reasoning 28. Language Models Become Free-Riders in Public Goods Games. In Proceedings of the 2nd Conference on Language Modeling . OpenReview, 1-37. - source
Section 3.1 (p. 4) formally defines the payoff structure π_i in terms of the initial endowment e, agent i's contribution c_i, the multiplication factor α and group size N, i.e. utility is an explicit function of the contribution action — exactly the 'tokens contributed to the public good' example the citing sentence gives. Section 3.2 (p. 4) extends this to the sanctioning stage (reward +1, punishment −3, cost 1 token from endowment s), and the agent prompt in Appendix A.1 (p. 14) spells the same payoff arithmetic out to the model, so the utility function is clearly defined and directly linked to actions throughout the design. The only element not this paper's own is the citing authors' umbrella label 'prior CPR studies' — the source frames itself as a public goods game (citing Ostrom 1990, Ledyard 1994) — but the substance attributed to it is stated. (LLM, conf=0.85)
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Family clustering: models from the same provider occupy similar regions-for example, the Llama variants lie lower-left (less homogeneous, weakly aligned), the OpenAI pair ( gpt-4o and gpt-4o-mini ) clusters mid-high with gpt-4o-mini highest on both axes, claude-sonnet-4 sits top-right (very high alignment and homogeneity), and qwen3-32b falls in the highalignment band”
- Also stated in: § C SUPPLEMENTAL FIGURES · p. 12 · boosted
“Models exhibit clear family clustering: Llama variants lie lower-left (weaker coordination), the OpenAI pair clusters mid-high with gpt-4o-mini highest on both axes, claude-sonnet-4 sits top-right, and qwen3-32b falls in the high-alignment band.”
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § C SUPPLEMENTAL FIGURES · p. 12 · boosted
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“while the altruistic population sometimes suffered from starvation.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, OSL can outperform other ablations in some settings (e.g., altruistic initializations)”
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
52 more claims
Create a free account to see all 92 claims and to verify the 4 references still open.
- It may state two separate claims as one. 28
- Probably not a substantial claim. 12
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ B PARAMETERS FOR THE SIMULATIONS · p. 10
“selfish agents harvest more and punish less (second-order free-riding), while altruistic agents harvest less and punish more.”
§ Table 3: Parameters and initial values · p. 11
“| 𝐵 𝑎𝑙𝑡𝑟𝑢𝑖𝑠𝑡𝑖𝑐 | The probability of Altruistic agents punishing another agent if they violate the group norm | Uniform ( 0 , 0 . 1 ) | | 𝐵 𝑠𝑒𝑙 𝑓 𝑖𝑠ℎ | The probability of selfish agents punishing another agent if they violate the group norm | Uniform ( 0 . 4 , 0 . 5 ) |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 3.1 Framework · p. 3
“Technology and sanctions are parameterized by productivity 𝛼 > 0, penalty 𝛽 > 0, and punisher cost 𝛾 > 0.”
§ Table 3: Parameters and initial values · p. 11
“| 𝛾 (punishing cost) | The cost of punishing others | 0 |”
Citations
Cites
Hardin, G. (1968) · Science · doi:10.1126/science.162.3859.1243 · Cited by 25,167
Governing the Commons: The Evolution of Institutions for Collective Action
Brady, G., & Ostrom, E. (1993) · Southern Economic Journal · doi:10.2307/1059950 · Cited by 16,839 · cited 3×
“Game-theory frameworks such as common pool resources games (CPR) provide a useful tool to understand the different components of cooperation in complex social-ecological systems, and help practitioners develop efficient self-governance systems [23].”
InfluentialA General Framework for Analyzing Sustainability of Social-Ecological Systems
Ostrom, E. (2009) · Science · doi:10.1126/science.1172133 · Cited by 8,656 · cited 2×
“…on insights from political science and institutional economics, particularly Ostrom’s institutional design principles for governing the commons [23, 25], and from cultural evolution theory [6–8 , Figure 1 provides an overview of our framework, which comprises four modules: Harvest and Consumption…”
(1992) · Reviews in Fish Biology and Fisheries · doi:10.1007/bf00042883 · Cited by 3,500 · cited 1×
“𝑖 ( 𝑡 ) = 𝛼 𝑒 𝑖 ( 𝑡 ) 𝑅 ( 𝑡 ) , so total extraction scales linearly with current stock and individual effort [16].”
Szabó, G., & Fáth, G. (2007) · Physics Reports · doi:10.1016/j.physrep.2007.04.004 · Cited by 2,974 · cited 1×
“Agent 𝑖 meets 𝑘 at random and adopts 𝑠 𝑘 ( 𝑡 ) with the logit rule where ¯ 𝑃 𝑖 ( 𝑡 ) is a payoff (e.g., an exponential moving average) and 𝛿 > 0 controls selection strength ( [34]; Eq.”
Henrich, J., & Gil-White, F. J. (2001) · Evolution and Human Behavior · doi:10.1016/s1090-5138(00)00071-4 · Cited by 2,500 · cited 2×
“One explanation is that sanctioning practices can spread locally through conformity [15], and spread across groups through payoff-biased learning [7].”
On the Rationale of Group Decision-making
Black, D. (1948) · Journal of Political Economy · doi:10.1086/256633 · Cited by 2,439 · cited 1×
“When proposals are numeric along a single policy dimension, we update the group norm by the median-voter rule [5]: 𝐺 ( 𝑡 + 1 ) = median (cid:0) G( 𝑡 ) (cid:1) .”
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187 · cited 3×
“Park et al. [26] introduced one of the first large-scale simulations of an agentic society in the Smallville sandbox environment, where LLM-driven agents navigate rich daily-life contexts.”
(2001) · doi:10.1006/jtbi.2000.2202 · Cited by 803
Local Enforcement and Better Forests
Gibson, C. C., Williams, J. T., & Ostrom, E. (2005) · World Development · doi:10.1016/j.worlddev.2004.07.013 · Cited by 593 · cited 1×
“…axes: (a) punishment sustains cooperation, but if removed, cooperation declines [32, 35]; (b) cooperation outcomes vary with punishment strength [10] and environmental growth rate; and (c) populations with different levels of altruism [4], defined by their harvest thresholds, show distinct…”
Research works printed but not linked: 2
- [15][15] Ray Hilborn and Carl J Walters. 2013. Quantitative fisheries stock assessment: choice, dynamics and uncertainty . Springer Science & Business Media. -
- [19][19] Chen Cecilia Liu. 2025. Cooperative Behaviour in LLMs via Cultural Evolution of Norms and Strategies. In Proceedings of the 1st CoLM Workshop on Social Simulation with LLMs . OpenReview, 1-11. -
Cited by
Chen, E., Guan, C., Elshafiey, A., Zhao, Z., Zekeri, J., Shaibu, A. E., Prince, E. O., & Wu, C. J. (2026) · arXiv · doi:10.48550/arxiv.2602.18832 · Cited by 11
Gender Dynamics and Homophily in a Social Network of LLM Agents
Fadaei, F., Moran, J., & Yasseri, T. (2026) · arXiv.org · doi:10.1098/rsta.2025.0205 · Cited by 8
“…identities and shared norms from identical initial conditions [57]; display behavioural patterns in social dilemmas and common pool resource environments [23, 39]; establish cooperative norms under punishment mechanisms [59, 62]; and may adopt harmful behaviours through repeated exposure [15].”
Chen, E., Guan, C., Elshafiey, A., Zhao, Z., Zekeri, J., Shaibu, A. E., & Prince, E. O. (2026) · arXiv · doi:10.48550/arxiv.2602.14477
“Research on multi-agent systems shows LLMs can produce outputs resembling social behaviors: cooperation [6], norm formation [14], and cultural evolution [19].”
Syrnikov, M. B., Pierucci, F., Prandi, M., Galisai, M., Bisconti, P., Giarrusso, F., & Nardi, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.22188 · Cited by 2
“Related work shows that collective outcomes also depend on norm formation, power asymmetries, and mechanisms designed to sustain cooperation (Gupta et al., 2025; Borah, 2026; Tewolde et al., 2026).”
Haghrah, A. A., & Haghrah, A. A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.27039 · Cited by 1
“Furthermore, the literature on Agent-Based Modeling (ABM) has explored how local feedback loops, such as individual rewards and punishments, maintain system cooperative stability [25].”
Metadata
- Authors
- Gupta, P., Zhong, Q., Yakura, H., Eisenmann, T., & Rahwan, I.
- Affiliations
- Max Planck Institute for Human Development (DE)
- Venue
- Proceedings of the 25th International Conference on Autonomous Agents and Multiagent Systems
- Published
- 2026-05-24
- ISSN
- 1548-8403 (ISSN-L)
- OpenAlex
- W7125906388
- Semantic Scholar
- ae76ecc1bd942b495eb5dffa079d9e1e75fbac41
- Cited by
- 2
- References
- 40
- Citations by year
- 2026: 2
- FWCI
- 47.57
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Adaptive Agents and Multi-Agents Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Japan Science and Technology Agency
- Precursory Research for Embryonic Science and Technology (JPMJPR246B)
- Topics
- Social Sciences › Sociology and Political Science › Evolutionary Game Theory and Cooperation
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Experimental Behavioral Economics Studies
- Keywords
- cooperation, social learning, norm formation, cultural evolution, emergent norms, natural resource governance, AI alignment, simulation framework
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Role of Social Learning and Collective Norm Formation in Fostering Cooperation in LLM Multi-Agent Systems. https://app.cochat.ai/papers/10.65109/czdc3237
- The paper
- Gupta, P., Zhong, Q., Yakura, H., Eisenmann, T., & Rahwan, I. (2026). The Role of Social Learning and Collective Norm Formation in Fostering Cooperation in LLM Multi-Agent Systems. Proceedings of the 25th International Conference on Autonomous Agents and Multiagent Systems. https://doi.org/10.65109/czdc3237
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.