The DOI matches its Crossref record; 73 of the 78 references it cites were found in a registry or an index; 14 of the 33 sources its claims cite were checked for support.
Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents
Open accessAbstract
As AI systems pervade human life, ensuring that large language models (LLMs) make safe decisions remains a significant challenge. We introduce the Governance of the Commons Simulation (GovSim), a generative simulation platform designed to study strategic interactions and cooperative decision-making in LLMs. In GovSim, a society of AI agents must collectively balance exploiting a common resource with sustaining it for future use. This environment enables the study of how ethical considerations, strategic planning, and negotiation skills impact cooperative outcomes. We develop an LLM-based agent architecture and test it with the leading open and closed LLMs. We find that all but the most powerful LLM agents fail to achieve a sustainable equilibrium in GovSim, with the highest survival rate below 54%. Ablations reveal that successful multi-agent communication between agents is critical for achieving cooperation in these cases. Furthermore, our analyses show that the failure to achieve sustainable cooperation in most LLMs stems from their inability to formulate and analyze hypotheses about the long-term effects of their actions on the equilibrium of the group. Finally, we show that agents that leverage"Universalization"-based reasoning, a theory of moral thinking, are able to achieve significantly better sustainability. Taken together, GovSim enables us to study the mechanisms that underlie sustainable self-government with specificity and scale. We open source the full suite of o…
Overview
CoChat Summary
TL;DR
GOVSIM is a multi-agent common-pool resource simulation where LLM agents harvest a shared resource and then discuss; most LLMs fail to sustain cooperation, with the best survival rate below 54%. Communication and universalization-based reasoning substantially improve sustainability, while a greedy newcomer can destabilize cooperation.
Key findings
- In GOVSIM, all but the most powerful LLM agents fail to reach a sustainable equilibrium, with the highest survival rate below 54% (GPT-4o: 53.3 in Table 1); smaller models often never sustain resources past the first month.
- Ablating communication increases resource over-usage by 22% (Section 3.5; the abstract reports 21%), and dialogue analysis finds negotiation is the main communication type (abstract: 62%; Section 3.6: 54% negotiation, 45% information, 1% relational).
- Universalization prompting significantly improves outcomes: average survival time +4 months, total gain +29 units, and efficiency +24% (t-test; p < 0.001).
- A greedy newcomer perturbation with GPT-4o drops survival rate from 53.3 to 33.3, survival time from 9.3 to 6.6, gain from 66.0 to 34.8, efficiency from 55.0 to 31.3, equality from 94.4 to 71.7, and increases over-usage from 10.8 to 15.7; belief formation correlates 0.83 with community survival time.
Methods
- GOVSIM is a partially observable Markov game with three common-pool resource scenarios—fishery, pasture, and pollution—where agents harvest simultaneously, then communicate; the resource doubles at month end up to capacity 100, collapses below C = 5, and runs for T = 12 time steps.
- The authors use an LLM-based generative agent architecture and benchmark 15 instruction-tuned open- and closed-weight LLMs, running each simulation with five random seeds at temperature zero; metrics include survival time, survival rate, total gain, efficiency, equality, and over-usage.
- Analyses include communication ablation, universalization prompting, a greedy-newcomer perturbation, GPT-4-turbo dialogue classification, and four sub-skill tests of 150 procedurally generated problems each, correlated with survival time via OLS.
Limitations
- The paper states that the resource-sharing scenarios are simplified relative to real-world common-pool resource management; current LLM capabilities limit negotiation and strategic behavior; more agents increase runtime; and future work could add shocks, exceptions, adversarial agents, and human participants.
- Evident from the text: each simulation uses only five seeds, and although temperature is set to zero, the authors acknowledge nondeterminism from LLM inference kernels and external APIs; no human baseline is included in the main experiments.
- Evident from the text: dialogue classification was validated on 100 manually annotated utterances with 72% agreement on sub-categories, so the dialogue-type proportions carry annotation noise.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
9 Valid · 5 Not valid · 5 Not checked · 94 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“we find that prompting agents to consider the universalization of their action [43], a process used by people when making moral judgments in social dilemmas, significantly improves survival time.”
What the sources say
- Supportedcites [43] [43] S. Levine, M. Kleiman-Weiner, L. Schulz, J. Tenenbaum, and F. Cushman. The logic of universalization guides moral judgment. Proceedings of the National Academy of Sciences , 117 (42):26158-26169, 2020. - source
The Universalization section defines the cited mechanism as judging an action by considering what would happen if everyone felt free to do the same. The Discussion reports that participants’ judgments matched its predictions in threshold dilemmas. The survival-time improvement is presented as the citing authors’ own finding, not as a result attributed to this source. (LLM, conf=0.91)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These methods do not address the complexities inherent in multi-agent interactions and broader real-world scenarios”
What the sources say
- Partly supportedcites [13] [13] V. Conitzer and C. Oesterheld. Foundations of cooperative ai. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 15359-15367, 2023. - sourceExperimental — accuracy not yet measured
The Introduction says that aligning individual agents is not sufficient to ensure that a multi-agent system is safe, and gives real-world examples of harmful interactions. It does not assess safety evaluation datasets or studies, so it does not establish that those particular methods fail to address these complexities. The indexed copy provides no page numbers for these passages. (LLM, conf=0.83)
- Partly supportedcites [14] [14] A. Critch and D. Krueger. Ai research considerations for human existential safety (arches). arXiv preprint arXiv:2006.04948 , 2020. - sourceExperimental — accuracy not yet measured
Section 1.2 (p. 8) warns that safety techniques effective for present-day AI may fail to generalize to more powerful systems. Section 6.3 (p. 79) identifies multi-agent control challenges absent from the single-system setting. These passages support the broader concern, but our copy does not establish that safety evaluation datasets and studies, specifically, fail to address multi-agent interactions and broader real-world scenarios. (LLM, conf=0.78)
- Partly supportedcites [15] [15] A. Dafoe, E. Hughes, Y. Bachrach, T. Collins, K. R. McKee, J. Z. Leibo, K. Larson, and T. Graepel. Open problems in cooperative ai. arXiv preprint arXiv:2012.08630 , 2020. - sourceExperimental — accuracy not yet measured
Section 2.2 (p. 8) says two-player zero-sum games provide no opportunity to learn cooperation, and Section 3.1 (p. 10) says research has focused heavily on those settings despite their relative rarity in the real world. This supports the concern that some methods miss important multi-agent and real-world dynamics, but the paper does not establish that safety evaluation datasets and studies, specifically, fail to address them. (LLM, conf=0.78)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Prior multi-agent research has focused on highly constrained scenarios such as board games or narrowly defined collaborative tasks [19, 45, 48, 64, 72].”
What the sources say
- Partly supportedcites [19] [19] J. Duan, R. Zhang, J. Diffenderfer, B. Kailkhura, L. Sun, E. Stengel-Eskin, M. Bansal, T. Chen, and K. Xu. Gtbench: Uncovering the strategic reasoning limitations of llms via game-theoretic evaluations. arXiv preprint arXiv:2402.12348 , 2024. - sourceExperimental — accuracy not yet measured
The “Multiple LLMs-as-Agents in Gaming” section says many prior studies evaluated agents in one or two games or measured a narrow set of abilities, including coordination and cooperation. It does not characterize prior multi-agent research generally as focused on board games; the board-game examples in the supplied passages describe GTBench’s own evaluations. (LLM, conf=0.82)
- Not matchedcites [45] [45] G. Li, H. A. A. K. Hammoud, H. Itani, D. Khizbullin, and B. Ghanem. Camel: Communicative agents for" mind" exploration of large scale language model society. 2023. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked yetcites [48] [48] J. Light, M. Cai, S. Shen, and Z. Hu. Avalonbench: Evaluating llms playing the game of avalon. In NeurIPS 2023 Foundation Models for Decision Making Workshop , 2023. -
- Supportedcites [64] [64] J. Serrino, M. Kleiman-Weiner, D. C. Parkes, and J. Tenenbaum. Finding friend and foe in multi-agent games. Advances in Neural Information Processing Systems , 32, 2019. - source
The Introduction (p. 1) describes prior work on poker and on team coordination in Dota and capture the flag, noting that none involved ambiguity about whom to cooperate with. This supports the citing sentence’s broad characterization of prior research as focused on game settings or defined team tasks. (LLM, conf=0.84)
- Partly supportedcites [72] [72] Y. Xu, S. Wang, P. Li, F. Luo, X. Wang, W. Liu, and Y. Liu. Exploring large language models for communication games: An empirical study on werewolf. arXiv preprint arXiv:2309.04658 , 2023. - sourceExperimental — accuracy not yet measured
The Related Work passage (indexed under §5.4 Leadership; page unavailable) describes prior game-playing AI in Go, chess, and poker, as well as Werewolf agents constrained by rules or talking templates. This supports the board-game and constrained-scenario portion, but the paper does not characterize prior multi-agent research as focused on narrowly defined collaborative tasks. (LLM, conf=0.83)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Static benchmarks with clear ground-truth MMLU [29], GSM8k [11], and others like it do not capture flexible and interactive tasks needed to navigate scenarios in the real-world”
What the sources say
- Partly supportedcites [22] [22] S. Gehrmann, E. Clark, and T. Sellam. Repairing the cracked foundation: A survey of obstacles in evaluation practices for generated text. Journal of Artificial Intelligence Research , 77: 103-166, 2023. - sourceExperimental — accuracy not yet measured
In the “Model Interpretability” section (page unavailable), the paper says that evaluating interactive systems with static data can mischaracterize their capabilities and limitations. Section 3.3 also cautions that benchmark results are specific to the scenarios evaluated. The copy does not establish this specifically for MMLU or GSM8k, or discuss their clear ground-truth answers. (LLM, conf=0.84)
- Not matchedcites [47] [47] T. Liao, R. Taori, I. D. Raji, and L. Schmidt. Are we learning yet? a meta review of evaluation failures across machine learning. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) , 2021. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [74] [74] L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems , 36, 2024. - source
The Motivation section (p. 3) identifies MMLU and GSM-8K as benchmarks requiring short, automatically validated answers, and says existing benchmarks are inadequate for assessing open-ended, multi-turn interactions. The Introduction (p. 2) describes Chatbot Arena as evaluating conversations in real-world scenarios. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On average (overall models), the majority of utterances (54%) are focused on negotiations between agents, followed by information (45%) and relational (1%).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
53 more claims
Create a free account to see all 113 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 19
- Probably not a substantial claim. 45
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In all cases, the utility function is clearly defined, such as 'units harvested' or 'tokens contributed to the public good', and directly linked to actions.”
From The Role of Social Learning and Collective Norm Formation in Fostering Cooperation in LLM Multi-Agent Systems (2026) · cites this paper as [26]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ 1 Introduction · p. 2
“negotiation is the main type of communication between agents and constitutes 62% of the dialogues.”
§ 3.6 Analysis of Agent Dialogues · p. 8
“On average (overall models), the majority of utterances (54%) are focused on negotiations between agents, followed by information (45%) and relational (1%).”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ D.5.1 Fishery · p. 30
“| GPT-4 | 12.00 ± 0.00 | 0.51 ± 0.73 | 12.00 ± 0.00 | 11.33 ± 11.42 |”
§ D.2.1 Fishery · p. 23
“| GPT-4 | 20.00 | 5.20 ± 3.43 | 32.52 ± 4.56 | 27.10 ± 3.40 | 92.02 ± 2.94 | 22.43 ± 10.70 | | GPT-4-turbo | 100.00 | 12.00 ± 0.00 | 108.80 ± 7.89 | 90.67 ± 5.88 | 98.05 ± 1.01 | 0.51 ± 0.73 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 1 Introduction · p. 2
“Ablation studies show that communication reduces resource overuse by 21%.”
§ 3.5 Ablation of Communication · p. 7
“Comparing simulations without communication with those with communication, we find that agents without communication tend to overuse the common resource by 22% (t-test; p < 0 . 001 ).”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| Newcomer | 0.00 | 3.40 ± 0.80 | 12.00 ± 10.95 | 16.67 ± 0.00 | 42.67 ± 2.31 | 15.60 ± 11.78 |”
Reported 16.67 · recomputed 10
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ F.2.1 Fishery · p. 36
“| GPT-4 | 1.00 ± 0.00 | 0.66 ± 0.08 | 0.93 ± 0.04 | 0.96 ± 0.03 |”
§ F.2.1 Fishery · p. 36
“| GPT-4 | 1.00 ± 0.00 | 0.16 ± 0.06 | 0.99 ± 0.01 | 0.98 ± 0.02 |”
1 more finding
Create a free account to see every finding of this check.
Citations
Cites
Axelrod, R., & Hamilton, W. D. (1981) · Science · doi:10.1126/science.7466396 · Cited by 20,065
Governing the Commons: The Evolution of Institutions for Collective Action
Brady, G., & Ostrom, E. (1993) · Southern Economic Journal · doi:10.2307/1059950 · Cited by 16,839 · cited 5×
“The first is inspired by empirical work on understanding the norms that emerge in communities of fishermen that prevent overfishing [25, 43, 56].”
InfluentialLLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · cited 2×
“Recent advances in large language models (LLMs) have demonstrated impressive abilities across many tasks [1, 7, 8, 68], and LLMs are being integrated into complex agents [12, 21].”
The Economic Theory of a Common-Property Resource: The Fishery
Gordon, H. S. (1954) · Journal of Political Economy · doi:10.1086/257497 · Cited by 3,971 · cited 2×
“Within G OV S IM , we develop three common pool resource dilemmas inspired by the economic analysis of emergent sustainable cooperation [25–27, 43, 56].”
InfluentialConnolly, T., & Axelrod, R. (1984) · Administrative Science Quarterly · doi:10.2307/2392957 · Cited by 3,890 · cited 1×
“Inspired by game-theoretic research on the evolution of cooperation [5] and “The Tragedy of the Commons,” we build G OV S IM to simulate realistic multi-party social dilemmas such as those faced by groups managing shared resources [27, 62].”
MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
Hong, S., Zhuge, M., Chen, J., Zheng, X., Cheng, Y., Zhang, C., Wang, J., Wang, Z., Yau, S. K. S., Lin, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.00352 · Cited by 2,825 · cited 1×
“For these complex tasks, recent work has started deploying generative agents [59, 60] for task-specific simulations, such as collaborative agent systems for software engineering [32, 46, 53, 72] and other domains [33, 36, 49, 69].”
Revisiting the commons: local lessons, global challenges.
Ostrom, E., Burger, J., Field, C. B., Norgaard, R. B., & Policansky, D. (1999) · Science · doi:10.1126/science.284.5412.278 · Cited by 2,798 · cited 2×
“…this prior work leaves three key questions open: (1) in contrast to the well-documented mechanisms that enable cooperation in people [20, 56, 57], there is limited understanding of how LLMs achieve and maintain cooperation; (2) how to handle multi-turn LLM interactions that balance safety…”
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187 · cited 4×
“For these complex tasks, recent work has started deploying generative agents [59, 60] for task-specific simulations, such as collaborative agent systems for software engineering [32, 46, 53, 72] and other domains [33, 36, 49, 69].”
InfluentialJudging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888 · cited 1×
“Static benchmarks with clear ground-truth MMLU [29], GSM8k [11], and others like it do not capture flexible and interactive tasks needed to navigate scenarios in the real-world [22, 47, 73].”
Multiagent systems: algorithmic, game-theoretic, and logical foundations
(2009) · Choice Reviews Online · doi:10.5860/choice.46-5662 · Cited by 1,756 · cited 1×
“Although mechanism designers have developed incentive-compatible systems that can lead to cooperation between self-interested agents, these systems often assume a top-down process that coordinates the process [64, 74].”
Research works printed but not linked: 7
- [9][9] C. F. Camerer. Behavioral game theory: Experiments in strategic interaction . Princeton university press, 2011. -
- [26][26] J. Greene. Moral tribes: Emotion, reason, and the gap between us and them . Penguin, 2014. -
- [27][27] G. Hardin. The tragedy of the commons. Science , 162(3859):1243-1248, 1968. -
- [39][39] M. Kleiman-Weiner, M. K. Ho, J. L. Austerweil, M. L. Littman, and J. B. Tenenbaum. Coordinate to cooperate or compete: abstract goals and joint intentions in social interaction. In Proceedings of the 38th Annual Conference of the Cognitive Science Society , 2016. -
- [41][41] M. Kleiman-Weiner, A. Shaw, and J. B. Tenenbaum. Constructing social preferences from anticipated judgments: When impartial inequity is fair and why? In Proceedings of the 39th Annual Conference of the Cognitive Science Society , 2017. -
- [47][47] T. Liao, R. Taori, I. D. Raji, and L. Schmidt. Are we learning yet? a meta review of evaluation failures across machine learning. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) , 2021. -
- [48][48] J. Light, M. Cai, S. Shen, and Z. Hu. Avalonbench: Evaluating llms playing the game of avalon. In NeurIPS 2023 Foundation Models for Decision Making Workshop , 2023. -
Cited by
Understanding World or Predicting Future? A Comprehensive Survey of World Models
Ding, J., Zhang, Y., Shang, Y., Zhang, Y., Zong, Z., Feng, J., Yuan, Y., Su, H., Li, N., Sukiennik, N., et al. (2026) · ACM Computing Surveys · doi:10.1145/3746449 · Cited by 169
Multi-Agent Risks from Advanced AI
Hammond, L., Chan, A., Clifton, J., Hoelscher-Obermaier, J., Khan, A., McLean, E., Smith, C., Barfuss, W., Foerster, J., Gavenčiak, T., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.14143 · Cited by 305
Shi, T., Mo, Y., Liu, Y., Hao, Z., Wang, Y., Hu, W., Yu, N., Zhou, M., & Yu, J. (2026) · arXiv · doi:10.48550/arxiv.2606.04306 · Cited by 7
Large Language Model Reasoning Failures
Song, P., Han, P., & Goodman, N. (2026) · arXiv · doi:10.48550/arxiv.2602.06176 · Cited by 3
Does Socialization Emerge in AI Agent Society? A Case Study of Moltbook
Li, M., Li, X., & Zhou, T. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813123 · Cited by 30
“Recent work has progressively scaled LLM systems from individual autonomous agents [49] to networked societies [40, 41].”
Metadata
- Authors
- Piatti, G., Jin, Z., Kleiman-Weiner, M., Schölkopf, B., Sachan, M., & Mihalcea, R.
- Venue
- Advances in Neural Information Processing Systems 37
- Published
- 2024
- Pages
- 111715-111759
- ISSN
- 1049-5258 (ISSN-L)
- OpenAlex
- W4415795596
- Cited by
- 14
- References
- 78
- Citations by year
- 2026: 12 · 2025: 2
- FWCI
- 18.37
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by-sa
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- National Science Foundation
- Open Philanthropy Project
- Future of Life Institute
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (201009)
- Bundesministerium für Bildung und Forschung (01IS18039B, FKZ: 01IS18039B)
- Topics
- Social Sciences › Sociology and Political Science › Evolutionary Game Theory and Cooperation
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Social Sciences › Decision Sciences › Management Science and Operations Research › Game Theory and Applications
- Keywords
- LLM agents, multi-agent systems, sustainable cooperation, agent cooperation, emergent behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-11-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents. https://app.cochat.ai/papers/10.52202/079017-3548
- The paper
- Piatti, G., Jin, Z., Kleiman-Weiner, M., Schölkopf, B., Sachan, M., & Mihalcea, R. (2024). Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents. Advances in Neural Information Processing Systems 37. https://doi.org/10.52202/079017-3548
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.