Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents
Open accessThe DOI matches its Crossref record; 74 of the 78 references it cites were found in a registry or an index; 14 of the 33 sources its claims cite were checked for support.
Abstract
As AI systems pervade human life, ensuring that large language models (LLMs) make safe decisions remains a significant challenge. We introduce the Governance of the Commons Simulation (GovSim), a generative simulation platform designed to study strategic interactions and cooperative decision-making in LLMs. In GovSim, a society of AI agents must collectively balance exploiting a common resource with sustaining it for future use. This environment enables the study of how ethical considerations, strategic planning, and negotiation skills impact cooperative outcomes. We develop an LLM-based agent architecture and test it with the leading open and closed LLMs. We find that all but the most powerful LLM agents fail to achieve a sustainable equilibrium in GovSim, with the highest survival rate below 54%. Ablations reveal that successful multi-agent communication between agents is critical for achieving cooperation in these cases. Furthermore, our analyses show that the failure to achieve sustainable cooperation in most LLMs stems from their inability to formulate and analyze hypotheses about the long-term effects of their actions on the equilibrium of the group. Finally, we show that agents that leverage"Universalization"-based reasoning, a theory of moral thinking, are able to achieve significantly better sustainability. Taken together, GovSim enables us to study the mechanisms that underlie sustainable self-government with specificity and scale. We open source the full suite of o…
Overview
CoChat Summary
TL;DR
GOVSIM is a multi-agent common-pool resource simulation where LLM agents harvest a shared resource and then discuss; most LLMs fail to sustain cooperation, with the best survival rate below 54%. Communication and universalization-based reasoning substantially improve sustainability, while a greedy newcomer can destabilize cooperation.
Key findings
- In GOVSIM, all but the most powerful LLM agents fail to reach a sustainable equilibrium, with the highest survival rate below 54% (GPT-4o: 53.3 in Table 1); smaller models often never sustain resources past the first month.
- Ablating communication increases resource over-usage by 22% (Section 3.5; the abstract reports 21%), and dialogue analysis finds negotiation is the main communication type (abstract: 62%; Section 3.6: 54% negotiation, 45% information, 1% relational).
- Universalization prompting significantly improves outcomes: average survival time +4 months, total gain +29 units, and efficiency +24% (t-test; p < 0.001).
- A greedy newcomer perturbation with GPT-4o drops survival rate from 53.3 to 33.3, survival time from 9.3 to 6.6, gain from 66.0 to 34.8, efficiency from 55.0 to 31.3, equality from 94.4 to 71.7, and increases over-usage from 10.8 to 15.7; belief formation correlates 0.83 with community survival time.
Methods
- GOVSIM is a partially observable Markov game with three common-pool resource scenarios—fishery, pasture, and pollution—where agents harvest simultaneously, then communicate; the resource doubles at month end up to capacity 100, collapses below C = 5, and runs for T = 12 time steps.
- The authors use an LLM-based generative agent architecture and benchmark 15 instruction-tuned open- and closed-weight LLMs, running each simulation with five random seeds at temperature zero; metrics include survival time, survival rate, total gain, efficiency, equality, and over-usage.
- Analyses include communication ablation, universalization prompting, a greedy-newcomer perturbation, GPT-4-turbo dialogue classification, and four sub-skill tests of 150 procedurally generated problems each, correlated with survival time via OLS.
Limitations
- The paper states that the resource-sharing scenarios are simplified relative to real-world common-pool resource management; current LLM capabilities limit negotiation and strategic behavior; more agents increase runtime; and future work could add shocks, exceptions, adversarial agents, and human participants.
- Evident from the text: each simulation uses only five seeds, and although temperature is set to zero, the authors acknowledge nondeterminism from LLM inference kernels and external APIs; no human baseline is included in the main experiments.
- Evident from the text: dialogue classification was validated on 100 manually annotated utterances with 72% agreement on sub-categories, so the dialogue-type proportions carry annotation noise.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
9 Valid · 5 Not valid · 5 Not checked · 94 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“we find that prompting agents to consider the universalization of their action [43], a process used by people when making moral judgments in social dilemmas, significantly improves survival time.”
What the sources say
- Supportedcites [43] [43] S. Levine, M. Kleiman-Weiner, L. Schulz, J. Tenenbaum, and F. Cushman. The logic of universalization guides moral judgment. Proceedings of the National Academy of Sciences , 117 (42):26158-26169, 2020. - source
The Universalization section defines the cited mechanism as judging an action by considering what would happen if everyone felt free to do the same. The Discussion reports that participants’ judgments matched its predictions in threshold dilemmas. The survival-time improvement is presented as the citing authors’ own finding, not as a result attributed to this source. (LLM, conf=0.91)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These methods do not address the complexities inherent in multi-agent interactions and broader real-world scenarios”
What the sources say
- Partly supportedcites [13] [13] V. Conitzer and C. Oesterheld. Foundations of cooperative ai. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 15359-15367, 2023. - sourceExperimental — accuracy not yet measured
The Introduction says that aligning individual agents is not sufficient to ensure that a multi-agent system is safe, and gives real-world examples of harmful interactions. It does not assess safety evaluation datasets or studies, so it does not establish that those particular methods fail to address these complexities. The indexed copy provides no page numbers for these passages. (LLM, conf=0.83)
- Partly supportedcites [14] [14] A. Critch and D. Krueger. Ai research considerations for human existential safety (arches). arXiv preprint arXiv:2006.04948 , 2020. - sourceExperimental — accuracy not yet measured
Section 1.2 (p. 8) warns that safety techniques effective for present-day AI may fail to generalize to more powerful systems. Section 6.3 (p. 79) identifies multi-agent control challenges absent from the single-system setting. These passages support the broader concern, but our copy does not establish that safety evaluation datasets and studies, specifically, fail to address multi-agent interactions and broader real-world scenarios. (LLM, conf=0.78)
- Partly supportedcites [15] [15] A. Dafoe, E. Hughes, Y. Bachrach, T. Collins, K. R. McKee, J. Z. Leibo, K. Larson, and T. Graepel. Open problems in cooperative ai. arXiv preprint arXiv:2012.08630 , 2020. - sourceExperimental — accuracy not yet measured
Section 2.2 (p. 8) says two-player zero-sum games provide no opportunity to learn cooperation, and Section 3.1 (p. 10) says research has focused heavily on those settings despite their relative rarity in the real world. This supports the concern that some methods miss important multi-agent and real-world dynamics, but the paper does not establish that safety evaluation datasets and studies, specifically, fail to address them. (LLM, conf=0.78)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Prior multi-agent research has focused on highly constrained scenarios such as board games or narrowly defined collaborative tasks [19, 45, 48, 64, 72].”
What the sources say
- Partly supportedcites [19] [19] J. Duan, R. Zhang, J. Diffenderfer, B. Kailkhura, L. Sun, E. Stengel-Eskin, M. Bansal, T. Chen, and K. Xu. Gtbench: Uncovering the strategic reasoning limitations of llms via game-theoretic evaluations. arXiv preprint arXiv:2402.12348 , 2024. - sourceExperimental — accuracy not yet measured
The “Multiple LLMs-as-Agents in Gaming” section says many prior studies evaluated agents in one or two games or measured a narrow set of abilities, including coordination and cooperation. It does not characterize prior multi-agent research generally as focused on board games; the board-game examples in the supplied passages describe GTBench’s own evaluations. (LLM, conf=0.82)
- Not matchedcites [45] [45] G. Li, H. A. A. K. Hammoud, H. Itani, D. Khizbullin, and B. Ghanem. Camel: Communicative agents for" mind" exploration of large scale language model society. 2023. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked yetcites [48] [48] J. Light, M. Cai, S. Shen, and Z. Hu. Avalonbench: Evaluating llms playing the game of avalon. In NeurIPS 2023 Foundation Models for Decision Making Workshop , 2023. - source
- Supportedcites [64] [64] J. Serrino, M. Kleiman-Weiner, D. C. Parkes, and J. Tenenbaum. Finding friend and foe in multi-agent games. Advances in Neural Information Processing Systems , 32, 2019. - source
The Introduction (p. 1) describes prior work on poker and on team coordination in Dota and capture the flag, noting that none involved ambiguity about whom to cooperate with. This supports the citing sentence’s broad characterization of prior research as focused on game settings or defined team tasks. (LLM, conf=0.84)
- Partly supportedcites [72] [72] Y. Xu, S. Wang, P. Li, F. Luo, X. Wang, W. Liu, and Y. Liu. Exploring large language models for communication games: An empirical study on werewolf. arXiv preprint arXiv:2309.04658 , 2023. - sourceExperimental — accuracy not yet measured
The Related Work passage (indexed under §5.4 Leadership; page unavailable) describes prior game-playing AI in Go, chess, and poker, as well as Werewolf agents constrained by rules or talking templates. This supports the board-game and constrained-scenario portion, but the paper does not characterize prior multi-agent research as focused on narrowly defined collaborative tasks. (LLM, conf=0.83)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Static benchmarks with clear ground-truth MMLU [29], GSM8k [11], and others like it do not capture flexible and interactive tasks needed to navigate scenarios in the real-world”
What the sources say
- Partly supportedcites [22] [22] S. Gehrmann, E. Clark, and T. Sellam. Repairing the cracked foundation: A survey of obstacles in evaluation practices for generated text. Journal of Artificial Intelligence Research , 77: 103-166, 2023. - sourceExperimental — accuracy not yet measured
In the “Model Interpretability” section (page unavailable), the paper says that evaluating interactive systems with static data can mischaracterize their capabilities and limitations. Section 3.3 also cautions that benchmark results are specific to the scenarios evaluated. The copy does not establish this specifically for MMLU or GSM8k, or discuss their clear ground-truth answers. (LLM, conf=0.84)
- Not matchedcites [47] [47] T. Liao, R. Taori, I. D. Raji, and L. Schmidt. Are we learning yet? a meta review of evaluation failures across machine learning. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) , 2021. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [74] [74] L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems , 36, 2024. - source
The Motivation section (p. 3) identifies MMLU and GSM-8K as benchmarks requiring short, automatically validated answers, and says existing benchmarks are inadequate for assessing open-ended, multi-turn interactions. The Introduction (p. 2) describes Chatbot Arena as evaluating conversations in real-world scenarios. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On average (overall models), the majority of utterances (54%) are focused on negotiations between agents, followed by information (45%) and relational (1%).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
53 more claims
Create a free account to see all 113 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 19
- Probably not a substantial claim. 45
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In all cases, the utility function is clearly defined, such as 'units harvested' or 'tokens contributed to the public good', and directly linked to actions.”
From The Role of Social Learning and Collective Norm Formation in Fostering Cooperation in LLM Multi-Agent Systems (2026) · cites this paper as [26]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ 1 Introduction · p. 2
“negotiation is the main type of communication between agents and constitutes 62% of the dialogues.”
§ 3.6 Analysis of Agent Dialogues · p. 8
“On average (overall models), the majority of utterances (54%) are focused on negotiations between agents, followed by information (45%) and relational (1%).”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ D.5.1 Fishery · p. 30
“| GPT-4 | 12.00 ± 0.00 | 0.51 ± 0.73 | 12.00 ± 0.00 | 11.33 ± 11.42 |”
§ D.2.1 Fishery · p. 23
“| GPT-4 | 20.00 | 5.20 ± 3.43 | 32.52 ± 4.56 | 27.10 ± 3.40 | 92.02 ± 2.94 | 22.43 ± 10.70 | | GPT-4-turbo | 100.00 | 12.00 ± 0.00 | 108.80 ± 7.89 | 90.67 ± 5.88 | 98.05 ± 1.01 | 0.51 ± 0.73 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 1 Introduction · p. 2
“Ablation studies show that communication reduces resource overuse by 21%.”
§ 3.5 Ablation of Communication · p. 7
“Comparing simulations without communication with those with communication, we find that agents without communication tend to overuse the common resource by 22% (t-test; p < 0 . 001 ).”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| Newcomer | 0.00 | 3.40 ± 0.80 | 12.00 ± 10.95 | 16.67 ± 0.00 | 42.67 ± 2.31 | 15.60 ± 11.78 |”
Reported 16.67 · recomputed 10
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ F.2.1 Fishery · p. 36
“| GPT-4 | 1.00 ± 0.00 | 0.66 ± 0.08 | 0.93 ± 0.04 | 0.96 ± 0.03 |”
§ F.2.1 Fishery · p. 36
“| GPT-4 | 1.00 ± 0.00 | 0.16 ± 0.06 | 0.99 ± 0.01 | 0.98 ± 0.02 |”
1 more finding
Create a free account to see every finding of this check.
Citations
Cites
Axelrod, R., & Hamilton, W. D. (1981) · Science · doi:10.1126/science.7466396 · Cited by 20,065
Governing the Commons: The Evolution of Institutions for Collective Action
Brady, G., & Ostrom, E. (1993) · Southern Economic Journal · doi:10.2307/1059950 · Cited by 17,152 · cited 5×
“The first is inspired by empirical work on understanding the norms that emerge in communities of fishermen that prevent overfishing [25, 43, 56].”
InfluentialLLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · cited 2×
“Recent advances in large language models (LLMs) have demonstrated impressive abilities across many tasks [1, 7, 8, 68], and LLMs are being integrated into complex agents [12, 21].”
The Economic Theory of a Common-Property Resource: The Fishery
Gordon, H. S. (1954) · Journal of Political Economy · doi:10.1086/257497 · Cited by 3,971 · cited 2×
“Within G OV S IM , we develop three common pool resource dilemmas inspired by the economic analysis of emergent sustainable cooperation [25–27, 43, 56].”
InfluentialConnolly, T., & Axelrod, R. (1984) · Administrative Science Quarterly · doi:10.2307/2392957 · Cited by 3,890 · cited 1×
“Inspired by game-theoretic research on the evolution of cooperation [5] and “The Tragedy of the Commons,” we build G OV S IM to simulate realistic multi-party social dilemmas such as those faced by groups managing shared resources [27, 62].”
MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
Hong, S., Zhuge, M., Chen, J., Zheng, X., Cheng, Y., Zhang, C., Wang, J., Wang, Z., Yau, S. K. S., Lin, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.00352 · Cited by 2,825 · cited 1×
“For these complex tasks, recent work has started deploying generative agents [59, 60] for task-specific simulations, such as collaborative agent systems for software engineering [32, 46, 53, 72] and other domains [33, 36, 49, 69].”
Revisiting the commons: local lessons, global challenges.
Ostrom, E., Burger, J., Field, C. B., Norgaard, R. B., & Policansky, D. (1999) · Science · doi:10.1126/science.284.5412.278 · Cited by 2,798 · cited 2×
“…this prior work leaves three key questions open: (1) in contrast to the well-documented mechanisms that enable cooperation in people [20, 56, 57], there is limited understanding of how LLMs achieve and maintain cooperation; (2) how to handle multi-turn LLM interactions that balance safety…”
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187 · cited 4×
“For these complex tasks, recent work has started deploying generative agents [59, 60] for task-specific simulations, such as collaborative agent systems for software engineering [32, 46, 53, 72] and other domains [33, 36, 49, 69].”
InfluentialJudging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888 · cited 1×
“Static benchmarks with clear ground-truth MMLU [29], GSM8k [11], and others like it do not capture flexible and interactive tasks needed to navigate scenarios in the real-world [22, 47, 73].”
Multiagent systems: algorithmic, game-theoretic, and logical foundations
(2009) · Choice Reviews Online · doi:10.5860/choice.46-5662 · Cited by 1,756 · cited 1×
“Although mechanism designers have developed incentive-compatible systems that can lead to cooperation between self-interested agents, these systems often assume a top-down process that coordinates the process [64, 74].”
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526 · cited 1×
“Recent advances in large language models (LLMs) have demonstrated impressive abilities across many tasks [1, 7, 8, 68], and LLMs are being integrated into complex agents [12, 21].”
Concrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314 · cited 1×
“As LLMs become more capable and autonomous, ensuring their safety remains a critical concern [2, 3, 30].”
Rand, D. G., & Nowak, M. A. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.06.003 · Cited by 1,308
ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks
Gilardi, F., Alizadeh, M., & Kubli, M. (2023) · arXiv · doi:10.48550/arxiv.2303.15056 · Cited by 1,207
Order without Law: How Neighbors Settle Disputes
Ellickson, R. C. (2009) · Harvard University Press eBooks · doi:10.2307/j.ctvk12rdz · Cited by 1,205 · cited 2×
“However, this prior work leaves three key questions open: (1) in contrast to the well-documented mechanisms that enable cooperation in people [20, 56, 57], there is limited understanding of how LLMs achieve and maintain cooperation; (2) how to handle multi-turn LLM interactions that balance safety…”
ChatGPT outperforms crowd workers for text-annotation tasks
Gilardi, F., Alizadeh, M., & Kubli, M. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2305016120 · Cited by 1,197 · cited 1×
“To support interpretability, we categorize conversations into three high-level clusters: information sharing, negotiation, and relational interactions using the following taxonomy: Following Gilardi et al. [23], we used GPT-4-turbo to classify each utterance according to our defined taxonomy.”
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972 · cited 1×
“A society of LLM agents with the ability to communicate finds ways to flexibly cooperate and avoid collapse.”
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970 · cited 1×
“Refer to Xi et al. [70] for an extensive review.”
Origins of Human Cooperation and Morality
Tomasello, M., & Vaish, A. (2013) · Annual Review of Psychology · doi:10.1146/annurev-psych-113011-143812 · Cited by 916 · cited 1×
“Sustaining cooperation is an essential problem that enables individuals to achieve better outcomes than they could achieve on their own [62, 65, 67].”
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.229 · Cited by 904 · cited 1×
“In our research, we find that all but the most powerful LLM agents fail to achieve a sustainable equilibrium, with the highest survival rate below 54%.”
Training Verifiers to Solve Math Word Problems
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.14168 · Cited by 575 · cited 1×
“Static benchmarks with clear ground-truth MMLU [29], GSM8k [11], and others like it do not capture flexible and interactive tasks needed to navigate scenarios in the real-world [22, 47, 73].”
Aligning AI With Shared Human Values
Hendrycks, D., Burns, C., Basart, S., Critch, A., Li, J., Song, D., & Steinhardt, J. (2020) · arXiv · doi:10.48550/arxiv.2008.02275 · Cited by 517 · cited 1×
“Popular evaluation datasets for safety include E THIS [28], T RUTHFUL QA [50], and M ORAL E XCEPT QA [35].”
Managing extreme AI risks amid rapid progress
Bengio, Y., Hinton, G., Yao, A., Song, D., Abbeel, P., Darrell, T., Harari, Y. N., Zhang, Y.-Q., Xue, L., Shalev-Shwartz, S., et al. (2024) · Science · doi:10.1126/science.adn0117 · Cited by 349
Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. D. L., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.06825 · Cited by 340 · cited 1×
“We also tested open-weights models: Llama-2 (7B, 13B, 70B) [68], Llama-3 (8B, 70B) [51], Mistral (7B, 8x7B) [34], Qwen (72B, 110B) [6].”
Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.16609 · Cited by 332 · cited 1×
“We also tested open-weights models: Llama-2 (7B, 13B, 70B) [68], Llama-3 (8B, 70B) [51], Mistral (7B, 8x7B) [34], Qwen (72B, 110B) [6].”
CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society
Li, G., Hammoud, H. A. A. K., Itani, H., Khizbullin, D., & Ghanem, B. (2023) · arXiv · doi:10.48550/arxiv.2303.17760 · Cited by 315 · cited 1×
“For these complex tasks, recent work has started deploying generative agents [59, 60] for task-specific simulations, such as collaborative agent systems for software engineering [32, 46, 53, 72] and other domains [33, 36, 49, 69].”
Camel: Communicative agents for" mind" exploration of large language model society
Li, G., Hammoud, H., Itani, H., Khizbullin, D., & Ghanem, B. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2264 · Cited by 314 · cited 1×
“For these complex tasks, recent work has started deploying generative agents [59, 60] for task-specific simulations, such as collaborative agent systems for software engineering [32, 46, 53, 72] and other domains [33, 36, 49, 69].”
Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology ViewPreprint
Zhang, J., Xu, X., Liu, R., & Deng, S.-M. (2023) · arXiv.org · doi:10.48550/arxiv.2310.02124 · Cited by 300 · cited 1×
“For these complex tasks, recent work has started deploying generative agents [59, 60] for task-specific simulations, such as collaborative agent systems for software engineering [32, 46, 53, 72] and other domains [33, 36, 49, 69].”
Large language models empowered agent-based modeling and simulation: a survey and perspectives
Gao, C., Lan, X., Li, N., Yuan, Y., Ding, J., Zhou, Z., Xu, F., & Li, Y. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03611-3 · Cited by 294 · cited 1×
“Recent advances in large language models (LLMs) have demonstrated impressive abilities across many tasks [1, 7, 8, 68], and LLMs are being integrated into complex agents [12, 21].”
Measuring Massive Multitask Language Understanding
Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., & Steinhardt, J. (2020) · arXiv · doi:10.48550/arxiv.2009.03300 · Cited by 291 · cited 1×
“Static benchmarks with clear ground-truth MMLU [29], GSM8k [11], and others like it do not capture flexible and interactive tasks needed to navigate scenarios in the real-world [22, 47, 73].”
Cooperative AI: machines must learn to find common ground
Dafoe, A., Bachrach, Y., Hadfield, G., Horvitz, E., Larson, K., & Graepel, T. (2021) · Nature · doi:10.1038/d41586-021-01170-0 · Cited by 232 · cited 1×
“Thus, we need robust and safe AI that cooperates with us as well as (or better than) we can cooperate with each other [16].”
Life 3.0: being human in the age of artificial intelligence
Lauscher, A. (2019) · Internet Histories · doi:10.1080/24701475.2019.1565556 · Cited by 217 · cited 1×
“4 Contributions in the Context of Related Work AI Safety The primary objective of AI safety is to ensure that AI systems do not cause harm to humans [30, 54, 66].”
Social Simulacra: Creating Populated Prototypes for Social Computing Systems
Park, J. S., Popowski, L., Cai, C., Morris, M. R., Liang, P., & Bernstein, M. S. (2022) · Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3526113.3545616 · Cited by 212 · cited 1×
“For these complex tasks, recent work has started deploying generative agents [59, 60] for task-specific simulations, such as collaborative agent systems for software engineering [32, 46, 53, 72] and other domains [33, 36, 49, 69].”
When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment
Jin, Z., Levine, S., Adauto, F. G., Kamal, O., Sap, M., Sachan, M., Mihalcea, R. F., Tenenbaum, J. B., & Schölkopf, B. (2022) · neural information processing systems · doi:10.52202/068431-2063 · Cited by 177
Self-citationWhen to Make Exceptions: Exploring Language Models as Accounts of Human Moral JudgmentPreprint
Jin, Z., Levine, S., Gonzalez, F., Kamal, O., Sap, M., Sachan, M., Mihalcea, R., Tenenbaum, J. B., & Schölkopf, B. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2210.01478 · Cited by 147 · cited 1×
“Popular evaluation datasets for safety include E THIS [28], T RUTHFUL QA [50], and M ORAL E XCEPT QA [35].”
Self-citationMind2Web: Towards a Generalist Agent for the Web
Deng, X., Gu, Y., Zheng, B., Chen, S., Stevens, S., Wang, B., Sun, H., & Su, Y. (2023) · arXiv · doi:10.48550/arxiv.2306.06070 · Cited by 134 · cited 1×
“In contrast, more recent efforts evaluate LLMs on complex tasks that resemble real-world applications [18, 38, 75] or involve A/B testing with human feedback [10].”
(2023) · doi:10.52202/075280-1220 · Cited by 134
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text
Gehrmann, S., Clark, E., & Sellam, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13715 · Cited by 123 · cited 1×
“Static benchmarks with clear ground-truth MMLU [29], GSM8k [11], and others like it do not capture flexible and interactive tasks needed to navigate scenarios in the real-world [22, 47, 73].”
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
Chiang, W.-L., Zheng, L., Sheng, Y., Angelopoulos, A. N., Li, T., Li, D., Zhang, H., Zhu, B., Jordan, M., Gonzalez, J. E., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.04132 · Cited by 118 · cited 1×
“In contrast, more recent efforts evaluate LLMs on complex tasks that resemble real-world applications [18, 38, 75] or involve A/B testing with human feedback [10].”
A multi-agent reinforcement learning model of common-pool resource appropriation
Perolat, J., Leibo, J. Z., Zambaldi, V., Beattie, C., Tuyls, K., & Graepel, T. (2017) · arXiv · doi:10.48550/arxiv.1707.06600 · Cited by 109 · cited 1×
“We follow Perolat et al. [61] in defining a suite of metrics since in a mixed incentive repeated game like G OV S IM , no single scalar metric can track the entire state of the system.”
The AI economist: taxation policy design via two-level deep multiagent reinforcement learning
Zheng, S., Trott, A., Srinivasa, S., Parkes, D. C., & Socher, R. (2022) · Science Advances · doi:10.1126/sciadv.abk2607 · Cited by 105 · cited 1×
“Although mechanism designers have developed incentive-compatible systems that can lead to cooperation between self-interested agents, these systems often assume a top-down process that coordinates the process [64, 74].”
Unsolved Problems in ML Safety
Hendrycks, D., Carlini, N., Schulman, J., & Steinhardt, J. (2021) · arXiv · doi:10.48550/arxiv.2109.13916 · Cited by 95 · cited 2×
“As LLMs become more capable and autonomous, ensuring their safety remains a critical concern [2, 3, 30].”
Open Problems in Cooperative AI
Dafoe, A., Hughes, E., Bachrach, Y., Collins, T., McKee, K. R., Leibo, J. Z., Larson, K., & Graepel, T. (2020) · arXiv · doi:10.48550/arxiv.2012.08630 · Cited by 93 · cited 1×
“These methods do not address the complexities inherent in multi-agent interactions and broader real-world scenarios, and more effort is needed to guarantee the safety of multi-agent systems [13–15].”
Managing AI Risks in an Era of Rapid Progress
Bengio, Y., Hinton, G., Yao, A., Song, D., Abbeel, P., Harari, Y., Zhang, Y.-Q., Xue, L., Shalev-Shwartz, S., Hadfield, G. K., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2310.17688 · Cited by 89 · cited 1×
“Recent advances in large language models (LLMs) have demonstrated impressive abilities across many tasks [1, 7, 8, 68], and LLMs are being integrated into complex agents [12, 21].”
The logic of universalization guides moral judgment
Levine, S., Kleiman-Weiner, M., Schulz, L., Tenenbaum, J., & Cushman, F. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2014505117 · Cited by 88 · cited 5×
“The first is inspired by empirical work on understanding the norms that emerge in communities of fishermen that prevent overfishing [25, 43, 56].”
InfluentialSelf-citationWhat Would Jiminy Cricket Do? Towards Agents That Behave MorallyPreprint
Hendrycks, D., Mazeika, M., Zou, A., Patel, S., Zhu, C., Navarro, J., Song, D. X., Li, B., & Steinhardt, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.13136 · Cited by 78 · cited 1×
“Additional studies have explored the capabilities and potential issues of current LLMs [17, 31, 52].”
DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving AgentsPreprint
Nair, V., Schumacher, E., Tso, G., et al. (2023) · Clinical Natural Language Processing Workshop · doi:10.48550/arxiv.2303.17071 · Cited by 75 · cited 1×
“For these complex tasks, recent work has started deploying generative agents [59, 60] for task-specific simulations, such as collaborative agent systems for software engineering [32, 46, 53, 72] and other domains [33, 36, 49, 69].”
Evaluating Language Model Agency through Negotiations
Davidson, T. R., Veselovsky, V., Josifoski, M., Peyrard, M., Bosselut, A., Kosiński, M., & West, R. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2401.04536 · Cited by 68 · cited 1×
“Additional studies have explored the capabilities and potential issues of current LLMs [17, 31, 52].”
Learning a commonsense moral theory
Kleiman-Weiner, M., Saxe, R., & Tenenbaum, J. B. (2017) · Cognition · doi:10.1016/j.cognition.2017.03.005 · Cited by 53 · cited 1×
“When cooperation between agents (and humans) is possible, better outcomes for all through joint effort are possible [27, 39, 40, 62].”
Self-citationKant’s Groundwork of the Metaphysics of Morals
Mudd, S. (2014) · doi:10.1080/09672559.2014.896623 · Cited by 52 · cited 7×
“Comparing simulations without communication with those with communication, we find that agents without communication tend to overuse the common resource by 22% (t-test; p < 0 .”
InfluentialDERA: Enhancing large language model completions with dialog-enabled resolving agents
Nair, V., Schumacher, E., Tso, G., & Kannan, A. (2024) · Proceedings of the 6th Clinical Natural Language Processing Workshop · doi:10.18653/v1/2024.clinicalnlp-1.12 · Cited by 38
Awad, E., Levine, S., Loreggia, A., Mattei, N., Rahwan, I., Rossi, F., Talamadupula, K., Tenenbaum, J. B., & Max Kleiman-weiner, A. (2024) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-024-09667-4 · Cited by 36 · cited 1×
“…may need to handle a one-off choice of serious personal harm and group sustainability, e.g., one agent will experience harm unless they take more resources than permitted by an existing norm —-will other agents adapt and allow for such one-off exceptions without allowing for exploitation [4, 44]?”
Self-citationBrejzek, T., & Wallen, L. (2018) · doi:10.1080/23322551.2018.1469842 · Cited by 34 · cited 1×
“Popular evaluation datasets for safety include E THIS [28], T RUTHFUL QA [50], and M ORAL E XCEPT QA [35].”
Finding Friend and Foe in Multi-Agent Games
Serrino, J., Kleiman-Weiner, M., Parkes, D. C., & Tenenbaum, J. B. (2019) · arXiv · doi:10.48550/arxiv.1906.02330 · Cited by 33 · cited 1×
“We find that all but the most powerful LLM agents fail to achieve a sustainable equilibrium in G OV S IM , with the highest survival rate below 54%. constrained scenarios such as board games or narrowly defined collaborative tasks [19, 45, 48, 63, 71].”
Self-citationExploring Large Language Models for Communication Games: An Empirical Study on Werewolf
Xu, Y., Wang, S., Li, P., Luo, F., Wang, X., Liu, W., & Liu, Y. (2023) · arXiv · doi:10.48550/arxiv.2309.04658 · Cited by 32 · cited 2×
“We find that all but the most powerful LLM agents fail to achieve a sustainable equilibrium in G OV S IM , with the highest survival rate below 54%. constrained scenarios such as board games or narrowly defined collaborative tasks [19, 45, 48, 63, 71].”
Pan, A., Chan, J. S., Zou, A., Li, N., Basart, S., Woodside, T., Ng, J., Zhang, H., Emmons, S., & Hendrycks, D. (2023) · arXiv · doi:10.48550/arxiv.2304.03279 · Cited by 32 · cited 2×
“Most similar to G OV S IM is M ACHIAVELLI [58], where the authors investigate harmful behavior vs. reward maximization in a benchmark of single-agent choose-your-own-adventure games.”
WebArena: A Realistic Web Environment for Building Autonomous Agents
Zhou, S., Xu, F. F., Zhu, H., Zhou, X., Lo, R., Sridhar, A., Cheng, X., Ou, T., Bisk, Y., Fried, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.13854 · Cited by 31 · cited 1×
“In contrast, more recent efforts evaluate LLMs on complex tasks that resemble real-world applications [18, 38, 75] or involve A/B testing with human feedback [10].”
When rules are over-ruled: Virtual bargaining as a contractualist method of moral judgment
Levine, S., Max Kleiman-weiner, A., Chater, N., Cushman, F. A., & Tenenbaum, J. B. (2024) · Cognition · doi:10.1016/j.cognition.2024.105790 · Cited by 26 · cited 1×
“…may need to handle a one-off choice of serious personal harm and group sustainability, e.g., one agent will experience harm unless they take more resources than permitted by an existing norm —-will other agents adapt and allow for such one-off exceptions without allowing for exploitation [4, 44]?”
Self-citationAI Research Considerations for Human Existential Safety (ARCHES)
Critch, A., & Krueger, D. (2020) · arXiv · doi:10.48550/arxiv.2006.04948 · Cited by 25
Humanoid Agents: Platform for Simulating Human-like Generative Agents
Wang, Z., Chiu, Y. Y., & Chiu, Y. C. (2023) · arXiv · doi:10.48550/arxiv.2310.05418 · Cited by 21 · cited 1×
“For these complex tasks, recent work has started deploying generative agents [59, 60] for task-specific simulations, such as collaborative agent systems for software engineering [32, 46, 53, 72] and other domains [33, 36, 49, 69].”
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
Anwar, U., Saparov, A., Rando, J., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.09932 · Cited by 19 · cited 1×
“As LLMs become more capable and autonomous, ensuring their safety remains a critical concern [2, 3, 30].”
Conitzer, V., & Oesterheld, C. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.26791 · Cited by 19 · cited 1×
“These methods do not address the complexities inherent in multi-agent interactions and broader real-world scenarios, and more effort is needed to guarantee the safety of multi-agent systems [13–15].”
World Models for Math Story ProblemsPreprint
Opedal, A., Stoehr, N., Saparov, A., & Sachan, M. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2306.04347 · Cited by 19
Self-citationWar and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars
Hua, W., Fan, L., Li, L., Mei, K., Ji, J., Ge, Y., Hemphill, L., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2311.17227 · Cited by 15 · cited 1×
“For these complex tasks, recent work has started deploying generative agents [59, 60] for task-specific simulations, such as collaborative agent systems for software engineering [32, 46, 53, 72] and other domains [33, 36, 49, 69].”
Evaluating Language-Model Agents on Realistic Autonomous Tasks
Kinniment, M., Sato, L. J. K., Du, H., Goodrich, B., Hasin, M., Chan, L., Miles, L. H., Lin, T. R., Wijk, H., Burget, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.11671 · Cited by 15 · cited 2×
“These multi-agent studies complement existing single-agent AI safety benchmarks [38, 58].”
AgentSims: An Open-Source Sandbox for Large Language Model Evaluation
Lin, J., Zhao, H., Zhang, A., Wu, Y., Ping, H., & Chen, Q. (2023) · arXiv · doi:10.48550/arxiv.2308.04026 · Cited by 14 · cited 1×
“For these complex tasks, recent work has started deploying generative agents [59, 60] for task-specific simulations, such as collaborative agent systems for software engineering [32, 46, 53, 72] and other domains [33, 36, 49, 69].”
Theory of Minds: Understanding Behavior in Groups through Inverse Planning
Shum, M., Kleiman-Weiner, M., Littman, M. L., & Tenenbaum, J. B. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016163 · Cited by 14 · cited 1×
“Sustaining cooperation is an essential problem that enables individuals to achieve better outcomes than they could achieve on their own [62, 65, 67].”
Self-citationLarge Language Models Empowered Agent-based Modeling and Simulation: A Survey and PerspectivesPreprint
Gao, C., Lan, X., Li, N., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.11970 · Cited by 12
Assessing and dissociating virtues from the ‘bottom up’: A case study of generosity vs. fairness
Kraft-Todd, G. T., Kleiman-Weiner, M., & Young, L. (2023) · The Journal of Positive Psychology · doi:10.1080/17439760.2022.2154254 · Cited by 7 · cited 1×
“When multiple agents are involved, questions of fairness arise [41, 42].”
STEER: Assessing the Economic Rationality of Large Language Models
Raman, N., Lundy, T., Amouyal, S., Levine, Y., Leyton-Brown, K., & Tennenholtz, M. (2024) · arXiv · doi:10.48550/arxiv.2402.09552 · Cited by 5
Human Cooperation: Challenges for Science and Practice
Dreu, C. K. W. D. (2013) · Gothic.net · doi:10.1177/1529100613496959 · Cited by 4 · cited 3×
“Inspired by game-theoretic research on the evolution of cooperation [5] and “The Tragedy of the Commons,” we build G OV S IM to simulate realistic multi-party social dilemmas such as those faced by groups managing shared resources [27, 62].”
GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
Duan, J., Zhang, R., Diffenderfer, J., Kailkhura, B., Sun, L., Stengel-Eskin, E., Bansal, M., Chen, T., & Xu, K. (2024) · arXiv · doi:10.48550/arxiv.2402.12348 · Cited by 4 · cited 2×
“We find that all but the most powerful LLM agents fail to achieve a sustainable equilibrium in G OV S IM , with the highest survival rate below 54%. constrained scenarios such as board games or narrowly defined collaborative tasks [19, 45, 48, 63, 71].”
Lyfe Agents: Generative agents for low-cost real-time social interactions
Kaiya, Z., Naim, M., Kondic, J., Cortes, M., Ge, J., Luo, S., Yang, G. R., & Ahn, A. (2023) · arXiv · doi:10.48550/arxiv.2310.02172 · Cited by 2
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774 · cited 2×
“5, GPT-4, GPT-4-turbo, and GPT-4o [1] via OpenAI API, Claude-3 Haiku, Sonnet, and Opus via Anthropic API.”
Research works printed but not linked: 7
- [9][9] C. F. Camerer. Behavioral game theory: Experiments in strategic interaction . Princeton university press, 2011. -
- [26][26] J. Greene. Moral tribes: Emotion, reason, and the gap between us and them . Penguin, 2014. -
- [27][27] G. Hardin. The tragedy of the commons. Science , 162(3859):1243-1248, 1968. -
- [39][39] M. Kleiman-Weiner, M. K. Ho, J. L. Austerweil, M. L. Littman, and J. B. Tenenbaum. Coordinate to cooperate or compete: abstract goals and joint intentions in social interaction. In Proceedings of the 38th Annual Conference of the Cognitive Science Society , 2016. -
- [41][41] M. Kleiman-Weiner, A. Shaw, and J. B. Tenenbaum. Constructing social preferences from anticipated judgments: When impartial inequity is fair and why? In Proceedings of the 39th Annual Conference of the Cognitive Science Society , 2017. -
- [47][47] T. Liao, R. Taori, I. D. Raji, and L. Schmidt. Are we learning yet? a meta review of evaluation failures across machine learning. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) , 2021. -
- [48][48] J. Light, M. Cai, S. Shen, and Z. Hu. Avalonbench: Evaluating llms playing the game of avalon. In NeurIPS 2023 Foundation Models for Decision Making Workshop , 2023. -
Cited by
Understanding World or Predicting Future? A Comprehensive Survey of World Models
Ding, J., Zhang, Y., Shang, Y., Zhang, Y., Zong, Z., Feng, J., Yuan, Y., Su, H., Li, N., Sukiennik, N., et al. (2026) · ACM Computing Surveys · doi:10.1145/3746449 · Cited by 169
Multi-Agent Risks from Advanced AI
Hammond, L., Chan, A., Clifton, J., Hoelscher-Obermaier, J., Khan, A., McLean, E., Smith, C., Barfuss, W., Foerster, J., Gavenčiak, T., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.14143 · Cited by 305
Shi, T., Mo, Y., Liu, Y., Hao, Z., Wang, Y., Hu, W., Yu, N., Zhou, M., & Yu, J. (2026) · arXiv · doi:10.48550/arxiv.2606.04306 · Cited by 7
Large Language Model Reasoning Failures
Song, P., Han, P., & Goodman, N. (2026) · arXiv · doi:10.48550/arxiv.2602.06176 · Cited by 3
Does Socialization Emerge in AI Agent Society? A Case Study of Moltbook
Li, M., Li, X., & Zhou, T. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813123 · Cited by 30
“Recent work has progressively scaled LLM systems from individual autonomous agents [49] to networked societies [40, 41].”
Metadata
- Authors
- Piatti, G., Jin, Z., Kleiman-Weiner, M., Schölkopf, B., Sachan, M., & Mihalcea, R.
- Venue
- Advances in Neural Information Processing Systems 37
- Published
- 2024
- Pages
- 111715-111759
- ISSN
- 1049-5258 (ISSN-L)
- OpenAlex
- W4415795596
- Cited by
- 14
- References
- 78
- Citations by year
- 2026: 12 · 2025: 2
- FWCI
- 18.37
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by-sa
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- National Science Foundation
- Open Philanthropy Project
- Future of Life Institute
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (201009)
- Bundesministerium für Bildung und Forschung (01IS18039B, FKZ: 01IS18039B)
- Topics
- Social Sciences › Sociology and Political Science › Evolutionary Game Theory and Cooperation
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Social Sciences › Decision Sciences › Management Science and Operations Research › Game Theory and Applications
- Keywords
- LLM agents, multi-agent systems, sustainable cooperation, agent cooperation, emergent behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-11-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.