Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index; 5 of the 41 sources its claims cite were checked for support.
Abstract
A canonical social dilemma arises when resources are allocated to people, who can either reciprocate with interest or keep the proceeds. The right resource allocation mechanisms can encourage levels of reciprocation that sustain the commons. Here, in an iterated multiplayer trust game, we use deep reinforcement learning (RL) to design a social planner that promotes sustainable contributions from human participants. We first trained neural networks to behave like human players, creating a stimulated economy that allows us to study the dynamics of receipt and reciprocation. We use RL to train a mechanism to maximise aggregate return to players. The RL mechanism discovers a redistributive policy that leads to a large but also more equal surplus. The mechanism outperforms baseline mechanisms by conditioning its generosity on available resources and temporarily sanctioning defectors. Examining the RL policy allows us to develop a similar but explainable mechanism that is more popular among players.
Overview
CoChat Summary
TL;DR
In an iterated multiplayer trust game with a common-pool resource, deep reinforcement learning designed a social planner that promoted sustainable contributions from human participants, producing higher surplus and lower inequality than baseline mechanisms. A simpler explainable interpolating mechanism approximated the RL policy, achieved comparable or slightly lower surplus with higher equality, and was preferred by players.
Key findings
- In Exp. 1, the RL agent generated a surplus ~150% greater than the highest baseline (proportional) with a Gini coefficient just over ~0.2; it produced higher surplus than proportional (z = 3.25, p = 0.001), mixed (z = 4.38, p < 0.001), and equal (z = 6.31, p < 0.001) baselines, and lower Gini than proportional (z = 2.78, p < 0.01) but not than equal or mixed. 65% of RL-agent games were sustained to the end with at least one player, and 55% of these with all four players.
- The RL policy was more egalitarian when the pool was abundant and more punitive when resources were scarce; it temporarily excluded defectors for ~1–5 rounds and then made more generous re-inclusion offers. Lagged regressions showed the agent’s offer predicted subsequent reciprocation rather than the reverse.
- The explainable interpolating baseline, with mixing parameter w = (R/200)^22, matched the RL agent on surplus in Exp. 2 (both > proportional, z = 2.9, p < 0.01 and z = 2.52, p < 0.01; no difference between them) and had lower Gini than both proportional (z = 4.17, p < 0.001) and the RL agent (z = 3.67, p < 0.001). Human players preferred the interpolating agent as fairer, more understandable, and more prone to encourage cooperation.
- In Exp. 3, a retrained RL agent (M2) achieved higher surplus than the interpolating baseline (z = 2.35, p < 0.05), but the interpolating baseline generated lower Gini (z = 5.64, p < 0.001). In Exp. 4, M2 produced larger surplus than a proportional baseline with augmented instructions in game 1 (z = 2.22, p = 0.026), game 2 (z = 4.99, p < 0.001), and game 3 (z = 6.08, p < 0.001), and surplus grew under M2 from game 1 to 3 (z = 2.84, p = 0.005).
Methods
- The game was an infinitely repeated multiplayer trust game with p = 4 players, initial pool R0 = 200, growth factor r = 0.4, and a maximum pool size; each round the mechanism allocated endowments from the common pool, players chose how much to reciprocate, and the mechanism objective was to maximize aggregate player surplus over rounds and players.
- Human participants were recruited online via Prolific Academic; the final dataset contains 4,952 participants. Exp. 1 recruited 640 participants (160 groups of 4), with 40 groups each under equal (w = 1), mixed (w = 0.5), proportional (w = 0), and RL-agent mechanisms; later experiments used 40–80 groups per condition. No demographic data were collected.
- The pipeline collected human data under baseline mechanisms, trained recurrent neural network behavioral clones via imitation learning (BC1 and BC2), trained deep RL social planners using graph neural networks and policy gradient methods to maximize surplus against virtual players, then evaluated the mechanisms with new human participants. Analyses used Wilcoxon rank-sum tests, Gini coefficients, exclusion durations, and lagged regressions.
Limitations
- Stated in the paper: the participant cohort was drawn exclusively from the UK and USA, and no fine-grained demographic data were collected, so generalizability across gender, age, or other variables is unknown.
- Stated in the paper: the game differs from the classic CPR problem because players decide what to keep or reciprocate from a pre-set allocation rather than drawing down freely from the common pool; the mechanism’s success in this stylized game does not guarantee similar principles in more complex, naturalistic environments.
- Evident from the text: the RL policy remains hard to understand or explain, motivating the interpolating approximation; results come from a stylized online game with no communication or self-organization among players, and about 10% of games were excluded because players dropped out.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
4 Valid · 4 Not valid · 19 Not checked · 98 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The overall pattern is reminiscent of the successful ‘tit-for-two-tats’ or ‘generous tit-for-tat’ strategy in iterated prisoner’s dilemma, whereby the agent is prone to punish but quick to forgive37–39.”
§ Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem > Results
What the sources say
- Not checked yetcites [37] Axelrod, R. M. The Evolution of Cooperation Revised edition, Vol. 264 (Penguin Books, London; New York, 1990).
- Not checked yetcites [38] Fudenberg, D., Rand, D. G. & Dreber, A. Slow to anger and fast to forgive: cooperation in an uncertain world. Am. Econ. Rev. 102, 720–749 (2012). source
- Partly supportedcites [39] Nowak, M. A. Five rules for the evolution of cooperation. Science 314, 1560–1563 (2006). sourceExperimental — accuracy not yet measured
Nowak's 'Direct reciprocity' section (p. ~1 of our copy) does describe generous-tit-for-tat in the repeated Prisoner's Dilemma as a strategy that 'cooperates whenever you cooperate, but sometimes cooperates although you have defected (with probability 1 − c/b)' and adds 'Natural selection can promote forgiveness' — which covers the 'punish but quick to forgive' half of the claim for generous tit-for-tat only. However, our copy nowhere mentions 'tit-for-two-tats', so that alternative name is not attributable to this source. Moreover, the same passage qualifies the word 'successful': by various measures of success win-stay, lose-shift is 'more robust than either tit-for-tat or generous-tit-for-tat', so Nowak ranks generous tit-for-tat as an improvement on tit-for-tat under error-prone play rather than as the outstanding strategy (the 'winning strategy' he credits to Axelrod is plain tit-for-tat). (LLM, conf=0.72)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“similar positive correlations were observed for mixed conditions when games ended prematurely (r = 0.54, p < 0.01).”
§ Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem > Results
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In fact, simulations using virtual players predicted that games played under the agent mechanism would last an average of 271 ± 251 rounds, compared to 32 ± 28 and 105 ± 102 for the equal and proportional baselines respectively (Fig. S2).”
§ Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem > Results
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Surprisingly, the RL social planner identifies a resource allocation mechanism that successfully promotes sustainable behaviour among people, even without endogenous mechanisms that allows them to communicate or self-organise.”
§ Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem > Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Under proportional allocation (where any player that gives zero will receive no future allocations) we typically observe a pattern whereby several of the players are excluded early.”
§ Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem > Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
41 more claims
Create a free account to see all 125 claims and to verify the 36 references still open.
- Probably not a substantial claim. 48
- It may state two separate claims as one. 42
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem > Discussion
“successfully to promote sustainable exchange, in terms of the highest levels of return to recipients (surplus), equity (Gini) and inclusion (proportion of included players).”
§ Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem > Results
“The interpolating baseline in fact had lower Gini coefficient than both the proportional baseline (z = 4.17, p < 0.001) and the RL agent (z = 3.67, p < 0.001), indicating that it offered an excellent compromise between prosperity and equality.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem > Methods > Materials and methods
“No data were excluded from the analyses (except games with incomplete data, i.e. players dropping out).”
§ Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem > Methods > Human datasets
“We then excluded excess games to ensure balanced numbers of groups across conditions.”
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Hardin, G. (1968) · Science · doi:10.1126/science.162.3859.1243 · Cited by 25,167
Governing the Commons: The Evolution of Institutions for Collective Action
Ostrom, E. (1990) · doi:10.1017/cbo9780511807763 · Cited by 20,653
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
The graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005605 · Cited by 9,582
The Costs and Benefits of Ownership: A Theory of Vertical and Lateral Integration
Grossman, S. J., & Hart, O. D. (1986) · Journal of Political Economy · doi:10.1086/261404 · Cited by 9,352
Five Rules for the Evolution of Cooperation
Nowak, M. A. (2006) · Science · doi:10.1126/science.1133755 · Cited by 5,894
Altruistic punishment in humans
Fehr, E., & Gächter, S. (2002) · Nature · doi:10.1038/415137a · Cited by 5,616 · cited 1×
“For example, in multiplayer games, cheap talk 14,15 and onymity 16 sustain reciprocation, and participants will often opt-in to games with mechanisms that allow players to ostracise uncooperative group members at personal expense or that permit sanctioning schemes to punish free riders 17,18 .”
Trust, Reciprocity, and Social History
Berg, J., Dickhaut, J., & McCabe, K. (1995) · Games and Economic Behavior · doi:10.1006/game.1995.1027 · Cited by 5,548
Some Consequences of Having Too Little
Shah, A. K., Mullainathan, S., & Shafir, E. (2012) · Science · doi:10.1126/science.1222426 · Cited by 1,567 · cited 1×
“For example, cutting unemployment bene fi ts mayitselfcreatecircumstancesunfavourabletoreemployment 30 ,such as long-term ill-health 31 , leading to workers being permanently excluded from the labour market.”
Research works printed but not linked: 6
- Trust: Making and Breaking Cooperative Relations, Vol. 201–203 (B. Blackwell, New York, NY, USA, 1988).
- Ross, J. The economic theory of agency: the principal’s problem. Am. Econ. Rev. 63, 134–139 (1973).
- Dasgupta, P. & Heal, G. Economic Theory and Exhaustible Resources, Vol. 516 (Cambridge Univ. Pr, Cambridge, 2001).
- Kuznets, S. Economic gowth and income inequality. Am. Econ. Rev. 45, 1–28 (1955).
- Nowak, M. A. Evolutionary Dynamics: Exploring the Equations of Life Illustrated edition, Vol. 312 (Belknap Press of Harvard University Press, Cambridge, Mass, 2006).
- Axelrod, R. M. The Evolution of Cooperation Revised edition, Vol. 264 (Penguin Books, London; New York, 1990).
Cited by
Generative artificial intelligence enhances creativity but reduces the diversity of novel content
Doshi, A., & Hauser, O. (2023) · doi:10.2139/ssrn.4535536 · Cited by 1,350
Self-citationReinforcement learning in evolutionary game theory: A brief review of recent developments
Xie, K., & Szolnoki, A. (2025) · Applied Mathematics and Computation · doi:10.1016/j.amc.2025.129685 · Cited by 38
Embodied LLM Agents Learn to Cooperate in Organized Teams
Guo, X., Huang, K., Liu, J., Fan, W., Vélez, N., Wu, Q., Wang, H., Griffiths, T. L., & Wang, M. (2026) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2025.3637527 · Cited by 123
Climate Change, Intergenerational Fairness, and the Promises and Pitfalls of Artificial Intelligence
Hauser, O. (2025) · Environmental and Resource Economics · doi:10.1007/s10640-025-00986-y · Cited by 4
Self-citationDynamics of counterfactual attribution regret minimization in spatial public goods games
Yichao, W., & Tanimoto, J. (2025) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0302815 · Cited by 3
Metadata
- Authors
- Koster, R., Pîslar, M., Tacchetti, A., Balaguer, J., Liu, L., Elie, R., Hauser, O. P., Tuyls, K., Botvinick, M., & Summerfield, C.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- University of Exeter (GB)
- University of Oxford (GB)
- ORCID
- Venue
- Nature Communications
- Published
- 2025-03-22
- Volume
- 16
- Issue
- 1
- Pages
- 2824-2824
- ISSN
- 2041-1723 (ISSN-L)
- PMC
- PMC11929920
- OpenAlex
- W4408739195
- Cited by
- 12
- References
- 38
- Citations by year
- 2026: 8 · 2025: 4
- FWCI
- 12.42
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature Communications (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Open Research Exeter (University of Exeter) (submittedVersion, cc-by)
- Topics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Experimental Behavioral Economics Studies
- Social Sciences › Sociology and Political Science › Evolutionary Game Theory and Cooperation
- Social Sciences › Decision Sciences › Management Science and Operations Research › Game Theory and Applications
- Keywords
- common-pool resources, social dilemmas, reciprocity, trust game, deep reinforcement learning, social planner, resource allocation mechanisms, redistribution, sustainable behavior, repeated games
- MeSH terms
- Deep Learning, Reinforcement Machine Learning, Cooperative Behavior, Game Theory, Humans, Reinforcement, Psychology, Neural Networks, Computer, Trust, Resource Allocation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem. https://app.cochat.ai/papers/10.1038/s41467-025-58043-7
- The paper
- Koster, R., Pîslar, M., Tacchetti, A., Balaguer, J., Liu, L., Elie, R., Hauser, O. P., Tuyls, K., Botvinick, M., & Summerfield, C. (2025). Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem. Nature Communications. https://doi.org/10.1038/s41467-025-58043-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.