Formal contracts mitigate social dilemmas in multi-agent reinforcement learning
Open accessThe DOI matches its Crossref record; 52 of the 59 references it cites were found in a registry or an index, and 1 are not checked yet; none of the 13 sources its claims cite is checked for support yet.
Abstract
Multi-agent Reinforcement Learning (MARL) is a powerful tool for training autonomous agents acting independently in a common environment. However, it can lead to sub-optimal behavior when individual incentives and group incentives diverge. Humans are remarkably capable at solving these social dilemmas. It is an open problem in MARL to replicate such cooperative behaviors in selfish agents. In this work, we draw upon the idea of formal contracting from economics to overcome diverging incentives between agents in MARL. We propose an augmentation to a Markov game where agents voluntarily agree to binding transfers of reward, under pre-specified conditions. Our contributions are theoretical and empirical. First, we show that this augmentation makes all subgame-perfect equilibria of all Fully Observable Markov Games exhibit socially optimal behavior, given a sufficiently rich space of contracts. Next, we show that for general contract spaces, and even under partial observability, richer contract spaces lead to higher welfare. Hence, contract space design solves an exploration-exploitation tradeoff, sidestepping incentive issues. We complement our theoretical analysis with experiments. Issues of exploration in the contracting augmentation are mitigated using a training methodology inspired by multi-objective reinforcement learning: Multi-Objective Contract Augmentation Learning. We test our methodology in static, single-move games, as well as dynamic domains that simulate traffic, pollution management, and common pool resource management.
Checked 2026-11-19 · 1 reference not checked yet
Overview
CoChat Summary
TL;DR
The paper formalizes formal contracting as a voluntary, binding zero-sum reward-transfer augmentation to Markov games and shows that sufficiently rich contracts can make socially optimal behavior the only subgame-perfect equilibrium under full observability or detectable deviations. It also introduces MOCA, a multi-objective training method that improves welfare over separate training and gifting and often matches or exceeds joint training in Prisoner’s Dilemma, Public Goods, Harvest, Cleanup, and Emergency Merge.
Key findings
- With full contractible observations \(O_0(s,a)=(s,a)\) and the full contract space \(\Theta_{\text{full}}\), all subgame-perfect equilibria of the augmented fully observable Markov game are jointly optimal; the same holds for partially observable Markov games if deviations from a socially optimal profile are detectable and the contract space is sufficiently rich. If contractible observations are perfectly uninformative (\(\Omega_0=\{0\}\)), the augmented game has the same subgame-perfect equilibria as the original game.
- More expected-transfer-expressive contract spaces improve worst-case and best-case subgame-perfect welfare: if \(C_1 \preceq_{\text{ETE}} C_2\), then \(W(C_1)\le W(C_2)\) and \(B(C_1)\le B(C_2)\). Feature monotonicity follows when one feature map factors through another, and random-feature monotonicity holds under a Markov-chain condition. Arbitrary unconditional transfers are necessary for the welfare lower bound; without them, a “Bribing the Contract Proposer” example shows more contracts can lower equilibrium welfare.
- In experiments, a contract fining a Prisoner’s Dilemma defector 1.5 units makes cooperation dominant and \((C,C)\) the only Nash equilibrium. MOCA reaches joint optimality in Prisoner’s Dilemma for 2, 4, and 8 agents; in complex domains it at least matches joint training in almost all cases, except 2-agent Harvest and 2-agent Cleanup, and in Emergency Merge both vanilla contracting and MOCA significantly outperform separate training. In a 4-agent Cleanup monotonicity study, welfare decreases as the probability \(\alpha\) that cleaning is unobserved increases from 0 to 1, with a plateau for middle values and a sharp drop from \(\alpha=0.75\) to \(\alpha=1.0\).
Methods
- Theoretical design: defines fully and partially observable Markov games, contracts as zero-sum reward transfers conditioned on contractible observations, and a \(C\)-augmented game where agents propose contracts and unanimous acceptance is required. Analysis uses subgame-perfect equilibrium, welfare, expected-transfer expressiveness, and forcing contracts.
- Experimental design: trains with PPO via Ray RLlib and compares joint training, separate training, gifting, vanilla contracting, and MOCA. Domains are Prisoner’s Dilemma, Public Goods, Harvest, Cleanup, and Emergency Merge, with 2, 4, and 8 agents, 5 independent runs, 1M environment steps for static games, 10M for dynamic games, and 15M for the monotonicity study. MOCA has two phases: random contract sampling to estimate values, then frozen proposal/acceptance training with \(\eta=2\) sampled non-proposing agents.
- Data and code: no datasets were generated or analyzed; replication code is linked at https://github.com/Algorithmic-Alignment-Lab/contracts.
Limitations
- The paper states that a single contract proposer is crucial: with multiple proposers, subgame-perfect equilibria may be socially suboptimal. It also states fairness concerns—non-proposers are indifferent between accepting and rejecting, while the proposer captures surplus—and notes that limiting transfers may be needed for fairness at possible welfare cost.
- The paper states that contract spaces were hand-engineered and that scaling requires managing a complexity tradeoff, domain-independent contract design, and better sample efficiency. It also states that arbitrary unconditional transfers are necessary for its welfare-bound result, that contracting in POMGs does not generally reduce to DEC-POMDPs, and that collusion is not considered.
- Evident from the text: experiments rely on low-dimensional, hand-designed contract spaces and engineered features, so generalization to unengineered or real-world domains is untested; the paper itself notes the MOCA phase split is arbitrary, and the observed monotonicity decay is not uniform, with no strong pattern in which contract parameter is selected.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures











Verification
Claims the paper makes
0 Valid · 1 Not valid · 13 Not checked · 74 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Contracting improves system performance in several complex game environments.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We show, both game-theoretically and empirically, that providing agents with the option to contract allows for optimal system performance, in the sense of maximizing sum of agents' rewards, regardless of the initial reward signal.”
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Arbitrary unconditional transfers are necessary for Proposition 3 to hold. We provide a counterexample to Proposition 3 without arbitrary unconditional transfers in Sect. 4.5.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Hence, the statement continues to hold for a repeatedly proposing agent.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 3 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In Prisoner's Dilemma, contracting reaches joint optimality for 2, 4, and 8 agents.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
49 more claims
Create a free account to see all 88 claims and to verify the 3 references still open.
- It may state two separate claims as one. 24
- Probably not a substantial claim. 13
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4 More possible contracts improve welfare · p. 12
“Definition 7 We say a contracting space u1D6AF admits arbitrary unconditional transfers if and only if, for any u1D73D ∈ u1D6AF , and any v ∈ ℝ N , we have that u1D73D ′ ∈ u1D6AF , where u1D73D ′ is identical to u1D73D , except for an additional transfer of v at contract acceptance.”
§ 2.2 The contracting augmentation · p. 6
“Definition 3 (Contract) A contract is a function u1D73D ∶ Ω 0 → ℝ N whose range consists of zero-sum vectors”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 9 Conclusion and future work · p. 29
“We propose a specialized algorithm for contracting, MultiAgent Contract Augmentation Learning, and matches or exceeds the welfare of training all agents with one controller.”
§ 5.4 Training · p. 22
“However, it fails to match joint training in Public Goods”
Citations
Cites
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Advances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 8,890
Moral Hazard and Observability
Holmstrom, B. (1979) · The Bell Journal of Economics · doi:10.2307/3003320 · Cited by 8,311 · cited 1×
“In our proof of Theorem 1, we use a forcing contract , compare [13].”
(2012) · American Economic Review · doi:10.1257/aer.102.1.i · Cited by 8,043
COUNTERSPECULATION, AUCTIONS, AND COMPETITIVE SEALED TENDERS
Vickrey, W. (1961) · The Journal of Finance · doi:10.1111/j.1540-6261.1961.tb02789.x · Cited by 7,722 · cited 1×
“The setting of an agent proposing state-dependent reward transfers has received considerable attention in contract economics, compare the literature following [27], and mechanism design, compare [16, 28, 40].”
Myerson, R. B. (1981) · Mathematics of Operations Research · doi:10.1287/moor.6.1.58 · Cited by 5,920 · cited 1×
“The setting of an agent proposing state-dependent reward transfers has received considerable attention in contract economics, compare the literature following [27], and mechanism design, compare [16, 28, 40].”
(1994) · Cited by 5,513
The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver
Smith (1980) · IEEE Transactions on Computers · doi:10.1109/tc.1980.1675516 · Cited by 3,716 · cited 1×
“Smith [36] proposes the contract net protocol, which allocates tasks among agents and commits them to complete a particular task.”
Mussa, M., & Rosen, S. (1978) · Journal of Economic Theory · doi:10.1016/0022-0531(78)90085-6 · Cited by 3,306 · cited 1×
“The setting of an agent proposing state-dependent reward transfers has received considerable attention in contract economics, compare the literature following [27], and mechanism design, compare [16, 28, 40].”
The Folk Theorem in Repeated Games with Discounting or with Incomplete Information
Fudenberg, D., & Maskin, E. (1986) · Econometrica · doi:10.2307/1911307 · Cited by 2,395 · cited 1×
“Note that the system performance definition is complicated by the existence of potentially multiple equilibria, compare [7].”
Game Theory for Applied Economists
Gibbons, R. S. (1992) · Princeton University Press eBooks · doi:10.1515/9781400835881 · Cited by 1,466
Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., & Zaremba, W. (2017) · arXiv · doi:10.48550/arxiv.1707.01495 · Cited by 1,355
The Complexity of Decentralized Control of Markov Decision Processes
Bernstein, D. S., Givan, R., Immerman, N., & Zilberstein, S. (2002) · Mathematics of Operations Research · doi:10.1287/moor.27.4.819.297 · Cited by 1,192
Game Theory for Applied Economists
Gibbons, R. (1992) · Princeton University Press eBooks · doi:10.2307/j.ctvcmxrzd · Cited by 916 · cited 2×
“The research program on equilibrium refinements studies how to discipline off-path beliefs, compare [9, ch. 4].”
Four formal(izable) theories of the firm?
Gibbons, R. (2005) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2004.09.010 · Cited by 690
RLlib: Abstractions for Distributed Reinforcement Learning
Liang, E., Liaw, R., Moritz, P., Nishihara, R., Fox, R., Goldberg, K., Gonzalez, J. E., Jordan, M. I., & Stoica, I. (2017) · arXiv · doi:10.48550/arxiv.1712.09381 · Cited by 473
The StarCraft Multi-Agent Challenge
Samvelyan, M., Rashid, T., Schroeder de Witt, C., Farquhar, G., Nardelli, N., Rudner, T. G. J., Hung, C.-M., Torr, P. H. S., Foerster, J., & Whiteson, S. (2019) · International Joint Conference on Autonomous Agents and Multiagent Systems · doi:10.65109/lvzz5205 · Cited by 453
Learning modular neural network policies for multi-task and multi-robot transfer
Devin, C., Gupta, A., Darrell, T., Abbeel, P., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989250 · Cited by 310
Rusu, A. A., Colmenarejo, S. G., Gulcehre, C., Desjardins, G., Kirkpatrick, J., Pascanu, R., Mnih, V., Kavukcuoglu, K., & Hadsell, R. (2015) · arXiv · doi:10.48550/arxiv.1511.06295 · Cited by 270
Multi-agent Reinforcement Learning in Sequential Social Dilemmas
Leibo, J. Z., Zambaldi, V., Lanctot, M., Marecki, J., & Graepel, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.03037 · Cited by 263
Hardt, M., Megiddo, N., Papadimitriou, C., & Wootters, M. (2016) · Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science · doi:10.1145/2840728.2840730 · Cited by 223 · cited 1×
“Stackelberg learning has received a lot of attention in strategic Machine Learning [12, 25, 44] and has been used to learn large scale mechanisms such as auctions [3, 6].”
Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning
Parisotto, E., Ba, J. L., & Salakhutdinov, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06342 · Cited by 186
Distral: Robust Multitask Reinforcement Learning
Teh, Y. W., Bapst, V., Czarnecki, W. M., Quan, J., Kirkpatrick, J., Hadsell, R. T., Heess, N., & Pascanu, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.04175 · Cited by 172 · cited 1×
“One potential way to increase the sample efficiency of the first phase of MOCA is to leverage more of the literature on multi-task reinforcement learning methods [5, 30, 32, 38].”
Advantages of a leveled commitment contracting protocol
Sandholm, T., & Lesser, V. (1996) · Cited by 152
Multi-agent Reinforcement Learning in Sequential Social Dilemmas
Leibo, J. Z., Zambaldi, V., Lanctot, M., Marecki, J., & Graepel, T. (2017) · arXiv (Cornell University) · doi:10.5555/3091125.3091194 · Cited by 150
Optimal Auctions through Deep Learning: Advances in Differentiable Economics
Dütting, P., Feng, Z., Narasimhan, H., Parkes, D. C., & Ravindranath, S. S. (2023) · Journal of the ACM · doi:10.1145/3630749 · Cited by 145 · cited 1×
“Stackelberg learning has received a lot of attention in strategic Machine Learning [12, 25, 44] and has been used to learn large scale mechanisms such as auctions [3, 6].”
The Social Cost of Strategic Classification
Milli, S., Miller, J., Dragan, A. D., & Hardt, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287576 · Cited by 138 · cited 1×
“Stackelberg learning has received a lot of attention in strategic Machine Learning [12, 25, 44] and has been used to learn large scale mechanisms such as auctions [3, 6].”
Inequity aversion improves cooperation in intertemporal social dilemmas
Hughes, E., Leibo, J. Z., Phillips, M. G., Tuyls, K., Duéñez-Guzmán, E. A., Castañeda, A. G., Dunning, I., Zhu, T., McKee, K. R., Koster, R., et al. (2018) · arXiv · doi:10.48550/arxiv.1803.08884 · Cited by 134
Learning to Incentivize Other Learning AgentsPreprint
Yang, J., Li, A., Farajtabar, M., Sunehag, P., Hughes, E., & Zha, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06051 · Cited by 109
Vinitsky, E., Koster, R., Agapiou, J., Duéñez‐Guzmán, E. A., Vezhnevets, A. S., & Leibo, J. Z. (2023) · Collective Intelligence · doi:10.1177/26339137231162025 · Cited by 88
Evolution of commitment and level of participation in public goods games
Han, T. A., Pereira, L. M., & Lenaerts, T. (2017) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-016-9338-4 · Cited by 82 · cited 1×
“Han et al. [11] considers the idea of commitments, without transfers, in evolutionary game theory.”
The Mathematics of Tucker: A Sampler
Tucker, A. W. (1983) · The Two-Year College Mathematics Journal · doi:10.2307/3027092 · Cited by 80 · cited 2×
“In addition to classical static social dilemmas such as Prisoner’s Dilemma (compare [39]) and a public goods game (compare [17]), we also consider more complex social dilemmas such as the Harvest and Cleanup domains of Hughes et al. [14].”
Who Leads and Who Follows in Strategic Classification?Preprint
Zrnic, T., Mazumdar, E. V., Sastry, S., & Jordan, M. I. (2021) · Neural Information Processing Systems · doi:10.48550/arxiv.2106.12529 · Cited by 76
Monderer, D., & Tennenholtz, M. (2004) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1231 · Cited by 67
Negotiation and honesty in artificial intelligence methods for the board game of Diplomacy
Kramár, J., Eccles, T., Gemp, I., Tacchetti, A., McKee, K. R., Malinowski, M., Graepel, T., & Bachrach, Y. (2022) · Nature Communications · doi:10.1038/s41467-022-34473-5 · Cited by 66 · cited 1×
“Cooperation and prosociality in complex domains are of keen interest to MARL researchers, with related challenges including dilemmas like Gathering and Wolfpack [21], the StarCraft challenge [33], or more recently with MARL results in Diplomacy [20].”
Four Formal(Izable) Theories of the Firm?Preprint
Gibbons, R. S. (2004) · SSRN Electronic Journal · doi:10.2139/ssrn.596864 · Cited by 62
Formal and Informal Contracting: Theory and Evidence
Gil, R., & Zanarone, G. (2017) · Annual Review of Law and Social Science · doi:10.1146/annurev-lawsocsci-110316-113413 · Cited by 56 · cited 2×
“Formal contracts have been considered as an alternative to relational contracts in the fields of organizational Economics, see [10] and [8, 5.2.3].”
InfluentialDifferentiable Economics for Randomized Affine Maximizer Auctions
Curry, M. J., Sandholm, T., & Dickerson, J. P. (2022) · International Joint Conference on Artificial Intelligence · doi:10.48550/arxiv.2202.02872 · Cited by 49
Differentiable Economics for Randomized Affine Maximizer Auctions
Curry, M. J., Sandholm, T., & Dickerson, J. W. T. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/293 · Cited by 46
The Mathematics of Tucker: A Sampler
Tucker, A. W., & Straffin, P. D. (1983) · doi:10.1080/00494925.1983.11972687 · Cited by 45
Model-free conventions in multi-agent reinforcement learning with heterogeneous preferencesPreprint
Koster, R., McKee, K. R., Everett, R., Weidinger, L., Isaac, W. S., Hughes, E., Duéñez‐Guzmán, E. A., Graepel, T., Botvinick, M., & Leibo, J. Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.09054 · Cited by 38
Spurious normativity enhances learning of compliance and enforcement behavior in artificial agents
Köster, R., Hadfield-Menell, D., Everett, R., Weidinger, L., Hadfield, G. K., & Leibo, J. Z. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2106028118 · Cited by 37 · cited 1×
“Other related publications study the emergence and learning of social norms [19, 41] as well as conventions [18].”
Self-citationEmergent Prosociality in Multi-Agent Games Through Gifting
Wang, W. Z., Beliaev, M., Bıyık, E., Lazar, D. A., Pedarsani, R., & Sadigh, D. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/61 · Cited by 36
Gifting in Multi-Agent Reinforcement Learning
Lupu, A., & Precup, D. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/hmve9255 · Cited by 32
Monderer, D., & Tennenholtz, M. (2003) · ACM Conference on Electronic Commerce (EC) · doi:10.1145/779928.779931 · Cited by 29
Emergent Prosociality in Multi-Agent Games Through Gifting
Wang, W. Z., Beliaev, M., Biyik, E., Lazar, D. A., Pedarsani, R., & Sadigh, D. (2021) · International Joint Conference on Artificial Intelligence · doi:10.48550/arxiv.2105.06593 · Cited by 29
Learning to Resolve Alliance Dilemmas in Many-Player Zero-Sum Games
Hughes, E., Anthony, T. W., Eccles, T., Leibo, J. Z., Balduzzi, D., & Bachrach, Y. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/wuwb6150 · Cited by 23 · cited 3×
“Hughes et al. [15]’s contracts— as opposed to our concept of formal contracts —might be insufficient to induce the desired joint behavior, as agents will only commit to actions which improve their own individual welfare over the basic game, leading to Pareto optima, which may not be jointly optimal.”
InfluentialStrategic negotiations for extensive-form games
de Jonge, D., & Zhang, D. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09424-y · Cited by 15
Strategic ClassificationPreprint
Hardt, M., Megiddo, N., Papadimitriou, C. H., & Wootters, M. K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06980 · Cited by 15
Smith, R. G. (1981) · IEEE Transactions on Computers · doi:10.1109/tc.1981.1675799 · Cited by 13
The Importance of Credo in Multiagent Learning
Radke, D., Larson, K. S., & Brecht, T. B. (2023) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/qtfy6777 · Cited by 12
The Social Cost of Strategic ClassificationPreprint
Milli, S., Miller, J. P., Dragan, A. D., & Hardt, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.08460 · Cited by 7
Improving Social Welfare While Preserving Autonomy via a Pareto MediatorPreprint
McAleer, S., Lanier, J. B., Dennis, M., Baldi, P., & Fox, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.03927 · Cited by 5
Coco-Q: Learning in Stochastic Games with Side Payments
Sodomka, E., Hilliard, E., Littman, M. L., & Greenwald, A. (2013) · Cited by 5
The Importance of Credo in Multiagent LearningPreprint
Radke, D., Larson, K. S., & Brecht, T. B. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.07471 · Cited by 3
Vinitsky, E., Köster, R., Agapiou, J. P., Duéñez-Guzmán, E., Vezhnevets, A. S., & Leibo, J. Z. (2021) · arXiv · doi:10.48550/arxiv.2106.09012 · Cited by 2
Commitment and Participation in Public Goods Games
Han, T. A., Pereira, L. M., & Lenaerts, T. (2017) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/woyl6704
Strategic Negotiations for Extensive-Form Games
de Jonge, D., & Zhang, D. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/xlkz6682
The StarCraft Multi-Agent Challenge
Samvelyan, M., Rashid, T., de Witt, C. S., et al. (2019) · arXiv · doi:10.48550/arxiv.1902.04043
Cited by
Peer Rewarding for Multi-agent Coordination: Why and How
Li, Y., Hernandez, M. P., & Aydın, M. E. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39395-1_15
Pavliuchenkov, D., & Gubanov, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39395-1_35
Alqithami, S. (2026) · Mathematical and Computational Applications · doi:10.3390/mca31030069
The Cooperation and Competition Mechanisms In Multi-agent Reinforcement Learning
Li, P., Li, S., & Wang, J. (2026) · ITM Web of Conferences · doi:10.1051/itmconf/20268404004
“In the smart grid system, the decentralized energy system must coordinate to optimize energy production and consumption, but it also faces competition in the energy market [18].”
Xu, X. (2025) · Journal of Computational Methods in Sciences and Engineering · doi:10.1177/14727978251361392
Metadata
- Authors
- Haupt, A., Christoffersen, P., Damani, M., & Hadfield-Menell, D.
- Affiliations
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- Corresponding author
- Haupt, Andreas
- Christoffersen, Phillip
- Venue
- Autonomous Agents and Multi-Agent Systems
- Published
- 2024-10-18
- Volume
- 38
- Issue
- 2
- ISSN
- 1387-2532 (ISSN-L)
- 1573-7454
- OpenAlex
- W4403503820
- Cited by
- 5
- References
- 35
- Citations by year
- 2026: 4 · 2025: 1
- FWCI
- 0.74
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Autonomous Agents and Multi-Agent Systems (publishedVersion, cc-by)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
- Funders
- Effective Giving
- Massachusetts Institute of Technology
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Game Theory and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Keywords
- multi-agent reinforcement learning, social dilemmas, formal contracts, incentive alignment, subgame perfect equilibrium, exploration-exploitation balance, multi-objective reinforcement learning, stochastic games, partial observability, traffic management, pollution management, common-pool resources
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.