The Good Shepherd: An Oracle Agent for Mechanism Design
Open accessAbstract
From social networks to traffic routing, artificial learning agents are playing a central role in modern institutions. We must therefore understand how to leverage these systems to foster outcomes and behaviors that align with our own values and aspirations. While multiagent learning has received considerable attention in recent years, artificial agents have been primarily evaluated when interacting with fixed, non-learning co-players. While this evaluation scheme has merit, it fails to capture the dynamics faced by institutions that must deal with adaptive and continually learning constituents. Here we address this limitation, and construct agents ("mechanisms") that perform well when evaluated over the learning trajectory of their adaptive co-players ("participants"). The algorithm we propose consists of two nested learning loops: an inner loop where participants learn to best respond to fixed mechanisms; and an outer loop where the mechanism agent updates its policy based on experience. We report the performance of our mechanism agents when paired with both artificial learning agents and humans as co-players. Our results show that our mechanisms are able to shepherd the participants strategies towards favorable outcomes, indicating a path for modern institutions to effectively and automatically influence the strategies and behaviors of their constituents.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In applying tools from AI research to study resource allocation, we are building upon earlier work, in which we used deep RL to maximise reported human preferences (votes) over a resource allocation policy in a linear public goods game.
“In applying tools from AI research to study resource allocation, we are building upon earlier work, in which we used deep RL to maximise reported human preferences (votes) over a resource allocation policy in a linear public goods game24,25.”
From Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem (2025) · cites this paper as [25]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Learning to Communicate with Deep Multi-Agent Reinforcement Learning
Foerster, J. N., Assael, Y. M., de Freitas, N., & Whiteson, S. (2016) · arXiv · doi:10.48550/arxiv.1605.06676 · Cited by 1,217 · see citing works
Learning to Incentivize Other Learning AgentsPreprint
Yang, J., Li, A., Farajtabar, M., Sunehag, P., Hughes, E., & Hongyuan Zha, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06051 · Cited by 29 · see citing works
A Generalized Training Approach for Multiagent LearningPreprint
Müller, P. F., Omidshafiei, S., Rowland, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.12823 · Cited by 22 · see citing works
LEAD: Min-Max Optimization from a Physical PerspectivePreprint
Hemmat, R. A., Mitra, A., Lajoie, G., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13846
Flagged first, then by citations
Research works printed but not linked: 7
- [1][1] David Balduzzi, Sebastien Racaniere, James Martens, Jakob Foerster, Karl Tuyls, and Thore Graepel. The mechanics of n-player differentiable games. In International Conference on Machine Learning , pages 354-363. PMLR, 2018. -
- [3][3] George W Brown. Iterative solution of games by fictitious play. Activity analysis of production and allocation , 13(1):374-376, 1951. -
- [4][4] Jakob Foerster, Francis Song, Edward Hughes, Neil Burch, Iain Dunning, Shimon Whiteson, Matthew Botvinick, and Michael Bowling. Bayesian action decoder for deep multi-agent reinforcement learning. In International Conference on Machine Learning , pages 1942-1951. PMLR, 2019. -
- [6][6] Jakob N Foerster, Richard Y Chen, Maruan AlShedivat, Shimon Whiteson, Pieter Abbeel, and Igor Mordatch. Learning with opponent-learning awareness. arXiv preprint arXiv:1709.04326 , 2017. - · arXiv
- [13][13] D. Robinson and D. Goforth. The Topology of the 2x2 Games: A New Periodic Table . Routledge advances in game theory. Routledge, 2005. -
- [16][16] Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Marc Lanctot, Nicolas Sonnerat, Joel Z Leibo, Karl Tuyls, et al. Value-decomposition networks for cooperative multi-agent learning. arXiv preprint arXiv:1706.05296 , 2017. - · arXiv
- [17][17] Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Micha¨ el Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature , 575(7782):350-354, 2019. -
Cited by
Sorted by citations
Koster, R., Pîslar, M., Tacchetti, A., Balaguer, J., Liu, L., Elie, R., Hauser, O. P., Tuyls, K., Botvinick, M., & Summerfield, C. (2025) · Nature Communications · doi:10.1038/s41467-025-58043-7 · Cited by 12 · see citing works
Self-citationMulti-Agent Risks from Advanced AI
Hammond, L., Chan, A., Clifton, J., Hoelscher-Obermaier, J., Khan, A., McLean, E., Smith, C., Barfuss, W., Foerster, J., Gavenčiak, T., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.14143 · Cited by 10 · see citing works
- Authors
- Balaguer, J., Koster, R., Summerfield, C., & Tacchetti, A.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- arXiv
- 2202.10135
- OpenAlex
- W4221164687
- Cited by
- 4
- References
- 4
- Citations by year
- 2025: 3 · 2023: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- RePEc: Research Papers in Economics (submittedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Game Theory and Applications
- Social Sciences › Decision Sciences › Management Science and Operations Research › Auction Theory and Applications
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Game Theory and Voting Systems
- Keywords
- mechanism design, multi-agent learning, best response learning, human-agent interaction, strategic behavior, institutions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
