CoChat
Sign Up
Cited by 39Reinforcement Learning in RoboticsGame Theory and ApplicationsAdvanced Bandit Algorithms Research

The DOI matches its Crossref record; the references it cites are not checked yet.

Exploration-exploitation in multi-agent learning: Catastrophe theory meets game theory

Open access
Leonardos, Stefanos, Piliouras, Georgios·2022·Artificial Intelligence
DOI 10.1016/j.artint.2021.103653
Open this paper in CoChat — free account

Overview

Integrity signals

Registry

Crossref · 10.1016/j.artint.2021.103653

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Game theory has been a booming field with several representative privacy-preserving techniques coming out in the past decade.

    “Game theory has been a booming field with several representative privacy-preserving techniques coming out in the past decade [122].”

    From Machine Unlearning: A Survey (2024) · cites this paper as [122]

Citations

The citation lists haven’t been fetched yet

References 88 · Cited by 39

Research works printed but not linked: 50

  1. Camerer, 1999, Experience-weighted attraction learning in normal form games, Econometrica, 67, 827, 10.1111/1468-0262.00054 · DOI
  2. Ho, 2007, Self-tuning experience weighted attraction learning in games, J. Econ. Theory, 133, 177, 10.1016/j.jet.2005.12.008 · DOI
  3. Camerer, 1998, Experience-weighted attraction learning in coordination games: probability rules, heterogeneity, and time-variation, J. Math. Psychol., 42, 305, 10.1006/jmps.1998.1217 · DOI
  4. Bowling, 2002, Multiagent learning using a variable learning rate, Artif. Intell., 136, 215, 10.1016/S0004-3702(02)00121-2 · DOI
  5. Badia, 2020, Agent57: Outperforming the Atari Human Benchmark, vol. 119, 507
  6. Omidshafiei, 2019, α-rank: multi-agent evaluation by evolution, Sci. Rep., 9, 9937, 10.1038/s41598-019-45619-9 · DOI
  7. Rowland, 2019, Multiagent evaluation under incomplete information, vol. 32, 12291
  8. Kaisers, 2010, Frequency adjusted multi-agent q-learning, vol. 1, 309, 10.65109/VLUE9705 · DOI
  9. Bloembergen, 2015, Evolutionary dynamics of multi-agent learning: a survey, J. Artif. Intell. Res., 53, 659, 10.1613/jair.4818 · DOI
  10. Sato, 2003, Coupled replicator equations for the dynamics of learning in multiagent systems, Phys. Rev. E, 67, 10.1103/PhysRevE.67.015206 · DOI
  11. Sato, 2005, Stability and diversity in collective adaptation, Phys. D: Nonlinear Phenom., 210, 21, 10.1016/j.physd.2005.06.031 · DOI
  12. Tuyls, 2012, Multiagent learning: basics, challenges, and prospects, AI Mag., 33, 41
  13. Mazumdar, 2020, On gradient-based learning in continuous games, SIAM J. Math. Data Sci., 2, 103, 10.1137/18M1231298 · DOI
  14. Wolpert, 2012, Hysteresis effects of changing the parameters of noncooperative games, Phys. Rev. E, 85, 10.1103/PhysRevE.85.036102 · DOI
  15. Sanders, 2018, The prevalence of chaotic dynamics in games with many players, Sci. Rep., 8, 4902, 10.1038/s41598-018-22013-5 · DOI
  16. Kianercy, 2012, Dynamics of Boltzmann Q learning in two-player two-action games, Phys. Rev. E, 85, 10.1103/PhysRevE.85.041145 · DOI
  17. Alós-Ferrer, 2010, The logit-response dynamics, Games Econ. Behav., 68, 413, 10.1016/j.geb.2009.08.004 · DOI
  18. Kaelbling, 1996, Reinforcement learning: a survey, J. Artif. Intell. Res., 4, 237, 10.1613/jair.301 · DOI · Paper page
  19. Mertikopoulos, 2016, Learning in games via reinforcement and regularization, Math. Oper. Res., 41, 1297, 10.1287/moor.2016.0778 · DOI
  20. McKelvey, 1995, Quantal response equilibria for normal form games, Games Econ. Behav., 10, 6, 10.1006/game.1995.1023 · DOI
  21. Kwoon, 2017, A continuous-time approach to online optimization, J. Dyn. Games, 4, 125, 10.3934/jdg.2017008 · DOI
  22. Leslie, 2005, Individual Q-learning in normal form games, SIAM J. Control Optim., 44, 495, 10.1137/S0363012903437976 · DOI
  23. Coucheney, 2015, Penalty-regulated dynamics and robust learning procedures in games, Math. Oper. Res., 40, 611, 10.1287/moor.2014.0687 · DOI
  24. Tuyls, 2006, An evolutionary dynamical analysis of multi-agent learning in iterated games, Auton. Agents Multi-Agent Syst., 12, 115, 10.1007/s10458-005-3783-9 · DOI
  25. Göcke, 2002, Various concepts of hysteresis applied in economics, J. Econ. Surv., 16, 167, 10.1111/1467-6419.00163 · DOI
  26. Romero, 2015, The effect of hysteresis on equilibrium selection in coordination games, J. Econ. Behav. Organ., 111, 88, 10.1016/j.jebo.2014.12.029 · DOI
  27. Li, 2018, Visualizing the loss landscape of neural nets, vol. 31, 6389
  28. Kianercy, 2014, Critical transitions in a game theoretic model of tumour metabolism, Interface Focus, 4, 10.1098/rsfs.2014.0014 · DOI
  29. Leonardos, 2020, Catastrophe by Design in Population Games: Destabilizing Wasteful Locked-In Technologies, vol. 12495, 473
  30. Tuyls, 2007, What evolutionary game theory tells us about multiagent learning, Artificial Intelligence, 171, 406, 10.1016/j.artint.2007.01.004 · DOI
  31. Galla, 2013, Complex dynamics in learning complicated games, Proc. Natl. Acad. Sci., 110, 1232, 10.1073/pnas.1109672110 · DOI
  32. Leonardos, 2021, Exploration-exploitation in multi-agent learning: catastrophe theory meets game theory, Proceedings of the AAAI Conference on Artificial Intelligence, 35, 11263, 10.1609/aaai.v35i13.17343 · DOI
  33. Ben-Porat, 2018, A game-theoretic approach to recommendation systems with strategic content providers, vol. 31
  34. Swenson, 2018, On best-response dynamics in potential games, SIAM J. Control Optim., 56, 2734, 10.1137/17M1139461 · DOI
  35. Perolat, 2021, From Poincaré recurrence to convergence in imperfect information games: finding equilibrium via regularization, vol. 139, 8525
  36. Mguni, 2021, Learning in nonzero-sum stochastic games with potentials, vol. 139, 7688
  37. Watkins, 1992, Technical note: Q-learning, Mach. Learn., 8, 279
  38. Roughgarden, 2015, Intrinsic robustness of the price of anarchy, J. ACM, 62, 10.1145/2806883 · DOI
  39. Bai, 2020, Provable Self-Play Algorithms for Competitive Reinforcement Learning, vol. 119, 551
  40. Smith, 2019, Super-convergence: very fast training of neural networks using large learning rates, vol. 11006, 369
  41. Schmidt, 2003, Playing safe in coordination games:: the roles of risk dominance, payoff dominance, and history of play, Games Econ. Behav., 42, 281, 10.1016/S0899-8256(02)00552-3 · DOI
  42. Kim, 1996, Equilibrium selection in n-person coordination games, Games Econ. Behav., 15, 203, 10.1006/game.1996.0066 · DOI
  43. Vinyals, 2019, Grandmaster level in StarCraft II using multi-agent reinforcement learning, Nature, 575, 350, 10.1038/s41586-019-1724-z · DOI · Paper page
  44. Mahajan, 2019, MAVEN: multi-agent variational exploration, vol. 32
  45. Cai, 2016, Zero-sum polymatrix games: a generalization of minmax, Math. Oper. Res., 41, 648, 10.1287/moor.2015.0745 · DOI
  46. Hwang, 2020, Strategic decompositions of normal form games: zero-sum games and potential games, Games Econ. Behav., 122, 370, 10.1016/j.geb.2020.05.003 · DOI
  47. Dafoe, 2021, Cooperative ai: machines must learn to find common ground, Nature, 7857, 33, 10.1038/d41586-021-01170-0 · DOI · Paper page
  48. Agarwal, 2020, Optimality and approximation with policy gradient methods in Markov decision processes, vol. 125, 64
  49. Daskalakis, 2020, Independent policy gradient methods for competitive reinforcement learning, vol. 33, 5527
  50. Monderer, 1996, Potential games, Games Econ. Behav., 14, 124, 10.1006/game.1996.0044 · DOI

Metadata

Authors
Leonardos, S., & Piliouras, G.
Affiliations
Singapore University of Technology and Design (SG)
Corresponding author
Leonardos, Stefanos
ORCID
Venue
Artificial Intelligence
Publisher
Elsevier BV
Published
2022-03
Volume
304
Pages
103653
ISSN
0004-3702 (ISSN-L)
DOI
10.1016/j.artint.2021.103653
OpenAlex
W3113152939
Cited by
39
References
88
Citations by year
2026: 9 · 2025: 11 · 2024: 7 · 2023: 5 · 2022: 5
FWCI
1.68
Type
journal article
Language
en
Open access
Licence
Locations
  • Artificial Intelligence (publishedVersion)
  • Research Portal (King's College London) (publishedVersion, cc-by-nc-nd)
Funders
  • National Research Foundation Singapore (NRF-NRFF2018-07, AISG2-RP-2020-016, NRF2019-NRF-ANR095)
  • Singapore University of Technology and Design (PIE-SGP-AI-2018-01)
  • Agency for Science Technology and Research (A20H6b0151)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Game Theory and Applications
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
Keywords
exploration-exploitation balance, multi-agent learning, quantal response equilibrium, bounded rationality, catastrophe theory, bifurcation theory, phase transitions, equilibrium selection, regret bounds, exploration costs

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review