CoChat

Optimization for Reinforcement Learning: From a single agent to cooperative agents

Open access
Lee, Donghwan, He, Niao, Kamalaruban, Parameswaran, Cevher, Volkan · 2020 · IEEE Signal Processing Magazine

The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.

Abstract

Fueled by recent advances in deep neural networks, reinforcement learning (RL) has been in the limelight because of many recent breakthroughs in artificial intelligence, including defeating humans in games (e.g., chess, Go, StarCraft), self-driving cars, smart-home automation, and service robots, among many others. Despite these remarkable achievements, many basic tasks can still elude a single RL agent. Examples abound, from multiplayer games, multirobots, cellular-antenna tilt control, traffic-control systems, and smart power grids to network management.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/msp.2020.2976000

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 19

  1. bertsekas, 1996, Neuro-Dynamic Programming
  2. dai, 0, Boosting the actor with dual critic, Proc Intl Conf on Learning Representations
  3. puterman, 2014, Markov Decision Processes Discrete Stochastic Dynamic Programming
  4. dai, 0, SBEED: Convergent reinforcement learning with nonlinear function approximation, Proc Int Conf Machine Learning, 1133
  5. dai, 0, Learning from conditional distributions via dual embeddings, Proc Int Conf Artificial Intelligence Statistics, 1458
  6. sutton, 0, A convergent O(n) temporal-difference algorithm for off-policy learning with linear function approximation, Proc Advances Neural Information Processing System Conf, 1609
  7. lee, 0, Target-based gradient TD-learning, Proc Int Conf Machine Learning, 3713
  8. leibo, 0, Multi-agent reinforcement learning in sequential social dilemmas, Proc 16th Conf Autonomous Agents and MultiAgent Systems, 464
  9. dalal, 0, Finite sample analyses for TD(0) with function approximation, Proc AAAI Conf Artificial Intelligence, 6144
  10. bhandari, 2018, A finite time analysis of temporal difference learning with linear function approximation
  11. hong, 0, Prox-PDA: The proximal primal-dual algorithm for fast distributed nonconvex optimization and learning over networks, Proc 34th Int Conf Mach Learn, 1529
  12. schulman, 0, Trust region policy optimization, Proc 31st Int Conf Mach Learn, 1889
  13. claus, 0, The dynamics of reinforcement learning in cooperative multiagent systems, Proc 15th Nat Conf Artificial Intelligence, 746
  14. zhang, 2018, Finite-sample analyses for fully decentralized multi-agent reinforcement learning
  15. wai, 0, Multi-agent reinforcement learning via double averaging primal-dual optimization, Proc Advances in Neural Information Processing Systems, 9672
  16. foerster, 0, Stabilising experience replay for deep multi-agent reinforcement learning, Proc 34th Int Conf Mach Learn, 70, 1146
  17. lauer, 0, An algorithm for distributed reinforcement learning in cooperative multi-agent systems, Proc 17th Int Conf Machine Learning, 535
  18. lauer, 0, Reinforcement learning for stochastic cooperative multi-agent systems, Proc 3rd Int Joint Conf Autonomous Agents and Multiagent Systems, 3, 1516
  19. peshkin, 0, Learning to cooperate via policy search, Proc 16th Conf Uncertainty in Artificial Intelligence, 489

Cited by

See all 113

Metadata

Authors
Lee, D., He, N., Kamalaruban, P., & Cevher, V.
Affiliations
  • Purdue University West Lafayette (US)
  • University of Illinois Urbana-Champaign (US)
  • École Polytechnique Fédérale de Lausanne (CH)
  • Rice University (US)
Venue
IEEE Signal Processing Magazine
Published
2020-05
Volume
37
Issue
3
Pages
123-135
ISSN
  • 1053-5888 (ISSN-L)
  • 1558-0792
OpenAlex
W3023368031
Cited by
113
References
42
Citations by year
2026: 7 · 2025: 21 · 2024: 17 · 2023: 15 · 2022: 22 · 2021: 26 · 2020: 4
FWCI
6.58
Type
journal article
Language
en
Open access
Locations
  • IEEE Signal Processing Magazine (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
Funders
NSF (NSF-CRII-1755829)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
  • Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
Keywords
reinforcement learning, multi-agent reinforcement learning, cooperative agents, optimization, deep reinforcement learning, multi-robot systems, traffic management, smart grid, network management

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.