Cited by 273Reinforcement Learning in RoboticsDomain Adaptation and Few-Shot LearningMulti-Agent Systems and Negotiation

A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

Open access
Silva, Felipe Leno Da, Costa, Anna Helena Reali·2019·Journal of Artificial Intelligence Research

Abstract

Multiagent Reinforcement Learning (RL) solves complex tasks that require coordination with other agents through autonomous exploration of the environment. However, learning a complex task from scratch is impractical due to the huge sample complexity of RL algorithms. For this reason, reusing knowledge that can come from previous experience or other agents is indispensable to scale up multiagent RL algorithms. This survey provides a unifying view of the literature on knowledge reuse in multiagent RL. We define a taxonomy of solutions for the general knowledge reuse problem, providing a comprehensive discussion of recent progress on knowledge reuse in Multiagent Systems (MAS) and of techniques for knowledge reuse across agents (that may be actuating in a shared environment or not). We aim at encouraging the community to work towards reusing all the knowledge sources available in a MAS. For that, we provide an in-depth discussion of current lines of research and open questions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1613/jair.1.11396

Cites

159 of 159 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The pretraining approach has been successful for various machine vision problems.

    “The pretraining approach has been successful for various machine vision problems28–30.”

    From Cellpose 2.0: how to train your own model (2022) · cites this paper as [29]

    Section 5.8 describes pre-training a deep network from demonstrations for reinforcement learning, but does not address success across machine-vision problems. Section 6.3 discusses image-derived features in video games, not the success of pretraining for machine vision. Searches for pre-training in machine vision and pretrained networks in computer vision found no passage establishing the claim; page numbers were unavailable in our copy. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

11 of the 170 references couldn’t be matched to a known paper.

Research works printed but not linked: 17

  1. Abel, D., Salvatier, J., Stuhlmüller, A., & Evans, O. (2016). Agent-Agnostic Human-in- the-Loop Reinforcement Learning. In Proceedings of the NIPS Future of Interactive Learning Machines Workshop.
  2. Barto, A. G., Thomas, P. S., & Sutton, R. S. (2017). Some Recent Applications of Re- inforcement Learning. In Proceedings of the 18th Yale Workshop on Adaptive and Learning Systems.
  3. Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016). OpenAI Gym. CoRR, abs/1606.01540. · arXiv
  4. Castaneda, A. O. (2016). Deep Reinforcement Learning Variants of Multi-Agent Learning Algorithms. Ph.D. thesis, University of Edinburgh.
  5. Diuk, C. (2009). An Object-Oriented Representation for Efficient Reinforcement Learning. Ph.D. thesis, Rutgers University.
  6. Du, Y., de la Cruz, G. V., Irwin, J., & Taylor, M. E. (2016). Initial Progress in Transfer for Deep Reinforcement Learning Algorithms. In Proceedings of Deep Reinforcement Learning: Frontiers and Challenges Workshop at IJCAI.
  7. Fitzgerald, T., Bullard, K., Thomaz, A., & Goel, A. (2016). Situated Mapping for Transfer Learning. In Proceedings of the 4th Annual Conference on Advances in Cognitive Systems, pp. 1-14.
  8. PMLR.
  9. Isele, D., Rostami, M., & Eaton, E. (2016). Using Task Features for Zero-Shot Knowl- edge Transfer in Lifelong Learning. In Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI), pp. 1620-1626.
  10. Judah, K., Fern, A. P., Dietterich, T. G., & Tadepalii, P. (2014). Active Imitation Learning: Formal and Practical Reductions to I.I.D. Learning. Journal of Machine Learning Research (JMLR), 15 (1), 3925-3963.
  11. Kelly, S., & Heywood, M. I. (2015). Knowledge Transfer from Keepaway Soccer to Half- Field Offense through Program Symbiosis: Building Simple Programs for a Complex Task. In Proceedings of the 17th Conference on Genetic and Evolutionary Computation (GECCO), pp. 1143-1150. ACM.
  12. Peng, B., MacGlashan, J., Loftin, R., Littman, M. L., Roberts, D. L., & Taylor, M. E. (2016b). An Empirical Study of Non-expert Curriculum Design for Machine Learners. In Proceedings of the IJCAI Interactive Machine Learning Workshop.
  13. Perico, D. H., & Bianchi, R. A. C. (2013). Use of Heuristics from Demonstrations to Speed Up Reinforcement Learning. In Proceedings of the 12th Brazilian Symposium on Intelligent Automation (SBAI).
  14. Ramakrishnan, R., Narasimhan, K., & Shah, J. (2016). Interpretable Transfer for Reinforce- ment Learning based on Object Similarities. In Proceedings of the IJCAI Interactive Machine Learning Workshop.
  15. Schaal, S. (1997). Learning from Demonstration. In Advances in Neural Information Pro- cessing Systems (NIPS), pp. 1040-1046.
  16. Silva, F. L. D., & Costa, A. H. R. (2017b). Towards Zero-Shot Autonomous Inter-Task Map- ping through Object-Oriented Task Description. In Proceedings of the 1st Workshop on Transfer in Reinforcement Learning (TiRL).
  17. Watkins, C. J., & Dayan, P. (1992). Q-Learning. Machine Learning, 8 (3), 279-292.

Cited by

Sorted by citations

Authors
Silva, F. L. D., & Costa, A. H. R.
Affiliations
Universidade de São Paulo (BR)
Venue
Journal of Artificial Intelligence Research
Published
2019-03-11
Volume
64
Pages
645-703
ISSN
1076-9757 (ISSN-L)
OpenAlex
W2921955147
Cited by
273
References
170
Citations by year
2026: 18 · 2025: 33 · 2024: 38 · 2023: 43 · 2022: 40 · 2021: 56 · 2020: 34 · 2019: 10 · 2018: 1
FWCI
15.97
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
Journal of Artificial Intelligence Research (publishedVersion, cc-by)
Funders
  • University of Texas at Austin
  • Fundação de Amparo à Pesquisa do Estado de São Paulo (16/21047-3, 2015/, 2015/16310-4, 2018/00344-5)
  • Conselho Nacional de Desenvolvimento Científico e Tecnológico (425860/2016-7, 307027/2017-1, 2017-1)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
Keywords
multi-agent reinforcement learning, transfer learning, knowledge reuse, multi-agent systems, sample complexity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review