CoChat
Sign Up
PreprintCited by 1,307Reinforcement Learning in RoboticsDomain Adaptation and Few-Shot LearningNeural Networks and Applications

Progressive Neural Networks

Open access
Rusu, Andrei A., Rabinowitz, Neil C., Desjardins, Guillaume, Soyer, Hubert, Kirkpatrick, James, Kavukcuoglu, Koray, Pascanu, Razvan, Hadsell, Raia·2016·arXiv
DOI 10.48550/arxiv.1606.04671arXiv 1606.04671
Open this paper in CoChat — free accountOpen PDF

Abstract

Learning to solve complex sequences of tasks--while both leveraging transfer and avoiding catastrophic forgetting--remains a key obstacle to achieving human-level intelligence. The progressive networks approach represents a step forward in this direction: they are immune to forgetting and can leverage prior knowledge via lateral connections to previously learned features. We evaluate this architecture extensively on a wide variety of reinforcement learning tasks (Atari and 3D maze games), and show that it outperforms common baselines based on pretraining and finetuning. Using a novel sensitivity measure, we demonstrate that transfer occurs at both low-level sensory and high-level control layers of the learned policy.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1606.04671

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Results in [56] show that there is a trade-off between controller robustness and fine-tuning steps.

    “The analysed task is a Marble maze game driven by a robotic arm and the results show that there is a trade-off between controller robustness and ne-tuning steps.”

    From Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning (2021) · cites this paper as [56]

    Section 5.4 (p. 8) describes a 3D maze-navigation game controlled through movement and looking actions, not a marble maze driven by a robotic arm. The cited paper discusses transfer learning and finetuning, but the passages do not report the claimed trade-off between controller robustness and fine-tuning steps. (LLM, conf=0.84)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Training a policy to pivot a tool held in the robot's gripper in a simulator with randomized friction and action delays works in the real world and is robust to errors in estimation of the system parameters.

    “Similarly, in [2], the authors train a policy to pivot a tool held in the robot's gripper in a simulator with randomized friction and action delays, and find that it works in the real world and is robust to errors in estimation of the system parameters.”

    From Domain randomization for transferring deep neural networks from simulation to the real world (2017) · cites this paper as [2]

    Section 5, Experiments (p. 4), explicitly identifies the paper’s three evaluation domains as Pong variants, Atari games, and 3D maze games. That account of what the authors tested contradicts the attribution of a robot-gripper tool-pivoting experiment; the cited paper does not report the claimed real-world or parameter-estimation results. (LLM, conf=0.84)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Van Baar et al. [56] shows benefits of using DR and A3C algorithm [86] (on-policy) for learning the controller, with respect to not using DR.

    “Van Baar et al. [56] shows the bene ts of using DR and Asynchronous Actor Critic Agent (A3C) algorithm [86] (on-policy) for learning the controller, with respect to not using DR.”

    From Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning (2021) · cites this paper as [56]

    Section 5.1 (p. 4) says the paper uses A3C to learn a policy and value function, supporting the citation’s identification of the algorithm. It does not report Van Baar et al.’s domain-randomization comparison or establish that A3C was used to learn that study’s controller; it also credits the A3C framework to another work. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • The idea of adding new units to neural networks is present in the continual-learning literature.

    “The idea of adding new units to neural networks is present in the continual-learning literature92,116,117.”

    From Loss of plasticity in deep continual learning (2024) · cites this paper as [117]

  • EWC, Progressive Neural Networks, and PackNet prevent catastrophic forgetting.

    “EWC [9], Progressive Neural Networks [24], and PackNet [25] prevent catastrophic forgetting.”

    From From Goldfish to Elephant: Long-Term Memory for Large Language Models via Selective Storage and Hierarchical Compression (2026) · cites this paper as [24]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 22

  1. [1][1] Forest Agostinelli, Michael R Anderson, and Honglak Lee. Adaptive multi-column deep neural networks with application to robust image denoising. In Advances in Neural Information Processing Systems , 2013. -
  2. [3][3] M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research (JAIR) , 47:253-279, 2013. -
  3. [4][4] Yoshua Bengio. Deep learning of representations for unsupervised and transfer learning. In JMLR: Workshop on Unsupervised and Transfer Learning , 2012. -
  4. [5][5] Dan C. Ciresan, Ueli Meier, and Jürgen Schmidhuber. Multi-column deep neural networks for image classification. In Conf. on Computer Vision and Pattern Recognition , 2012. -
  5. [6][6] Scott E. Fahlman and Christian Lebiere. The cascade-correlation learning architecture. In Advances in Neural Information Processing Systems , 1990. -
  6. [7][7] G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science , 313(5786):504-507, July 2006. -
  7. [8][8] Goeff Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. CoRR , abs/1503.02531, 2015. - · arXiv
  8. [9][9] Yann LeCun, John S. Denker, and Sara A. Solla. Optimal brain damage. In Advances in Neural Information Processing Systems , 1990. -
  9. [10][10] Min Lin, Qiang Chen, and Shuicheng Yan. Network in network. In Proc. of Int'l Conference on Learning Representations (ICLR) , 2013. -
  10. [11][11] G. Mesnil, Y. Dauphin, X. Glorot, S. Rifai, Y. Bengio, I. Goodfellow, E. Lavoie, X. Muller, G. Desjardins, D. Warde-Farley, P. Vincent, A. Courville, and J. Bergstra. Unsupervised and transfer learning challenge: a deep learning approach. In JMLR W& CP: Proc. of the Unsupervised and Transfer Learning challenge and workshop , volume 27, 2012. -
  11. [12][12] V. Mnih, Kk Kavukcuoglu, D. Silver, A. Rusu, J. Veness, M. Bellemare, A. Graves, M. Riedmiller, A. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis. Human-level control through deep reinforcement learning. Nature , 518(7540):529-533, 2015. -
  12. [13][13] Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In Int'l Conf. on Machine Learning (ICML) , 2016. -
  13. [14][14] Emilio Parisotto, Lei Jimmy Ba, and Ruslan Salakhutdinov. Actor-mimic: Deep multitask and transfer reinforcement learning. In Proc. of Int'l Conference on Learning Representations (ICLR) , 2016. -
  14. [16][16] Artem Rozantsev, Mathieu Salzmann, and Pascal Fua. Beyond sharing weights for deep domain adaptation. CoRR , abs/1603.06432, 2016. - · arXiv
  15. [17][17] A. Rusu, S. Colmenarejo, Ç. Gülçehre, G. Desjardins, J. Kirkpatrick, R. Pascanu, V. Mnih, K. Kavukcuoglu, and R. Hadsell. Policy distillation. abs/1511.06295, 2016. - · arXiv
  16. [18][18] Paul Ruvolo and Eric Eaton. Ella: An efficient lifelong learning algorithm. In Proceedings of the 30th International Conference on Machine Learning (ICML-13) , June 2013. -
  17. [19][19] Daniel L. Silver, Qiang Yang, and Lianghao Li. Lifelong machine learning systems: Beyond learning algorithms. In AAAI Spring Symposium: Lifelong Machine Learning , 2013. -
  18. [20][20] Matthew E. Taylor and Peter Stone. An introduction to inter-task transfer for reinforcement learning. AI Magazine , 32(1):15-34, 2011. -
  19. [21][21] Alexander V. Terekhov, Guglielmo Montone, and J. Kevin O'Regan. Knowledge Transfer in Deep Block-Modular Neural Networks , pages 268-279. Springer International Publishing, Cham, 2015. -
  20. [22][22] C. Tessler, S. Givony, T. Zahavy, D. J. Mankowitz, and S. Mannor. A Deep Hierarchical Approach to Lifelong Learning in Minecraft. ArXiv e-prints , 2016. -
  21. [23][23] Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In Advances in Neural Information Processing Systems , pages 3320-3328, 2014. -
  22. [24][24] Guanyu Zhou, Kihyuk Sohn, and Honglak Lee. Online incremental feature learning with denoising autoencoders. In Proc. of Int'l Conf. on Artificial Intelligence and Statistics (AISTATS) , pages 1453-1461, 2012.

Cited by

Sorted by citations

Authors
Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., & Hadsell, R.
Affiliations
Google DeepMind (United Kingdom) (GB)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2016
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1606.04671
arXiv
1606.04671
OpenAlex
W4319988532
Cited by
1,307
References
0
Citations by year
2026: 6 · 2025: 98 · 2024: 248 · 2023: 303 · 2022: 220 · 2021: 183 · 2020: 101 · 2019: 81 · 2018: 44 · 2017: 22 · 2016: 1
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Keywords
progressive neural networks, catastrophic forgetting, sequence learning, lateral connections, reinforcement learning, Atari games, sensitivity analysis, policy learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review