CoChat
Sign Up
PreprintCited by 9

The DOI matches its DataCite record; 9 of the 9 references it cites were found in a registry or an index.

Neural Episodic Control

Pritzel, Alexander, Uria, Benigno, Srinivasan, Sriram, Puigdomènech, Adrià, Vinyals, Oriol, Hassabis, Demis, Wierstra, Daan, Blundell, Charles·2017·arXiv
DOI 10.48550/arxiv.1703.01988arXiv 1703.01988
Open this paper in CoChat — free account

Abstract

Deep reinforcement learning methods attain super-human performance in a wide range of environments. Such methods are grossly inefficient, often taking orders of magnitudes more data than humans to achieve reasonable performance. We propose Neural Episodic Control: a deep reinforcement learning agent that is able to rapidly assimilate new experiences and act upon them. Our agent uses a semi-tabular representation of the value function: a buffer of past experience containing slowly changing state representations and rapidly updated estimates of the value function. We show across a wide range of environments that our agent learns significantly faster than other state-of-the-art, general purpose deep reinforcement learning agents.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1703.01988

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 9 references

Research works printed but not linked: 35

  1. Ba, Jimmy, Hinton, Geoffrey E, Mnih, Volodymyr, Leibo, Joel Z, and Ionescu, Catalin. Using fast weights to attend to the recent past. In Advances In Neural Information Processing Systems, pp. 4331-4339, 2016.
  2. Bakker, Bram, Zhumatiy, Viktor, Gruener, Gabriel, and Schmidhuber, Jürgen. A robot that reinforcement-learns to identify and memorize important previous observations. In Intelligent Robots and Systems, 2003.(IROS 2003). Proceedings. 2003 IEEE/RSJ International Conference on, volume 1, pp. 430-435. IEEE, 2003.
  3. Bellemare, M. G., Naddaf, Y., Veness, J., and Bowling, M. The arcade learning environment: An evaluation plat- form for general agents. Journal of Artificial Intelligence Research, 47:253-279, 06 2013.
  4. Bentley, Jon Louis. Multidimensional binary search trees used for associative searching. Commun. ACM, 18(9): 509-517, September 1975.
  5. Blundell, Charles, Uria, Benigno, Pritzel, Alexander, Li, Yazhe, Ruderman, Avraham, Leibo, Joel Z, Rae, Jack, Wierstra, Daan, and Hassabis, Demis. Model-free episodic control. arXiv preprint arXiv:1606.04460, 2016. · arXiv
  6. Duan, Yan, Schulman, John, Chen, Xi, Bartlett, Peter L, Sutskever, Ilya, and Abbeel, Pieter. Rl 2 : Fast reinforce- ment learning via slow reinforcement learning. arXiv preprint arXiv:1611.02779, 2016. · arXiv
  7. Fernando, Chrisantha, Banarse, Dylan, Blundell, Charles, Zwols, Yori, Ha, David, Rusu, Andrei A, Pritzel, Alexan- der, and Wierstra, Daan. Pathnet: Evolution channels gradient descent in super neural networks. arXiv preprint arXiv:1701.08734, 2017. · arXiv
  8. Gabel, Thomas and Riedmiller, Martin. Cbr for state value function approximation in reinforcement learning. In International Conference on Case-Based Reasoning, pp. 206-221. Springer, 2005.
  9. Harutyunyan, Anna, Bellemare, Marc G, Stepleton, Tom, and Munos, Rémi. Q (\ lambda) with off-policy cor- rections. In International Conference on Algorithmic Learning Theory, pp. 305-320. Springer, 2016.
  10. Hausknecht, Matthew and Stone, Peter. Deep recurrent q- learning for partially observable mdps. arXiv preprint arXiv:1507.06527, 2015. · arXiv
  11. He, Frank S, Liu, Yang, Schwing, Alexander G, and Peng, Jian. Learning to play in a day: Faster deep reinforce- ment learning by optimality tightening. arXiv preprint arXiv:1611.01606, 2016. · arXiv
  12. Hinton, Geoffrey E and Plaut, David C. Using fast weights to deblur old memories. In Proceedings of the ninth annual conference of the Cognitive Science Society, pp. 177-186, 1987.
  13. Lengyel, M. and Dayan, P. Hippocampal contributions to control: The third way. In NIPS, volume 20, pp. 889-896, 2007.
  14. Miller, Alexander, Fisch, Adam, Dodge, Jesse, Karimi, Amir-Hossein, Bordes, Antoine, and Weston, Jason. Key- value memory networks for directly reading documents. arXiv preprint arXiv:1606.03126, 2016. · arXiv
  15. Mnih, Volodymyr, Kavukcuoglu, Koray, Silver, David, Rusu, Andrei A, Veness, Joel, Bellemare, Marc G, Graves, Alex, Riedmiller, Martin, Fidjeland, Andreas K, Ostro- vski, Georg, et al. Human-level control through deep re- inforcement learning. Nature, 518(7540):529-533, 2015.
  16. Mnih, Volodymyr, Badia, Adria Puigdomenech, Mirza, Mehdi, Graves, Alex, Lillicrap, Timothy P, Harley, Tim, Silver, David, and Kavukcuoglu, Koray. Asynchronous methods for deep reinforcement learning. In International Conference on Machine Learning, 2016.
  17. Munos, Remi and Moore, Andrew W. Barycentric interpola- tors for continuous space and time reinforcement learning. In NIPS, pp. 1024-1030, 1998.
  18. Munos, Rémi, Stepleton, Tom, Harutyunyan, Anna, and Bellemare, Marc. Safe and efficient off-policy reinforce- ment learning. In Advances in Neural Information Pro- cessing Systems, pp. 1046-1054, 2016.
  19. Oh, Junhyuk, Guo, Xiaoxiao, Lee, Honglak, Lewis, Richard L, and Singh, Satinder. Action-conditional video prediction using deep networks in atari games. In Ad- vances in Neural Information Processing Systems, pp. 2845-2853, 2015.
  20. Oh, Junhyuk, Chockalingam, Valliappa, Lee, Honglak, et al. Control of memory, active perception, and action in minecraft. In Proceedings of The 33rd International Conference on Machine Learning, pp. 2790-2799, 2016.
  21. Osband, Ian, Blundell, Charles, Pritzel, Alexander, and Van Roy, Benjamin. Deep exploration via bootstrapped dqn. In Advances In Neural Information Processing Sys- tems, pp. 4026-4034, 2016.
  22. Peng, Jing and Williams, Ronald J. Incremental multi-step q-learning. Machine learning, 22(1-3):283-290, 1996.
  23. Rezende, Danilo Jimenez, Mohamed, Shakir, and Wierstra, Daan. Stochastic backpropagation and approximate infer- ence in deep generative models. In Proceedings of The 31st International Conference on Machine Learning, pp. 1278-1286, 2014.
  24. Rusu, Andrei A, Rabinowitz, Neil C, Desjardins, Guillaume, Soyer, Hubert, Kirkpatrick, James, Kavukcuoglu, Koray, Pascanu, Razvan, and Hadsell, Raia. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016. · arXiv
  25. Santamaría, Juan C, Sutton, Richard S, and Ram, Ashwin. Experiments with reinforcement learning in problems with continuous state and action spaces. Adaptive behav- ior, 6(2):163-217, 1997.
  26. Schaul, Tom, Quan, John, Antonoglou, Ioannis, and Sil- ver, David. Prioritized experience replay. CoRR, abs/1511.05952, 2015a. · arXiv
  27. Schaul, Tom, Quan, John, Antonoglou, Ioannis, and Silver, David. Prioritized experience replay. arXiv preprint arXiv:1511.05952, 2015b. · arXiv
  28. Sukhbaatar, Sainbayar, Weston, Jason, Fergus, Rob, et al. End-to-end memory networks. In Advances in neural information processing systems, pp. 2440-2448, 2015.
  29. Sutton, Richard S and Barto, Andrew G. Reinforcement learning: An introduction. MIT press, 1998.
  30. van Hasselt, H., Guez, A., Hessel, M., and Silver, D. Learn- ing functions across many orders of magnitudes. ArXiv e-prints, February 2016.
  31. Vezhnevets, Alexander, Mnih, Volodymyr, Osindero, Si- mon, Graves, Alex, Vinyals, Oriol, Agapiou, John, et al. Strategic attentive writer for learning macro-actions. In Advances in Neural Information Processing Systems, pp. 3486-3494, 2016.
  32. Vinyals, Oriol, Blundell, Charles, Lillicrap, Tim, Wierstra, Daan, et al. Matching networks for one shot learning. In Advances in Neural Information Processing Systems, pp. 3630-3638, 2016.
  33. Wang, Jane X, Kurth-Nelson, Zeb, Tirumala, Dhruva, Soyer, Hubert, Leibo, Joel Z, Munos, Remi, Blundell, Charles, Kumaran, Dharshan, and Botvinick, Matt. Learning to reinforcement learn. arXiv preprint arXiv:1611.05763, 2016. · arXiv
  34. Watkins, Christopher JCH and Dayan, Peter. Q-learning. Machine learning, 8(3-4):279-292, 1992.
  35. Watkins, Christopher John Cornish Hellaby. Learning from delayed rewards. PhD thesis, University of Cambridge England, 1989.

Cited by

Sorted by citations

See all 9

Metadata

Authors
Pritzel, A., Uria, B., Srinivasan, S., Puigdomènech, A., Vinyals, O., Hassabis, D., Wierstra, D., & Blundell, C.
Venue
arXiv
Publisher
arXiv
Published
2017
DOI
10.48550/arxiv.1703.01988
arXiv
1703.01988
References
9
Type
Preprint
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Neural Episodic Control. https://app.cochat.ai/papers/10.48550/arxiv.1703.01988
The paper
Pritzel, A., Uria, B., Srinivasan, S., Puigdomènech, A., Vinyals, O., Hassabis, D., Wierstra, D., & Blundell, C. (2017). Neural Episodic Control. arXiv. https://doi.org/10.48550/arxiv.1703.01988