VIME: Variational Information Maximizing Exploration
Open accessThe DOI matches its OpenAlex record; 36 of the 36 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 2×
“Examples include acting randomly using (cid:15) -greedy or Boltzmann exploration [7], and utilizing Gaussian noise on the controls in policy gradient methods [8].”
BackgroundMethodIntroduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Using confidence bounds for exploitation-exploration trade-offs
Auer, P. L. (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944941 · Cited by 2,266
Weight Uncertainty in Neural Networks
Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1505.05424 · Cited by 1,342
Benchmarking Deep Reinforcement Learning for Continuous Control
Duan, Y., Chen, X., Houthooft, R., Schulman, J., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1604.06778 · Cited by 1,114
Self-citationPractical Variational Inference for Neural Networks
Graves, A. (2011) · Cited by 1,078
Keeping the neural networks simple by minimizing the description length of the weights
Hinton, G. E., & van Camp, D. (1993) · Proceedings of the sixth annual conference on Computational learning theory - COLT '93 · doi:10.1145/168304.168306 · Cited by 994
Simple statistical gradient-following algorithms for connectionist reinforcement learning
Williams, R. J. (1992) · Machine Learning · doi:10.1023/a:1022672621406 · Cited by 817
What is intrinsic motivation? A typology of computational approaches
Oudeyer, P.-Y. (2007) · Frontiers in Neurorobotics · doi:10.3389/neuro.12.006.2007 · Cited by 811
Near-Optimal Reinforcement Learning in Polynomial Time
Kearns, M., & Singh, S. (2002) · Machine Learning · doi:10.1023/a:1017984413808 · Cited by 792
Near-optimal Regret Bounds for Reinforcement Learning
Jaksch, T., Ortner, R., & Auer, P. L. (2010) · Cited by 784
Formal theory of creativity, fun, and intrinsic motivation (1990–2010)
Schmidhuber, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2056368 · Cited by 747
Curious model-building control systems
Schmidhuber, J. (1991) · [Proceedings] 1991 IEEE International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1991.170605 · Cited by 541
Bayesian Surprise Attracts Human Attention
Itti, L., & Baldi, P. (2005) · neural information processing systems · Cited by 533
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303765208377 · Cited by 467
Deep Exploration via Bootstrapped DQN
Osband, I., Blundell, C., Pritzel, A., & Roy, B. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04621 · Cited by 434
Action-Conditional Video Prediction using Deep Networks in Atari Games
Oh, J., Guo, X., Lee, H., Lewis, R. L., & Singh, S. P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.08750 · Cited by 419
Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models
Stadie, B. C., Levine, S., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.00814 · Cited by 280
Self-citationVariational Information Maximisation for Intrinsically Motivated Reinforcement Learning
Mohamed, S., & Rezende, D. J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.08731 · Cited by 258
Policy Search for Motor Primitives in Robotics
Kober, J., & Peters, J. (2014) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-319-03194-1_4 · Cited by 220
Near-Bayesian exploration in polynomial time
Kolter, J. Z., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553441 · Cited by 218
Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning ProgressPreprint
Lopes, M. C., Lang, T., Toussaint, M., & Oudeyer, P. (2012) · Cited by 206
Reinforcement learning by reward-weighted regression for operational space control
Peters, J., & Schaal, S. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273590 · Cited by 187
An information-theoretic approach to curiosity-driven reinforcement learning
Still, S., & Precup, D. (2012) · Theory in Biosciences · doi:10.1007/s12064-011-0142-z · Cited by 165
Planning to be surprised: optimal bayesian exploration in dynamic environments
Sun, Y., Gomez, F., & Schmidhuber, J. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-22887-2_5 · Cited by 134
Exploration in metric state spaces
Kakade, S. M., Kearns, M. J., & Langford, J. C. (2003) · ScholarlyCommons (University of Pennsylvania) · Cited by 107
Intrinsically motivated model learning for developing curious robots
Hester, T., & Stone, P. (2015) · Artificial Intelligence · doi:10.1016/j.artint.2015.05.002 · Cited by 94
Learning and exploration in action-perception loops
Little, D. Y., & Sommer, F. T. (2013) · Frontiers in Neural Circuits · doi:10.3389/fncir.2013.00037 · Cited by 89
Generalization and Exploration via Randomized Value Functions
Osband, I., Roy, B. V., & Wen, Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.0635 · Cited by 75
Schmidhuber, J. (2007) · Springer eBooks · doi:10.1007/978-3-540-75488-6_3 · Cited by 66
Exploration from Demonstration for Interactive Reinforcement Learning
Subramanian, K. A., Isbell, C. L., & Thomaz, A. L. (2016) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/2936924.2936990 · Cited by 64
PAC Optimal Exploration in Continuous Space Markov Decision Processes
Pazis, J., & Parr, R. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8678 · Cited by 61
Ghazi-Zahedi, K., Martius, G., & Ay, N. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00801 · Cited by 12
Information Theoretically Aided Reinforcement Learning for Embodied AgentsPreprint
Montúfar, G., Ghazi-Zahedi, K., & Ay, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09735 · Cited by 12
Cited by
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
A Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 3,111
An Introduction to Variational Autoencoders
Kingma, D. P., & Welling, M. (2019) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000056 · Cited by 2,609
Curiosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815
Metadata
- Authors
- Houthooft, R., Chen, X., Duan, Y., Schulman, J., Turck, F. D., & Abbeel, P.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2016-05-31
- Volume
- 29
- Pages
- 1109-1117
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1605.09674
- OpenAlex
- W2417786368
- Cited by
- 472
- References
- 36
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.