Parameter Space Noise for Exploration
Open accessThe DOI matches its OpenAlex record; 32 of the 32 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
On the Theory of the Brownian Motion
Uhlenbeck, G. E., & Ornstein, L. S. (1930) · Physical Review · doi:10.1103/physrev.36.823 · Cited by 4,482
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution
Rechenberg, I. (1973) · Frommann-Holzboog eBooks · Cited by 2,440
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161 · cited 1×
“For discrete-action environments, we use the Arcade Learning Environment (ALE) [26] benchmark along with a standard DQN implementation.”
MethodCuriosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815 · cited 1×
“Although a plethora of methods have been proposed to tackle this challenge in high-dimensional and/or continuous-action MDPs, they often rely on complex additional structures such as counting tables [1], density modeling of the state space [2], learned dynamics models [3–5], or self-supervised curiosity [6].”
Benchmarking Deep Reinforcement Learning for Continuous Control
Duan, Y., Chen, X., Houthooft, R., Schulman, J., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1604.06778 · Cited by 1,114
Self-citationEvolution Strategies as a Scalable Alternative to Reinforcement Learning
Salimans, T., Ho, J., Chen, X., Sidor, S., & Sutskever, I. (2017) · arXiv · doi:10.48550/arxiv.1703.03864 · Cited by 1,027
Self-citationNear-Optimal Reinforcement Learning in Polynomial Time
Kearns, M., & Singh, S. (2002) · Machine Learning · doi:10.1023/a:1017984413808 · Cited by 792
Peters, J., & Schaal, S. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.11.026 · Cited by 738
Noisy Networks for Exploration
Fortunato, M., Azar, M. G., Piot, B., Menick, J., Osband, I., Graves, A., Mnih, V., Munos, R., Hassabis, D., Pietquin, O., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.10295 · Cited by 543
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303765208377 · Cited by 467
Ba, J. L., Kiros, J., & Hinton, G. E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.06450 · Cited by 440
Deep Exploration via Bootstrapped DQN
Osband, I., Blundell, C., Pritzel, A., & Roy, B. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04621 · Cited by 434
Count-Based Exploration with Neural Density Models
Ostrovski, G., Bellemare, M. G., Oord, A. V. D., & Munos, R. (2017) · arXiv · doi:10.48550/arxiv.1703.01310 · Cited by 377 · cited 1×
“…plethora of methods have been proposed to tackle this challenge in high-dimensional and/or continuous-action MDPs, they often rely on complex additional structures such as counting tables [1], density modeling of the state space [2], learned dynamics models [3–5], or self-supervised curiosity [6].”
BackgroundIncentivizing Exploration In Reinforcement Learning With Deep Predictive Models
Stadie, B. C., Levine, S., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.00814 · Cited by 280
Self-citationWierstra, D., Schaul, T., Peters, J., & Schmidhuber, J. (2008) · 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence) · doi:10.1109/cec.2008.4631255 · Cited by 246
Parameter-exploring policy gradients
Sehnke, F., Osendorfer, C., Rückstieß, T., Graves, A., Peters, J., & Schmidhuber, J. (2010) · Neural Networks · doi:10.1016/j.neunet.2009.12.004 · Cited by 236
Policy Search for Motor Primitives in Robotics
Kober, J., & Peters, J. (2014) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-319-03194-1_4 · Cited by 220
Exponential natural evolution strategies
Glasmachers, T., Schaul, T., Yi, S., Wierstra, D., & Schmidhuber, J. (2010) · Proceedings of the 12th annual conference on Genetic and evolutionary computation · doi:10.1145/1830483.1830557 · Cited by 198
Near-optimal Regret Bounds for Reinforcement Learning
Auer, P. L., Jaksch, T., & Ortner, R. (2008) · Cited by 197
High dimensions and heavy tails for natural evolution strategies
Schaul, T., Glasmachers, T., & Schmidhuber, J. (2011) · Annual Conference on Genetic and Evolutionary Computation · doi:10.1145/2001576.2001692 · Cited by 193
Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play
Sukhbaatar, S., Lin, Z., Kostrikov, I., Synnaeve, G., Szlam, A., & Fergus, R. (2017) · arXiv · doi:10.48550/arxiv.1703.05407 · Cited by 149 · cited 1×
“In the context of deep reinforcement learning, a large variety of techniques have been proposed to improve exploration [5, 3, 1, 7, 2, 36, 37].”
BackgroundSurprise-Based Intrinsic Motivation for Deep Reinforcement Learning
Achiam, J., & Sastry, S. (2017) · arXiv · doi:10.48550/arxiv.1703.01732 · Cited by 139 · cited 1×
“In this section, we evaluate whether parameter noise enables existing RL algorithms to learn on environments with very sparse rewards, where uncorrelated action noise generally fails [7, 4].”
BackgroundStochastic search using the natural gradient
Sun, Y., Wierstra, D., Schaul, T., & Schmidhuber, J. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553522 · Cited by 79
State-Dependent Exploration for Policy Gradient Methods
Rückstieß, T., Felder, M., & Schmidhuber, J. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87481-2_16 · Cited by 78
A Natural Evolution Strategy for Multi-objective Optimization
Glasmachers, T., Schaul, T., & Schmidhuber, J. (2010) · Parallel Problem Solving from Nature · doi:10.1007/978-3-642-15844-5_63 · Cited by 34
Cited by
Deep Reinforcement Learning That Matters
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., & Meger, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11694 · Cited by 1,685
Noisy Networks for Exploration
Fortunato, M., Azar, M. G., Piot, B., Menick, J., Osband, I., Graves, A., Mnih, V., Munos, R., Hassabis, D., Pietquin, O., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.10295 · Cited by 543
Exploration in deep reinforcement learning: A survey
Ladosz, P., Weng, L., Kim, M., & Oh, H. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.003 · Cited by 525
“noise is by comparing the decision made by the noisy and noiseless policy [156].”
BackgroundExploration by Random Network Distillation
Burda, Y., Edwards, H., Storkey, A., & Klimov, O. (2018) · arXiv · doi:10.48550/arxiv.1810.12894 · Cited by 527
“Other methods of exploration include adversarial self-play (Sukhbaatar et al., 2018), maximizing empowerment (Gregor et al., 2017), parameter noise (Plappert et al., 2017; Fortunato et al., 2017), identifying diverse policies (Eysenbach et al., 2018; Achiam et al., 2018), and using ensembles of…”
MethodSuch, F. P., Madhavan, V., Conti, E., Lehman, J., Stanley, K. O., & Clune, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.06567 · Cited by 525
Metadata
- Authors
- Plappert, M., Houthooft, R., Dhariwal, P., Sidor, S., Chen, R. Y., Chen, X., Asfour, T., Abbeel, P., & Andrychowicz, M.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2017-06-06
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1706.01905
- OpenAlex
- W2623491082
- Cited by
- 346
- References
- 32
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.