Simple random search provides a competitive approach to reinforcement learning
Open accessThe DOI matches its OpenAlex record; 17 of the 17 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 1×
“Such methods have successfully produced RL agents that surpass human players in video games and games such as Go [20, 33].”
MethodMastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · cited 7×
“For the performance of these methods we used values reported by Rajeswaran et al. [27], Salimans et al. [28], Schulman et al. [31], and Haarnoja et al. [10].”
BackgroundMethodResultInfluentialAsynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 2×
“Mnih et al. [21] showed that actor-critic methods, popular for variance reduction in policy gradient algorithms, can be asynchronously parallelized for fast training of policies for Atari video games and MuJoCo models.”
MethodSoft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594 · cited 10×
“Sensitivity to hyperparameters: It has been correctly noted in the literature that RL methods should not be sensitive to hyperparameter choices if one hopes to apply them in practice [10].”
BackgroundMethodResultInfluentialMuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
Deep Reinforcement Learning That Matters
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., & Meger, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11694 · Cited by 1,685
Cited by
AutoAugment: Learning Augmentation Strategies From Data
Cubuk, E. D., Zoph, B., Mané, D., Vasudevan, V., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00020 · Cited by 2,733
Deep reinforcement learning for autonomous driving: A survey
Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A. A. A., Yogamani, S., & Perez, P. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3054625 · Cited by 2,342
AutoAugment: Learning Augmentation Policies from Data
Cubuk, E. D., Zoph, B., Mane, D., Vasudevan, V., & Le, Q. V. (2018) · arXiv · doi:10.48550/arxiv.1805.09501 · Cited by 1,106
A better balance in metaheuristic algorithms: Does it exist?
Morales-Castañeda, B., Zaldívar, D., Cuevas, E., Fausto, F., & Rodríguez, A. (2020) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2020.100671 · Cited by 436
Reward Constrained Policy Optimization
Tessler, C., Mankowitz, D. J., & Mannor, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11074 · Cited by 237
“As our experiments show, the coefficient is not shared across domains, a coefficient which leads to a satisfying behavior on one domain may lead to catastrophic failure on the other (issues also seen in Leike et al. (2017) and Mania et al. (2018)).”
BackgroundHow Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experiments
Colas, C., Sigaud, O., & Oudeyer, P.-Y. (2018) · arXiv · Cited by 194
Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO
Engstrom, L., Ilyas, A., Santurkar, S., Tsipras, D., Janoos, F., Rudolph, L., & Madry, A. (2020) · arXiv · doi:10.48550/arxiv.2005.12729 · Cited by 146
“Rajeswaran et al. (2017) and Mania et al. (2018) also demonstrate that on many of the benchmark tasks, the performance of PPO and TRPO can be matched by fairly elementary randomized search approaches.”
BackgroundDeep Reinforcement Learning at the Edge of the Statistical Precipice
Agarwal, R., Schwarzer, M., Castro, P. S., Courville, A., & Bellemare, M. G. (2021) · arXiv · doi:10.48550/arxiv.2108.13264 · Cited by 142
Deep learning generalizes because the parameter-function map is biased towards simple functions
Valle-Pérez, G., Camargo, C. Q., & Louis, A. A. (2018) · arXiv · doi:10.48550/arxiv.1805.08522 · Cited by 105
Multi-Game Decision Transformers
Lee, K.-H., Nachum, O., Yang, M. (., Lee, L., Freeman, D., Guadarrama, S., Fischer, I., Xu, W., Jang, E., Michalewski, H., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2025 · Cited by 54
Observational Overfitting in Reinforcement Learning
Song, X., Jiang, Y., Tu, S., Du, Y., & Neyshabur, B. (2019) · arXiv · doi:10.48550/arxiv.1912.02975 · Cited by 48
“While there have been numerous works which avoid overparametrization on simplifying policies (Rajeswaran et al., 2017; Mania et al., 2018) or compactifying networks (Choromanski et al.”
BackgroundTian, H., Chen, S.-C., & Shyu, M.-L. (2020) · Information Systems Frontiers · doi:10.1007/s10796-020-10023-6 · Cited by 34
Post-Estimation Smoothing: A Simple Baseline for Learning with Side Information
Rolf, E., Jordan, M. I., & Recht, B. (2020) · arXiv · doi:10.48550/arxiv.2003.05955 · Cited by 3
Self-citation
Metadata
- Authors
- Mania, H., Guy, A., & Recht, B.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2018-03-19
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1803.07055
- OpenAlex
- W2789525339
- Cited by
- 148
- References
- 17
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.