The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Relative Entropy Policy Search
Open accessAbstract
Policy search is a successful approach to reinforcement learning. However, policy improvements often result in the loss of information. Hence, it has been marred by premature convergence and implausible solutions. As first suggested in the context of covariant policy gradients, many of these problems may be addressed by constraining the information loss. In this paper, we continue this path of reasoning and suggest the Relative Entropy Policy Search (REPS) method. The resulting method differs significantly from previous policy gradient approaches and yields an exact update step. It can be shown to work well on typical reinforcement learning benchmark problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
(1999) · Cited by 5,147
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Baxter, L. A. (1995) · Technometrics · doi:10.1080/00401706.1995.10484354 · Cited by 2,415
(2020) · Machine Learning and Visual Perception · doi:10.1515/9783110595567-011 · Cited by 1,930
A Survey on Policy Search for Robotics
Deisenroth, M. P. (2011) · Foundations and Trends in Robotics · doi:10.1561/2300000021 · Cited by 798
Kakade, S. M. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 775
Peters, J., & Schaal, S. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.11.026 · Cited by 738
Self-citationThe Linear Programming Approach to Approximate Dynamic Programming
de Farias, D. P., & Van Roy, B. (2003) · Operations Research · doi:10.1287/opre.51.6.850.24925 · Cited by 687
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (2005) · Cited by 588
(1997) · An Introduction to Natural Computation · doi:10.7551/mitpress/3917.003.0017 · Cited by 492
Policy search for motor primitives in robotics
Kober, J., & Peters, J. (2011) · Machine Learning · doi:10.1007/s10994-010-5223-6 · Cited by 427
Self-citationPolicy Search for Motor Primitives in Robotics
Kober, J., & Peters, J. (2014) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-319-03194-1_4 · Cited by 220
Self-citationEfficient Reinforcement Learning using Gaussian Processes
Deisenroth, M. P. (2010) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.5445/ksp/1000019799 · Cited by 163
Bagnell, J. A., & Schneider, J. (2018) · Figshare · doi:10.1184/r1/6552458 · Cited by 148
Using Local Trajectory Optimizers to Speed Up Global Optimization in Dynamic Programming
Atkeson, C. G. (1993) · Cited by 78
Variational methods for Reinforcement Learning
Furmston, T., & Barber, D. (2010) · UCL Discovery (University College London) · Cited by 61
Learning from Scarce ExperiencePreprint
Peshkin, L., & Shelton, C. R. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0204043 · Cited by 61
Biases and Variance in Value Function Estimates
Mannor, S., Simester, D. I., Sun, P., & Tsitsiklis, J. N. (2004) · Cited by 4
Cited by
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
“The idea of constraining policy gradient updates was explored earlier through the use of “natural gradient” updates [46] and also by means of the Kullback-Leibler (KL) divergence [48, 88].”
A Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160
Self-citationDeep reinforcement learning for robotic manipulation with asynchronous off-policy updates
Gu, S., Holly, E., Lillicrap, T., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989385 · Cited by 1,424
Metadata
- Authors
- Peters, J., Mulling, K., & Altun, Y.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2010-07-05
- Volume
- 24
- Issue
- 1
- Pages
- 1607-1612
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W1499669280
- Cited by
- 455
- References
- 21
- FWCI
- 7.02
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.