The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
A novel DDPG method with prioritized experience replay
Abstract
Recently, a state-of-the-art algorithm, called deep deterministic policy gradient (DDPG), has achieved good performance in many continuous control tasks in the MuJoCo simulator. To further improve the efficiency of the experience replay mechanism in DDPG and thus speeding up the training process, in this paper, a prioritized experience replay method is proposed for the DDPG algorithm, where prioritized sampling is adopted instead of uniform sampling. The proposed DDPG with prioritized experience replay is tested with an inverted pendulum task via OpenAI Gym. The experimental results show that DDPG with prioritized experience replay can reduce the training time and improve the stability of the training process, and is less sensitive to the changes of some hyperparameters such as the size of replay buffer, minibatch and the updating rate of the target network.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767
On the Theory of the Brownian Motion
Uhlenbeck, G. E., & Ornstein, L. S. (1930) · Physical Review · doi:10.1103/physrev.36.823 · Cited by 4,482
Deterministic Policy Gradient Algorithms
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014) · HAL · Cited by 3,275
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
Reinforcement learning for robots using neural networks
Lin, L.-J. (1992) · Defense Technical Information Center (DTIC) · Cited by 1,042
Prioritized sweeping: Reinforcement learning with less data and less time
Moore, A. W., & Atkeson, C. G. (1993) · Machine Learning · doi:10.1007/bf00993104 · Cited by 823
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016) · arXiv · doi:10.48550/arxiv.1606.01540 · Cited by 582 · cited 6×
“We will focus on these problems in our future work and apply the proposed algorithm to more difficult tasks like hopper and walker [17].”
MethodInfluentialDopaminergic neurons promote hippocampal reactivation and spatial memory persistence
McNamara, C. G., Tejero-Cantero, Á., Trouche, S., Campo-Urriza, N., & Dupret, D. (2014) · Nature Neuroscience · doi:10.1038/nn.3843 · Cited by 486
Quantum Reinforcement Learning
Dong, D., Chen, C., Li, H., & Tarn, T.-J. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2008.925743 · Cited by 410
Self-citationRewarded Outcomes Enhance Reactivation of Experience in the Hippocampus
Singer, A. C., & Frank, L. M. (2009) · Neuron · doi:10.1016/j.neuron.2009.11.016 · Cited by 385
Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning
Guo, X., Singh, S. P., Lee, H., Lewis, R. L., & Wang, X. (2014) · neural information processing systems · Cited by 367
Learning Continuous Control Policies by Stochastic Value Gradients
Heess, N., Wayne, G., Silver, D., Lillicrap, T., Tassa, Y., & Erez, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.09142 · Cited by 328
Simulation tools for model-based robotics: Comparison of Bullet, Havok, MuJoCo, ODE and PhysX
Erez, T., Tassa, Y., & Todorov, E. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7139807 · Cited by 312
Experience Replay for Real-Time Reinforcement Learning Control
Adam, S., Busoniu, L., & Babuska, R. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2011.2106494 · Cited by 255
Weighted importance sampling for off-policy learning with linear function approximation
Mahmood, A. R., van Hasselt, H. P., & Sutton, R. S. (2014) · neural information processing systems · Cited by 181
Fidelity-Based Probabilistic Q-Learning for Control of Quantum Systems
Chen, C., Dong, D., Li, H., Chu, J., & Tarn, T. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2283574 · Cited by 154
Self-citationQuantum robot: structure, algorithms and applications
Dong, D., Chen, C., Zhang, C., & Chen, Z. (2006) · Robotica · doi:10.1017/s0263574705002596 · Cited by 69
Self-citationQuantum Robot: Structure, Algorithms and Applications
Dong, D. Y., Chen, C. L., Zhang, C. B., et al. (2005) · CERN Document Server (European Organization for Nuclear Research) · Cited by 63
Self-citationQuantum Reinforcement Learning
Dong, D., Chen, C., & Chen, Z. (2005) · Lecture notes in computer science · doi:10.1007/11539117_97 · Cited by 44
Self-citation
Cited by
Ganesh, A. H., & Xu, B. (2022) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2021.111833 · Cited by 285
Li, Y., Wang, R., & Yang, Z. (2022) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2021.3105529 · Cited by 269
Target tracking control of UAV through deep reinforcement learning
Ma, B., Liu, Z., Zhao, W., Yuan, J., Long, H., Wang, X., & Yuan, Z. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3249900 · Cited by 91
Yuan, X., Hu, S., Ni, W., Liu, R. P., & Wang, X. (2023) · IEEE Transactions on Communications · doi:10.1109/tcomm.2023.3238062 · Cited by 33
Metadata
- Authors
- Hou, Y., Liu, L., Wei, Q., Xu, X., & Chen, C.
- Venue
- 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 316-321
- ISSN
- 1062-922X (ISSN-L)
- OpenAlex
- W2775482448
- Cited by
- 254
- References
- 29
- FWCI
- 3.43
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.