Model-Free reinforcement learning with continuous action in practice
Open accessThe DOI matches its Crossref record; 18 of the 19 references it cites were found in a registry or an index.
Abstract
Reinforcement learning methods are often considered as a potential solution to enable a robot to adapt to changes in real time to an unpredictable environment. However, with continuous action, only a few existing algorithms are practical for real-time learning. In such a setting, most effective methods have used a parameterized policy structure, often with a separate parameterized value function. The goal of this paper is to assess such actor-critic methods to form a fully specified practical algorithm. Our specific contributions include 1) developing the extension of existing incremental policy-gradient algorithms to use eligibility traces, 2) an empirical comparison of the resulting algorithms using continuous actions, 3) the evaluation of a gradient-scaling technique that can significantly improve performance. Finally, we apply our actor-critic algorithm to learn on a robotic platform with a fast sensorimotor cycle (10ms). Overall, these results constitute an important step towards practical real-time learning control with continuous action.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Self-citationLearning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Self-citation(1999) · Cited by 5,147
Self-citationSimple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Reinforcement Learning in Continuous Time and Space
Doya, K. (2000) · Neural Computation · doi:10.1162/089976600300015961 · Cited by 960
Peters, J., & Schaal, S. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.11.026 · Cited by 738
Autonomous Helicopter Aerobatics through Apprenticeship Learning
Abbeel, P., Coates, A., & Ng, A. Y. (2010) · The International Journal of Robotics Research · doi:10.1177/0278364910371999 · Cited by 584
Policy gradient reinforcement learning for fast quadrupedal locomotion
Kohl, N., & Stone, P. (2004) · IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004 · doi:10.1109/robot.2004.1307456 · Cited by 564
Natural actor–critic algorithms
Bhatnagar, S., Sutton, R. S., Ghavamzadeh, M., & Lee, M. (2009) · Automatica · doi:10.1016/j.automatica.2009.07.008 · Cited by 559
Self-citationStochastic policy gradient reinforcement learning on a simple 3D biped
Tedrake, R., Zhang, T. W., & Seung, H. S. (2005) · 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566) · doi:10.1109/iros.2004.1389841 · Cited by 264
Cited by
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Deterministic Policy Gradient Algorithms
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014) · HAL · Cited by 3,275
Self-citationSurvey on computational-intelligence-based UAV path planning
Zhao, Y., Zheng, Z., & Liu, Y. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.05.033 · Cited by 418
Lian, R., Peng, J., Wu, Y., Tan, H., & Zhang, H. (2020) · Energy · doi:10.1016/j.energy.2020.117297 · Cited by 357
Traffic light control using deep policy‐gradient and value‐function‐based reinforcement learning
Mousavi, S. S., Schukat, M., & Howley, E. (2017) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2017.0153 · Cited by 354
A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers
Wang, P., Chan, C.-Y., & de La Fortelle, A. (2018) · 2018 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2018.8500556 · Cited by 299
Günther, J., Pilarski, P. M., Helfrich, G., Shen, H., & Diepold, K. (2014) · Procedia Technology · doi:10.1016/j.protcy.2014.09.007 · Cited by 303
Self-citationDeep Learning for Video Game Playing
Justesen, N., Bontrager, P., Togelius, J., & Risi, S. (2020) · IEEE Transactions on Games · doi:10.1109/tg.2019.2896986 · Cited by 257
Reinforcement learning in sustainable energy and electric systems: a survey
Yang, T., Zhao, L., Li, W., & Zomaya, A. Y. (2020) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2020.03.001 · Cited by 234
Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation
Wang, L., Zhang, W., He, X., & Zha, H. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219961 · Cited by 223
Darla Improving zero-shot transfer in reinforcement learning
Higgins, I., Pal, A., Rusu, A. A., Matthey, L., Burgess, C. P., Pritzel, A., Botvinick, M., Blundell, C., & Lerchner, A. (2017) · arXiv · doi:10.48550/arxiv.1707.08475 · Cited by 219
Duan, J., Eben Li, S., Guan, Y., Sun, Q., & Cheng, B. (2020) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2019.0317 · Cited by 189
Adaptive low-level control of autonomous underwater vehicles using deep reinforcement learning
Carlucho, I., De Paula, M., Wang, S., Petillot, Y., & Acosta, G. G. (2018) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.05.016 · Cited by 186
Günther, J., Pilarski, P. M., Helfrich, G., Shen, H., & Diepold, K. (2016) · Mechatronics · doi:10.1016/j.mechatronics.2015.09.004 · Cited by 176
Self-citationReal-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising
Jin, J., Song, C., Li, H., Gai, K., Wang, J., & Zhang, W. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3272021 · Cited by 160
Vignon, C., Rabault, J., & Vinuesa, R. (2023) · Physics of Fluids · doi:10.1063/5.0143913 · Cited by 161
Reward Augmented Maximum Likelihood for Neural Structured Prediction
Norouzi, M., Bengio, S., Chen, Z., Jaitly, N., Schuster, M., Wu, Y., & Schuurmans, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.00150 · Cited by 77
Intelligent resource allocation management for vehicles network: An A3C learning approach
Chen, M., Wang, T., Ota, K., Dong, M., Zhao, M., & Liu, A. (2020) · Computer Communications · doi:10.1016/j.comcom.2019.12.054 · Cited by 129
Deep Reinforcement Learning-Based Spectrum Allocation in Integrated Access and Backhaul Networks
Lei, W., Ye, Y., & Xiao, M. (2020) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2020.2992628 · Cited by 134
Reinforcement Learning–Based Energy Management Strategy for a Hybrid Electric Tracked Vehicle
Liu, T., Zou, Y., Liu, D., & Sun, F. (2015) · Energies · doi:10.3390/en8077243 · Cited by 107
Survey of machine learning techniques in spacecraft control design
Широбоков, М. Г., Trofimov, S. P., & Ovchinnikov, M. Y. (2021) · Acta Astronautica · doi:10.1016/j.actaastro.2021.05.018 · Cited by 117
Ma, D., Zhou, B., Song, X., & Dai, H. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3107258 · Cited by 104
A survey on visual navigation for artificial agents with deep reinforcement learning
Zeng, F., Wang, C., & Ge, S. S. (2020) · IEEE Access · doi:10.1109/access.2020.3011438 · Cited by 98
Kim, M., Han, D.-K., Park, J., & Kim, J. (2020) · Applied Sciences · doi:10.3390/app10020575 · Cited by 103
Adaptive Regulation of Motor Variability
Dhawale, A. K., Miyamoto, Y. R., Smith, M. A., & Ölveczky, B. P. (2019) · Current Biology · doi:10.1016/j.cub.2019.08.052 · Cited by 97
Deep Reinforcement Learning as a Job Shop Scheduling Solver: A Literature Review
Cunha, B., Madureira, A. M., Fonseca, B., & Coelho, D. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-14347-3_34 · Cited by 115
Machine Learning: Models, Challenges, and Research Directions
Khoei, T. T., & Kaabouch, N. (2023) · Future Internet · doi:10.3390/fi15100332 · Cited by 121
Deep reinforcement learning approach for MPPT control of partially shaded PV systems in Smart Grids
Ávila, L. O., Paula, M. D., Trimboli, M., & Carlucho, I. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106711 · Cited by 97
Bias in Natural Actor-Critic Algorithms
Thomas, P. S. (2014) · Cited by 93
Reinforcement Mechanism Design for e-commerce
Cai, Q., Filos-Ratsikas, A., Tang, P., & Zhang, Y. (2018) · International World Wide Web Conference (WWW) · doi:10.1145/3178876.3186039 · Cited by 84
Yang, S., Yang, B., Wong, H., & Kang, Z. (2019) · Knowledge-Based Systems · doi:10.1016/j.knosys.2019.07.026 · Cited by 78
Learning quadrotor dynamics for precise, safe, and agile flight control
Saviolo, A., & Loianno, G. (2023) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2023.03.009 · Cited by 98
Plume Tracing via Model-Free Reinforcement Learning Method
Hu, H., Song, S., & Chen, C. L. P. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2885374 · Cited by 77
Multi-timescale nexting in a reinforcement learning robot
Modayil, J., White, A., & Sutton, R. S. (2014) · Adaptive Behavior · doi:10.1177/1059712313511648 · Cited by 90
Self-citationSetting up a Reinforcement Learning Task with a Real-World Robot
Mahmood, A. R., Korenkevych, D., Komer, B., & Bergstra, J. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593894 · Cited by 72
The State of AI-Empowered Backscatter Communications: A Comprehensive Survey
Xu, F., Hussain, T., Ahmed, M., Ali, K., Mirza, M. A., Khan, W. U., Ihsan, A., & Han, Z. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3299210 · Cited by 76
UAV Autonomous Tracking and Landing Based on Deep Reinforcement Learning Strategy
Xie, J., Peng, X., Wang, H., Niu, W., & Zheng, X. (2020) · Sensors · doi:10.3390/s20195630 · Cited by 59
Wang, K., Wang, H., Wang, H., Feng, J., Li, Y., Yang, J., & Chen, Z. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.121186 · Cited by 59
Reinforcement Learning for Autonomous Process Control in Industry 4.0: Advantages and Challenges
Nievas, N., Pagès‐Bernaus, A., Bonada, F., Echeverria, L., & Domingo, X. (2024) · Applied Artificial Intelligence · doi:10.1080/08839514.2024.2383101 · Cited by 64
Parvaresh, N., & Kantarcı, B. (2023) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2023.3251297 · Cited by 61
Multi-Task Recommendations with Reinforcement Learning
Liu, Z., Tian, J., Cai, Q., Zhao, X., Gao, J., Liu, S., Chen, D., He, T., Zheng, D., Jiang, P., et al. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583467 · Cited by 73
Policy Gradient and Actor–Critic Learning in Continuous Time and Space: Theory and AlgorithmsPreprint
Jia, Y., & Zhou, X. Y. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.3969101 · Cited by 156
Reinforcement learning for systems pharmacology-oriented and personalized drug design
Tan, R. K., Liu, Y., & Xie, L. (2022) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2022.2072288 · Cited by 60
Learning to swim in potential flow
Jiao, Y., Ling, F., Heydari, S., Heess, N., Merel, J., & Kanso, E. (2021) · Physical Review Fluids · doi:10.1103/physrevfluids.6.050505 · Cited by 47
Deep reinforcement learning-based control of chemo-drug dose in cancer treatment
Mashayekhi, H., Nazari, M., Jafarinejad, F., & Meskin, N. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107884 · Cited by 58
Value-free reinforcement learning: policy optimization as a minimal model of operant behavior
Bennett, D., Niv, Y., & Langdon, A. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.04.020 · Cited by 55
Xu, J., Huang, F., Wu, D., Cui, Y., Yan, Z., & Zhang, K. (2021) · Ocean Engineering · doi:10.1016/j.oceaneng.2021.109794 · Cited by 46
Shen, S., Shen, G., Dai, Z., Zhang, K., Kong, X., & Li, J. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3418488 · Cited by 49
Deep reinforcement learning based conflict detection and resolution in air traffic control
Wang, Z., Li, H., Wang, J., & Shen, F. C. (2019) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2018.5357 · Cited by 46
A Reinforcement Learning based Path Planning Approach in 3D Environment
Kulathunga, G. (2022) · Procedia Computer Science · doi:10.1016/j.procs.2022.10.217 · Cited by 48
Cooperative traffic signal control through a counterfactual multi-agent deep actor critic approach
Song, X., Zhou, B., & Ma, D. (2024) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2024.104528 · Cited by 53
MinAtar: An Atari-Inspired Testbed for Thorough and Reproducible Reinforcement Learning ExperimentsPreprint
Young, K., & Tian, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03176 · Cited by 180
Generative Adversarial Networks for De Novo Molecular Design
Lee, Y. J., Kahng, H., & Kim, S. B. (2021) · Molecular Informatics · doi:10.1002/minf.202100045 · Cited by 44
Reinforcement learning for wind-farm flow control: Current state and future actions
Abkar, M., Zehtabiyan-Rezaie, N., & Iosifidis, A. (2023) · Theoretical and Applied Mechanics Letters · doi:10.1016/j.taml.2023.100475 · Cited by 46
Vasan, G., & Pilarski, P. M. (2017) · International Conference on Rehabilitation Robotics · doi:10.1109/icorr.2017.8009453 · Cited by 40
Self-citationRodoshi, R. T., Song, Y., & Choi, W. (2021) · IEEE Access · doi:10.1109/access.2021.3128516 · Cited by 43
Proximal Reinforcement Learning: A New Theory of Sequential Decision Making in Primal-Dual SpacesPreprint
Mahadevan, S., Liu, B., Thomas, P. S., Dabney, W., Giguere, S., Jacek, N., Gemp, I., & Liu, J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.6757 · Cited by 62
Towards a common implementation of reinforcement learning for multiple robotic tasks
Martínez-Tenor, Á., Fernández‐Madrigal, J., Cruz‐Martín, A., & González-Jiménez, J. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.11.011 · Cited by 36
Deep Reinforcement Learning in Human Activity Recognition: A Survey and Outlook
Nikpour, B., Sinodinos, D., & Armanfard, N. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3360990 · Cited by 50
Soft Policy Gradient Method for Maximum Entropy Deep Reinforcement Learning
Shi, W., Song, S., & Wu, C. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/475 · Cited by 56
MPPT for PV systems using deep reinforcement learning algorithms
Ávila, L. O., Paula, M. D., Carlucho, I., & Reinoso, C. R. S. (2019) · IEEE Latin America Transactions · doi:10.1109/tla.2019.9011547 · Cited by 35
Real-time prediction learning for the simultaneous actuation of multiple prosthetic joints
Pilarski, P. M., Dick, T., & Sutton, R. S. (2013) · International Conference on Rehabilitation Robotics · doi:10.1109/icorr.2013.6650435 · Cited by 39
Self-citationOptimizing Attention for Sequence Modeling via Reinforcement Learning
Fei, H., Zhang, Y., Ren, Y., & Ji, D. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3053633 · Cited by 33
The Assistive Multi-Armed Bandit
Chan, L., Hadfield-Menell, D., Srinivasa, S. S., & Dragan, A. D. (2019) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2019.8673234 · Cited by 56
Policy Analysis of Adaptive Traffic Signal Control Using Reinforcement Learning
Genders, W., & Razavi, S. (2019) · Journal of Computing in Civil Engineering · doi:10.1061/(asce)cp.1943-5487.0000859 · Cited by 35
Foukas, X., Marina, M. K., & Kontovasilis, K. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2927067 · Cited by 29
Human-in-the-loop reinforcement learning
Liang, H., Lu, Y., Cheng, H., Tu, W., & Xu, M. (2017) · Chinese Automation Congress · doi:10.1109/cac.2017.8243575 · Cited by 36
Rezaei, M., & Nezamabadi–pour, H. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11066-w · Cited by 42
ASPW-DRL: assembly sequence planning for workpieces via a deep reinforcement learning approach
Zhao, M., Guo, X., Zhang, X., Fang, Y., & Ou, Y. (2019) · Assembly Automation · doi:10.1108/aa-11-2018-0211 · Cited by 28
A Robust Approach for Continuous Interactive Actor-Critic Algorithms
Millán-Arias, C., Fernandes, B., Cruz, F., Dazeley, R., & Fernandes, S. M. M. (2021) · IEEE Access · doi:10.1109/access.2021.3099071 · Cited by 26
Robustness of quantum-enhanced adaptive phase estimation
Palittapongarnpim, P., & Sanders, B. C. (2019) · Physical Review A · doi:10.1103/physreva.100.012106 · Cited by 27
OFFER: Off-Environment Reinforcement Learning
Ciosek, K., & Whiteson, S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10810 · Cited by 49
Sequential Recommendation for Optimizing Both Immediate Feedback and Long-term Retention
Liu, Z., Liu, S., Zhang, Z., Cai, Q., Zhao, X., Zhao, K., Hu, L., Jiang, P., & Gai, K. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657829 · Cited by 46
Feng, R., Khan, W., Faheem, M., Wang, J., Subhan, F. E., & Bhutta, M. S. (2025) · IEEE Access · doi:10.1109/access.2025.3527047 · Cited by 31
Sensing flow gradients is necessary for learning autonomous underwater navigation
Jiao, Y., Hang, H., Merel, J., & Kanso, E. (2025) · Nature Communications · doi:10.1038/s41467-025-58125-6 · Cited by 23
Improved DDPG based on enhancing decision evaluation for path planning in high-density environments
Xue, J., He, M., Chen, J., Dong, B., & Zheng, Y. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127378 · Cited by 27
Model-Free Intelligent Diabetes Management Using Machine Learning
Bastani, M. (2014) · ERA: Education and Research Archive (University of Alberta) · doi:10.7939/r34q7qx0s · Cited by 30
Experimental performance of graph neural networks on random instances of max-cut
Yao, W., Bandeira, A. S., & Villar, S. (2019) · Wavelets and Sparsity · doi:10.1117/12.2529608 · Cited by 22
Model-Free Reinforcement Learning for Financial Portfolios: A Brief SurveyPreprint
Sato, Y. (2019) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.1904.04973 · Cited by 46
Actor-Critic-Based Resource Allocation for Multi-Modal Optical Networks
Yan, B., Zhao, Y., Li, Y., Yu, X., Zhang, J., Wang, Y., Yan, L., & Rahman, S. (2018) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/glocomw.2018.8644190 · Cited by 21
Chen, S.-W., Yuan, Q., Li, J.-M., He, H.-S., Li, S., Jiang, X.-F., & Yang, J. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3417241 · Cited by 18
Bussieweke, F., Mula, J., & Bolarín, F. C. (2024) · International Journal of Production Research · doi:10.1080/00207543.2024.2383293 · Cited by 30
Das, A., Ni, Z., & Zhong, X. (2023) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2023.109210 · Cited by 25
Value-Approximation based Deep Reinforcement Learning Techniques: An Overview
Sewak, M., Sahay, S. K., & Rathore, H. (2020) · International Conference on Computing, Communication and Automation · doi:10.1109/iccca49541.2020.9250787 · Cited by 20
ADAPTIVE STEP-SIZES FOR REINFORCEMENT LEARNING
Dabney, W. (2014) · Scholarworks (University of Massachusetts Amherst) · doi:10.7275/6053347.0 · Cited by 20
An Adaptive Threshold for the Canny Edge With Actor-Critic Algorithm
Choi, K.-H., & Ha, J.-E. (2023) · IEEE Access · doi:10.1109/access.2023.3291593 · Cited by 19
Muesli: Combining Improvements in Policy OptimizationPreprint
Hessel, M., Danihelka, I., Viola, F., Guez, A., Schmitt, S., Sifre, L., Weber, T., Silver, D., & Hasselt, H. V. (2021) · International Conference on Machine Learning · doi:10.48550/arxiv.2104.06159 · Cited by 70
Learning to schedule heuristics for the simultaneous stochastic optimization of mining complexes
Yaakoubi, Y., & Dimitrakopoulos, R. G. (2023) · Computers & Operations Research · doi:10.1016/j.cor.2023.106349 · Cited by 22
Projected Natural Actor-Critic
Thomas, P. S., Dabney, W., Giguere, S., & Mahadevan, S. (2013) · neural information processing systems · Cited by 18
Li, L., Li, D., Song, T., & Xu, X. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2808203 · Cited by 17
A survey on applications of reinforcement learning in spatial resource allocation
Zhang, D., Wang, M.-Y., Mango, J., Li, X., & Xu, X. (2024) · Computational Urban Science · doi:10.1007/s43762-024-00127-z · Cited by 12
GeNGA: A Generalization of Natural Gradient Ascent with Positive and Negative Convergence Results
Thomas, P. S. (2014) · International Conference on Machine Learning · Cited by 13
Average-reward model-free reinforcement learning: a systematic review and literature mapping
Dewanto, V., Dunn, G., Eshragh, A., Gallagher, M. R., & Roosta, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08920 · Cited by 47
Traffic Engineering in Software-defined Networks using Reinforcement Learning: A Review
Dake, D. K., Gadze, J. D., Klogo, G. S., & Nunoo‐Mensah, H. (2021) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2021.0120541 · Cited by 20
Reinforcement Learning for Generalized Parameter Optimization in Elastic Optical Networks
Koch, R., Kühl, S., Morais, R. M., Spinnler, B., Schairer, W., Sommerkorn-Krombholz, B., & Pachnicke, S. M. (2021) · Journal of Lightwave Technology · doi:10.1109/jlt.2021.3123271 · Cited by 15
Policy-Approximation Based Deep Reinforcement Learning Techniques: An Overview
Sewak, M., Sahay, S. K., & Rathore, H. (2021) · Lecture notes in networks and systems · doi:10.1007/978-981-16-0739-4_47 · Cited by 15
Zhong, S., Liu, Q., Zhang, Z., & Fu, Q. (2018) · Frontiers of Computer Science · doi:10.1007/s11704-017-6222-6 · Cited by 14
Young, K., Wang, B., & Taylor, M. E. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/581 · Cited by 29
Proximal Parameter Distribution Optimization
Wang, X., Li, T., & Cheng, Y. (2019) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2931946 · Cited by 14
Mueangprasert, M., Chermprayong, P., & Boonlong, K. (2023) · International Symposium on Instrumentation, Control, Artificial Intelligence, and Robotics (ICA-SYMP) · doi:10.1109/ica-symp56348.2023.10044940 · Cited by 13
Metadata
- Authors
- Degris, T., Pilarski, P. M., & Sutton, R. S.
- Affiliations
- Institut national de recherche en sciences et technologies du numérique (FR)
- University of Alberta (CA)
- Venue
- 2012 American Control Conference (ACC)
- Publisher
- IEEE
- Published
- 2012-06
- Pages
- 2177-2182
- ISSN
- 0743-1619 (ISSN-L)
- OpenAlex
- W2024390895
- Cited by
- 239
- References
- 19
- Citations by year
- 2026: 4 · 2025: 12 · 2024: 26 · 2023: 41 · 2022: 14 · 2021: 32 · 2020: 28 · 2019: 34 · 2018: 20 · 2017: 10 · 2016: 3 · 2015: 3 · 2014: 10 · 2013: 2
- FWCI
- 3.68
- Type
- conference paper
- Language
- en
- Open access
- green
- inria.hal.science
- Licence
- other-oa
- Locations
- Proceedings of the ... American Control Conference/Proceedings of the American Control Conference (publishedVersion)
- HAL (submittedVersion, other-oa)
- Funders
- Alberta Innovates
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- model-free reinforcement learning, continuous actions, actor-critic, eligibility traces, gradient scaling, robot learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.