Cited by 544Adaptive Dynamic Programming ControlReinforcement Learning in RoboticsAdvanced Control Systems Optimization

The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.

Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data

Lewis, F L, Vamvoudakis, K G·2011·IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)

DOI 10.1109/tsmcb.2010.2043839

Open this paper in CoChat — free account

Abstract

Approximate dynamic programming (ADP) is a class of reinforcement learning methods that have shown their importance in a variety of applications, including feedback control of dynamical systems. ADP generally requires full information about the system internal states, which is usually not available in practical situations. In this paper, we show how to implement ADP methods using only measured input/output data from the system. Linear dynamical systems with deterministic behavior are considered herein, which are systems of great interest in the control system community. In control system theory, these types of methods are referred to as output feedback (OPFB). The stochastic equivalent of the systems dealt with in this paper is a class of partially observable Markov decision processes. We develop both policy iteration and value iteration algorithms that converge to an optimal controller that requires only OPFB. It is shown that, similar to Q -learning, the new methods have the important advantage that knowledge of the system dynamics is not needed for the implementation of these learning algorithms or for the OPFB control. Only the order of the system, as well as an upper bound on its "observability index," must be known. The learned OPFB controller is in the form of a polynomial autoregressive moving-average controller that has equivalent performance with the optimal state variable feedback gain.

Integrity signals

Registry

Crossref · 10.1109/tsmcb.2010.2043839

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Learning from Delayed Rewards
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,674
Neuro-dynamic Programming
(2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,601
Optimal Control
Lewis, F. L. (1986) · Cited by 2,957
Self-citation
Beyond regression : new fools for prediction and analysis in the behavioral sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,739
Approximate Dynamic Programming
Powell, W. B. (2011) · Wiley series in probability and statistics · doi:10.1002/9781118029176 · Cited by 2,359
Kronecker products and matrix calculus in system theory
Brewer, J. (1978) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1978.1084534 · Cited by 1,920
Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,722
Self-citation
Reinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,591
Self-citation

Show all 48 references

Research works printed but not linked: 3

sutton, 1998, Reinforcement LearningAn Introduction
werbos, 1992, Handbook of Intelligent Control
1992, Handbook of Intelligent Control

Sorted by citations

Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers
Lewis, F. L., Vrabie, D., & Vamvoudakis, K. G. (2012) · IEEE Control Systems · doi:10.1109/mcs.2012.2214134 · Cited by 1,122
Self-citation
Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics
Jiang, Y., & Jiang, Z. (2012) · Automatica · doi:10.1016/j.automatica.2012.06.096 · Cited by 1,079
Health behavior models in the age of mobile interventions: are our theories up to the task?
Riley, W. T., Rivera, D. E., Atienza, A. A., Nilsen, W., Allison, S. M., & Mermelstein, R. (2011) · Translational Behavioral Medicine · doi:10.1007/s13142-011-0021-7 · Cited by 978
Optimal and Autonomous Control Using Reinforcement Learning: A Survey
Kiumarsi, B., Vamvoudakis, K. G., Modares, H., & Lewis, F. L. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2773458 · Cited by 924
Self-citation
Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method
Zhang, H., Cui, L., Zhang, X., & Luo, Y. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2168538 · Cited by 645
Reinforcement -learning for optimal tracking control of linear discrete-time systems with unknown dynamics
Kiumarsi, B., Lewis, F. L., Modares, H., Karimpour, A., & Sistani, M. B. N. (2014) · Automatica · doi:10.1016/j.automatica.2014.02.015 · Cited by 575
Self-citation
Near-Optimal Control for Nonzero-Sum Differential Games of Continuous-Time Nonlinear Systems Using Single-Network ADP
Zhang, H., Cui, L., & Luo, Y. (2012) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2203336 · Cited by 465
Linear Quadratic Tracking Control of Partially-Unknown Continuous-Time Systems Using Reinforcement Learning
Modares, H., & Lewis, F. L. (2014) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2014.2317301 · Cited by 451
Self-citation
Reinforcement-Learning-Based Robust Controller Design for Continuous-Time Uncertain Nonlinear Systems Subject to Input Constraints
Liu, D., Yang, X., Wang, D., & Wei, Q. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2417170 · Cited by 440
Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems
Jiang, Y., & Jiang, Z. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2294968 · Cited by 419
Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design
Bian, T., & Jiang, Z.-P. (2016) · Automatica · doi:10.1016/j.automatica.2016.05.003 · Cited by 348
Model-Free Optimal Tracking Control via Critic-Only Q-Learning
Luo, B., Liu, D., Huang, T., & Wang, D. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2585520 · Cited by 334
Finite-Approximation-Error-Based Optimal Control Approach for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2012) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2216523 · Cited by 303
Online Synchronous Approximate Optimal Learning Algorithm for Multi-Player Non-Zero-Sum Games With Unknown Dynamics
Liu, D., Li, H., & Wang, D. (2014) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2013.2295351 · Cited by 261
An Event-Triggered ADP Control Approach for Continuous-Time System With Unknown Internal States
Zhong, X., & He, H. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2523878 · Cited by 263
Optimal Tracking Control of Unknown Discrete-Time Linear Systems Using Input-Output Measured Data
Kiumarsi, B., Lewis, F. L., Sistani, M. B. N., & Karimpour, A. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2384016 · Cited by 251
Self-citation
Off-Policy Actor-Critic Structure for Optimal Control of Unknown Systems With Disturbances
Song, R., Lewis, F. L., Wei, Q., & Zhang, H. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2421338 · Cited by 241
Self-citation
Q-learning for continuous-time linear systems: A model-free infinite horizon optimal control approach
Vamvoudakis, K. G. (2016) · Systems & Control Letters · doi:10.1016/j.sysconle.2016.12.003 · Cited by 245
Self-citation
Reinforcement learning and optimal adaptive control: An overview and implementation examples
Khan, S. G., Herrmann, G., Lewis, F. L., Pipe, T., & Melhuish, C. (2012) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2012.03.004 · Cited by 241
Self-citation
Output-feedback adaptive optimal control of interconnected systems based on robust adaptive dynamic programming
Gao, W., Jiang, Y., Jiang, Z.-P., & Chai, T. (2016) · Automatica · doi:10.1016/j.automatica.2016.05.008 · Cited by 234
Off-Policy Integral Reinforcement Learning Method to Solve Nonlinear Continuous-Time Multiplayer Nonzero-Sum Games
Song, R., Lewis, F. L., & Wei, Q. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582849 · Cited by 234
Self-citation
Adaptive Fault-Tolerant Tracking Control for MIMO Discrete-Time Systems via Reinforcement Learning Algorithm With Less Learning Parameters
Liu, L., Wang, Z., & Zhang, H. (2016) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2016.2517155 · Cited by 232
Policy Gradient Adaptive Dynamic Programming for Data-Based Optimal Control
Luo, B., Liu, D., Wu, H., Wang, D., & Lewis, F. L. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2623859 · Cited by 224
Self-citation
Adaptive Resilient Event-Triggered Control Design of Autonomous Vehicles With an Iterative Single Critic Learning Framework
Zhang, K., Su, R., Zhang, H., & Tian, Y. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3053269 · Cited by 215
Distributed Optimal Consensus Control for Nonlinear Multiagent System With Unknown Dynamic
Zhang, J., Zhang, H., & Feng, T. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2728622 · Cited by 205
Leader-to-Formation Stability of Multiagent Systems: An Adaptive Optimal Control Approach
Gao, W., Jiang, Z., Lewis, F. L., & Wang, Y. (2018) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2018.2799526 · Cited by 197
Self-citation
Model-Freeλ-Policy Iteration for Discrete-Time Linear Quadratic Regulation
Yang, Y., Kiumarsi, B., Modares, H., & Xu, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3098985 · Cited by 199
Integral Q-learning and explorized policy iteration for adaptive optimal control of continuous-time linear systems
Lee, J. Y., Park, J. B., & Choi, Y. H. (2012) · Automatica · doi:10.1016/j.automatica.2012.06.008 · Cited by 191
Finite-Approximation-Error-Based Discrete-Time Iterative Adaptive Dynamic Programming
Wei, Q., Wang, F., Liu, D., & Yang, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2354377 · Cited by 188
Optimal Output-Feedback Control of Unknown Continuous-Time Linear Systems Using Off-policy Reinforcement Learning
Modares, H., Lewis, F. L., & Jiang, Z. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2477810 · Cited by 181
Self-citation
Barrier-Critic Adaptive Robust Control of Nonzero-Sum Differential Games for Uncertain Nonlinear Systems With State Constraints
Qin, C., Qiao, X., Wang, J., Zhang, D., Hou, Y., & Hu, S. (2023) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2023.3302656 · Cited by 174
Improved Deep Hybrid Networks for Urban Traffic Flow Prediction Using Trajectory Data
Duan, Z., Yang, Y., Zhang, K., Ni, Y., & Bajgain, S. (2018) · IEEE Access · doi:10.1109/access.2018.2845863 · Cited by 163
Robust Adaptive Dynamic Programming for Large-Scale Systems With an Application to Multimachine Power Systems
Jiang, Y., & Jiang, Z. (2012) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2012.2213353 · Cited by 155
Optimal Control for Unknown Discrete-Time Nonlinear Markov Jump Systems Using Adaptive Dynamic Programming
Zhong, X., He, H., Zhang, H., & Wang, Z. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2305841 · Cited by 153
Adaptive $Q$ -Learning for Data-Based Optimal Output Regulation With Experience Replay
Luo, B., Yang, Y., & Liu, D. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2821369 · Cited by 143
Multiple Actor-Critic Structures for Continuous-Time Optimal Control Using Input-Output Data
Song, R., Lewis, F. L., Wei, Q., Zhang, H., Jiang, Z., & Levine, D. S. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2399020 · Cited by 140
Self-citation
Reinforcement Learning-Based Linear Quadratic Regulation of Continuous-Time Systems Using Dynamic Output Feedback
Rizvi, S. A. A., & Lin, Z. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2886735 · Cited by 139
Relations between Model Predictive Control and Reinforcement Learning
Görges, D. (2017) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2017.08.747 · Cited by 136
A Learning-Based Power Management Method for Networked Microgrids Under Incomplete Information
Zhang, Q., Dehghanpour, K., Wang, Z., & Huang, Q. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2933502 · Cited by 136
Adaptive Constrained Optimal Control Design for Data-Based Nonlinear Discrete-Time Systems With Critic-Only Structure
Luo, B., Liu, D., & Wu, H. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2751018 · Cited by 134
Fault-tolerant control of a hydraulic servo actuator via adaptive dynamic programming
Stojanović, V. (2023) · Mathematical Modelling and Control · doi:10.3934/mmc.2023016 · Cited by 136
Robust Adaptive Dynamic Programming
Jiang, Y., & Jiang, Z. (2017) · Wiley eBooks · doi:10.1002/9781119132677.ch5 · Cited by 138
Optimal control for discrete-time affine non-linear systems using general value iteration
Li, H., & Liu, D. (2012) · IET Control Theory and Applications · doi:10.1049/iet-cta.2011.0783 · Cited by 133
Output feedback Q-learning for discrete-time linear zero-sum games with application to the H-infinity control
Rizvi, S. A. A., & Lin, Z. (2018) · Automatica · doi:10.1016/j.automatica.2018.05.027 · Cited by 131
Data-Based Optimal Control of Multiagent Systems: A Reinforcement Learning Design Approach
Zhang, J., Wang, Z., & Zhang, H. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2868715 · Cited by 125
Data-Driven $H_\infty$ Control for Nonlinear Distributed Parameter Systems
Luo, B., Huang, T., Wu, H., & Yang, X. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2461023 · Cited by 120
Output Feedback Q-Learning Control for the Discrete-Time Linear Quadratic Regulator Problem
Rizvi, S. A. A., & Lin, Z. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2870075 · Cited by 124
Robust Optimal Control Scheme for Unknown Constrained-Input Nonlinear Systems via a Plug-n-Play Event-Sampled Critic-Only Algorithm
Zhang, H., Zhang, K., Xiao, G., & Jiang, H. (2019) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2018.2889377 · Cited by 119
Adaptive Neural Network-Based Finite-Time Online Optimal Tracking Control of the Nonlinear System With Dead Zone
Ding, L., Li, S., Gao, H., Liu, Y., Huang, L., & Deng, Z. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2939424 · Cited by 113
Adaptive Suboptimal Output-Feedback Control for Linear Systems Using Integral Reinforcement Learning
Zhu, L. M., Modares, H., Gan, O. P., Lewis, F. L., & Yue, B. (2014) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2014.2322778 · Cited by 108
Self-citation
Data-driven control of hydraulic servo actuator: An event-triggered adaptive dynamic programming approach
Djordjevic, V., Tao, H., Song, X., He, S., Gao, W., & Stojanovic, V. (2023) · Mathematical Biosciences and Engineering · doi:10.3934/mbe.2023376 · Cited by 107
“Motivated by [44,47], the discrete-time HSA described by (3.”
Method
Robust Adaptive Dynamic Programming With an Application to Power Systems
Jiang, Y., & Jiang, Z. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2249668 · Cited by 106
Adaptive optimal control for a class of continuous-time affine nonlinear systems with unknown internal dynamics
Liu, D., Yang, X., & Li, H. (2012) · Neural Computing and Applications · doi:10.1007/s00521-012-1249-y · Cited by 102
Model‐based vs data‐driven adaptive control: An overview
Benosman, M. (2018) · International Journal of Adaptive Control and Signal Processing · doi:10.1002/acs.2862 · Cited by 101
Continuous-Time Q-Learning for Infinite-Horizon Discounted Cost Linear Quadratic Regulator Problems
Muthukumar, P., Modares, H., Lewis, F. L., & Aurangzeb, M. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2322116 · Cited by 97
Self-citation
Data-driven control of hydraulic servo actuator based on adaptive dynamic programming
Djordjevic, V., Stojanović, V., Tao, H., Song, X., He, S., & Gao, W. (2022) · Discrete and Continuous Dynamical Systems - S · doi:10.3934/dcdss.2021145 · Cited by 93
Data-Driven Finite-Horizon Approximate Optimal Control for Discrete-Time Nonlinear Systems Using Iterative HDP Approach
Mu, C., Wang, D., & He, H. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2752845 · Cited by 91
Adaptive Dynamic Programming for Stochastic Systems With State and Control Dependent Noise
Bian, T., Jiang, Y., & Jiang, Z. (2016) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2016.2550518 · Cited by 90
Adaptive optimal output tracking of continuous-time systems via output-feedback-based reinforcement learning
Chen, C., Xie, L., Xie, K., Lewis, F. L., & Xie, S. (2022) · Automatica · doi:10.1016/j.automatica.2022.110581 · Cited by 85
Self-citation
Reinforcement Q-Learning Algorithm for H ∞ Tracking Control of Unknown Discrete-Time Linear Systems
Peng, Y., Chen, Q., & Sun, W. (2019) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2957000 · Cited by 84
Approximate Optimal Control Design for Nonlinear One-Dimensional Parabolic PDE Systems Using Empirical Eigenfunctions and Neural Network
Luo, B., & Wu, H. (2012) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2012.2194781 · Cited by 82
Towards a Smarter Energy Management System for Hybrid Vehicles: A Comprehensive Review of Control Strategies
Xu, N., Kong, Y., Chu, L., Ju, H., Zhi-hua, Y., Xu, Z., & Xu, Z. (2019) · Applied Sciences · doi:10.3390/app9102026 · Cited by 82
Neural-network-based adaptive optimal tracking control scheme for discrete-time nonlinear systems with approximation errors
Wei, Q., & Liu, D. (2014) · Neurocomputing · doi:10.1016/j.neucom.2013.09.069 · Cited by 79
Robust Output Regulation and Reinforcement Learning-Based Output Tracking Design for Unknown Linear Discrete-Time Systems
Chen, C., Xie, L., Jiang, Y., Xie, K., & Xie, S. (2022) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2022.3172590 · Cited by 79
Robust optimal control for a class of nonlinear systems with unknown disturbances based on disturbance observer and policy iteration
Song, R., & Lewis, F. L. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.01.082 · Cited by 78
Self-citation
Data-driven model reference control of MIMO vertical tank systems with model-free VRFT and Q-Learning
Rădac, M., Precup, R., & Roman, R. (2018) · ISA Transactions · doi:10.1016/j.isatra.2018.01.014 · Cited by 76
Self-Learning Cruise Control Using Kernel-Based Least Squares Policy Iteration
Wang, J., Xu, X., Liu, D., Sun, Z., & Chen, Q. (2013) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2013.2271276 · Cited by 76
ADP-Based Online Tracking Control of Partially Uncertain Time-Delayed Nonlinear System and Application to Wheeled Mobile Robots
Li, S., Ding, L., Gao, H., Liu, Y., Huang, L., & Deng, Z. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2900326 · Cited by 76
Data-driven model-free slip control of anti-lock braking systems using reinforcement Q-learning
Rădac, M., & Precup, R. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.08.036 · Cited by 75
Adaptive Optimal Output Regulation of Time-Delay Systems via Measurement Feedback
Gao, W., & Jiang, Z. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2850520 · Cited by 75
Toward Expedited Impedance Tuning of a Robotic Prosthesis for Personalized Gait Assistance by Reinforcement Learning Control
Li, M., Wen, Y., Gao, X. H., Si, J., & Huang, H. (2021) · IEEE Transactions on Robotics · doi:10.1109/tro.2021.3078317 · Cited by 78
Design of a completely model free adaptive control in the presence of parametric, non-parametric uncertainties and random control signal delay
Tutsoy, Ö., Barkana, D. E., & Tugal, H. (2018) · ISA Transactions · doi:10.1016/j.isatra.2018.03.002 · Cited by 73
H∞ Tracking Control of Discrete-Time System With Delays via Data-Based Adaptive Dynamic Programming
Liu, Y., Zhang, H., Yu, R., & Xing, Z. (2019) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2946397 · Cited by 70
Linear Quadratic Control Using Model-Free Reinforcement Learning
Yaghmaie, F. A., Gustafsson, F., & Lennart Ljung, A. (2022) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2022.3145632 · Cited by 71
Data-Driven Coordinated Attack Policy Design Based on Adaptive $\mathcal {L}_2$ -Gain Optimal Theory
An, L., & Yang, G. (2017) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2017.2753466 · Cited by 70
H ∞ Tracking Control for Linear Discrete-Time Systems: Model-Free Q-Learning Designs
Yang, Y., Wan, Y., Zhu, J., & Lewis, F. L. (2020) · IEEE Control Systems Letters · doi:10.1109/lcsys.2020.3001241 · Cited by 65
Self-citation
Control of a nonlinear liquid level system using a new artificial neural network based reinforcement learning approach
Noel, M. M., & Pandian, B. J. (2014) · Applied Soft Computing · doi:10.1016/j.asoc.2014.06.037 · Cited by 64
Data-Based Adaptive Dynamic Programming for a Class of Discrete-Time Systems With Multiple Delays
Zhang, H., Liu, Y., Xiao, G., & Jiang, H. (2017) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2017.2758849 · Cited by 64
Data-Driven Distributed Output Consensus Control for Partially Observable Multiagent Systems
Jiang, H., & He, H. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2788819 · Cited by 64
Adaptive dynamic programming-based optimal control of unknown nonaffine nonlinear discrete-time systems with proof of convergence
Zhang, X., Zhang, H., Sun, Q., & Luo, Y. (2012) · Neurocomputing · doi:10.1016/j.neucom.2012.01.025 · Cited by 62
Adaptive neural network‐based optimal control of nonlinear continuous‐time systems in strict‐feedback form
Zargarzadeh, H., Dierks, T., & Jagannathan, S. (2013) · International Journal of Adaptive Control and Signal Processing · doi:10.1002/acs.2432 · Cited by 61
Discrete-time online learning control for a class of unknown nonaffine nonlinear systems using reinforcement learning
Yang, X., Liu, D., Wang, D., & Wei, Q. (2014) · Neural Networks · doi:10.1016/j.neunet.2014.03.008 · Cited by 61
Fully distributed adaptive control for output consensus of uncertain discrete-time linear multi-agent systems
Jiang, Y., Liu, L., & Feng, G. (2024) · Automatica · doi:10.1016/j.automatica.2024.111531 · Cited by 60
An adaptive decision-making method with fuzzy Bayesian reinforcement learning for robot soccer
Shi, H., Lin, Z., Zhang, S., Li, X., & Hwang, K. (2018) · Information Sciences · doi:10.1016/j.ins.2018.01.032 · Cited by 60
Model-free Adaptive Dynamic Programming Based Near-optimal Decentralized Tracking Control of Reconfigurable Manipulators
Zhao, B., & Li, Y. (2018) · International Journal of Control Automation and Systems · doi:10.1007/s12555-016-0711-5 · Cited by 59
LQ Secure Control for Cyber-Physical Systems Against Sparse Sensor and Actuator Attacks
An, L., & Yang, G. (2018) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2018.2878507 · Cited by 59
Model-Free control performance improvement using virtual reference feedback tuning and reinforcement Q-learning
Rădac, M., Precup, R., & Roman, R. (2016) · International Journal of Systems Science · doi:10.1080/00207721.2016.1236423 · Cited by 58
Input–Output Data-Based Output Antisynchronization Control of Multiagent Systems Using Reinforcement Learning Approach
Peng, Z., Zhao, Y., Hu, J., Luo, R., Ghosh, B. K., & Nguang, S. K. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3050768 · Cited by 58
Nonlinear Adaptive Flight Control Using Incremental Approximate Dynamic Programming and Output Feedback
Zhou, Y., Kampen, E.-J. V., & Chu, Q. P. (2016) · Journal of Guidance Control and Dynamics · doi:10.2514/1.g001762 · Cited by 58
Off-Policy Reinforcement Learning for Tracking in Continuous-Time Systems on Two Time Scales
Xue, W., Fan, J., Lopez, V. G., Jiang, Y., Chai, T., & Lewis, F. L. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3017461 · Cited by 58
Self-citation
Sliding Mode Control Based on Reinforcement Learning for T-S Fuzzy Fractional-Order Multiagent System With Time-Varying Delays
Yan, Y., Zhang, H., Sun, J., & Wang, Y. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3241070 · Cited by 57
Model-free optimal control design for a class of linear discrete-time systems with multiple delays using adaptive dynamic programming
Zhang, J., Zhang, H., Luo, Y., & Feng, T. (2014) · Neurocomputing · doi:10.1016/j.neucom.2013.12.038 · Cited by 57
Reduced-dimensional reinforcement learning control using singular perturbation approximations
Mukherjee, S., Bai, H., & Chakrabortty, A. (2021) · Automatica · doi:10.1016/j.automatica.2020.109451 · Cited by 57
Opacity Enforcement for Confidential Robust Control in Linear Cyber-Physical Systems
An, L., & Yang, G. (2019) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2019.2925498 · Cited by 55
Output feedback adaptive dynamic programming for linear differential zero-sum games
Rizvi, S. A. A., & Lin, Z. (2020) · Automatica · doi:10.1016/j.automatica.2020.109272 · Cited by 54
Approximate-optimal control algorithm for constrained zero-sum differential games through event-triggering mechanism
Mu, C., & Wang, K. (2018) · Nonlinear Dynamics · doi:10.1007/s11071-018-4713-0 · Cited by 53
Optimal control of a two‐wheeled self‐balancing robot by reinforcement learning
Guo, L., Rizvi, S. A. A., & Lin, Z. (2020) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.5058 · Cited by 53
Control performance monitoring and degradation recovery in automatic control systems: A review, some new results, and future perspectives
Ding, S. X., & Li, L. (2021) · Control Engineering Practice · doi:10.1016/j.conengprac.2021.104790 · Cited by 52
Online finite-horizon optimal learning algorithm for nonzero-sum games with partially unknown dynamics and constrained inputs
Cui, X., Zhang, H., Luo, Y., & Pei-fu, Z. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.12.021 · Cited by 52
Output-feedback Robust Tracking Control of Uncertain Systems via Adaptive Learning
Zhao, J., & Lv, Y. (2023) · International Journal of Control Automation and Systems · doi:10.1007/s12555-021-0882-6 · Cited by 52

Authors: Lewis, F. L., & Vamvoudakis, K. G.
Affiliations: The University of Texas at Arlington (US)
ORCID: Lewis, F L
Vamvoudakis, K G
Venue: IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2011-02
Volume: 41
Issue: 1
Pages: 14-25
ISSN: 1083-4419 (ISSN-L)
1941-0492
DOI: 10.1109/tsmcb.2010.2043839
OpenAlex: W2151966330
Cited by: 544
References: 48
Citations by year: 2026: 22 · 2025: 49 · 2024: 46 · 2023: 43 · 2022: 37 · 2021: 49 · 2020: 53 · 2019: 34 · 2018: 52 · 2017: 26 · 2016: 35 · 2015: 19 · 2014: 33 · 2013: 18 · 2012: 19
FWCI: 15.12
Type: journal article
Language: en
Open access: closed
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Locations: IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) (publishedVersion)
PubMed (publishedVersion)
Topics: Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
Keywords: approximate dynamic programming, partially observable Markov decision processes, output feedback control, linear dynamical systems, policy iteration, value iteration, Q-learning, observability index
MeSH terms: Algorithms, Artificial Intelligence, Feedback, Learning, Markov Chains, Reinforcement, Psychology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks