Cited by 544Adaptive Dynamic Programming ControlReinforcement Learning in RoboticsAdvanced Control Systems Optimization

The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.

Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data

Lewis, F L, Vamvoudakis, K G·2011·IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)

DOI 10.1109/tsmcb.2010.2043839

Open this paper in CoChat — free account

Abstract

Approximate dynamic programming (ADP) is a class of reinforcement learning methods that have shown their importance in a variety of applications, including feedback control of dynamical systems. ADP generally requires full information about the system internal states, which is usually not available in practical situations. In this paper, we show how to implement ADP methods using only measured input/output data from the system. Linear dynamical systems with deterministic behavior are considered herein, which are systems of great interest in the control system community. In control system theory, these types of methods are referred to as output feedback (OPFB). The stochastic equivalent of the systems dealt with in this paper is a class of partially observable Markov decision processes. We develop both policy iteration and value iteration algorithms that converge to an optimal controller that requires only OPFB. It is shown that, similar to Q -learning, the new methods have the important advantage that knowledge of the system dynamics is not needed for the implementation of these learning algorithms or for the OPFB control. Only the order of the system, as well as an upper bound on its "observability index," must be known. The learned OPFB controller is in the form of a polynomial autoregressive moving-average controller that has equivalent performance with the optimal state variable feedback gain.

Integrity signals

Registry

Crossref · 10.1109/tsmcb.2010.2043839

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Learning from Delayed Rewards
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,674
Neuro-dynamic Programming
(2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,601
Optimal Control
Lewis, F. L. (1986) · Cited by 2,957
Self-citation
Beyond regression : new fools for prediction and analysis in the behavioral sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,739
Approximate Dynamic Programming
Powell, W. B. (2011) · Wiley series in probability and statistics · doi:10.1002/9781118029176 · Cited by 2,359
Kronecker products and matrix calculus in system theory
Brewer, J. (1978) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1978.1084534 · Cited by 1,920
Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,722
Self-citation
Reinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,591
Self-citation

Show all 48 references

Research works printed but not linked: 3

sutton, 1998, Reinforcement LearningAn Introduction
werbos, 1992, Handbook of Intelligent Control
1992, Handbook of Intelligent Control

Sorted by citations

Items 201–300 of 544

Stochastic Optimal Design for Unknown Linear Discrete‐Time System Zero‐Sum Games in Input‐Output form Under Communication Constraints
Xu, H., Jagannathan, S., & Lewis, F. L. (2013) · Asian Journal of Control · doi:10.1002/asjc.804 · Cited by 16
Self-citation
Output feedback reinforcement learning based optimal output synchronisation of heterogeneous discrete‐time multi‐agent systems
Rizvi, S. A. A., & Lin, Z. (2019) · IET Control Theory and Applications · doi:10.1049/iet-cta.2018.6266 · Cited by 16
$H_\infty$ Control for a Class of Discrete-Time Systems via Data-Based Policy Iteration With Application to Wind Turbine Control
Liu, Y., Xing, Z., Chen, L., Xu, J. Y., Li, Y., & Wang, H. (2019) · IEEE Access · doi:10.1109/access.2019.2962583 · Cited by 16
Optimal tracking control for non‐zero‐sum games of linear discrete‐time systems via off‐policy reinforcement learning
Wen, Y., Zhang, H., Su, H., & Ren, H. (2020) · Optimal Control Applications and Methods · doi:10.1002/oca.2597 · Cited by 16
Reinforcement Q-Learning Incorporated With Internal Model Method for Output Feedback Tracking Control of Unknown Linear Systems
Chen, C., Sun, W., Zhao, G., & Peng, Y. (2020) · IEEE Access · doi:10.1109/access.2020.3011194 · Cited by 16
Fast Real-Time Reinforcement Learning for Partially-Observable Large-Scale Systems
Sadamoto, T., & Chakrabortty, A. (2020) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3058228 · Cited by 16
Optimal Design of Residual-Driven Dynamic Compensator Using Iterative Algorithms With Guaranteed Convergence
Zhang, Y., Yang, Y. Z., Ding, S. X., & Li, L. (2015) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2015.2450203 · Cited by 17
Optimal Control of Unknown Nonlinear Discrete‐Time Systems Using the Iterative Globalized Dual Heuristic Programming Algorithm
Liu, D., & Wang, D. (2012) · Wiley eBooks · doi:10.1002/9781118453988.ch3 · Cited by 16
Specified convergence rate guaranteed output tracking of discrete-time systems via reinforcement learning
Huang, C., Chen, C., Xie, K., Lewis, F. L., & Xie, S. (2024) · Automatica · doi:10.1016/j.automatica.2023.111490 · Cited by 15
Self-citation
A Note on State Parameterizations in Output Feedback Reinforcement Learning Control of Linear Systems
Rizvi, S. A. A., & Lin, Z. (2022) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2022.3228969 · Cited by 15
Optimal Output Feedback Tracking Control for Takagi–Sugeno Fuzzy Systems
Song, W., & Tong, S. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3443004 · Cited by 15
An Online Event-Triggered Near-Optimal Controller for Nash Solution in Interconnected System
Dhar, N. K., Verma, N. K., & Behera, L. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2969249 · Cited by 15
Internal reinforcement adaptive dynamic programming for optimal containment control of unknown continuous-time multi-agent systems
Zhang, J., Peng, Z., Hu, J., Zhao, Y., Luo, R., & Ghosh, B. K. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.06.106 · Cited by 15
An Iterative Data-Driven Linear Quadratic Method to Solve Nonlinear Discrete-Time Tracking Problems
Possieri, C., Incremona, G. P., Calafiore, G. C., & Ferrara, A. (2021) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2021.3056398 · Cited by 15
A New Method for Fault Tolerant Control through Q-Learning
Hua, C., Ding, S. X., & Shardt, Y. A. W. (2018) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2018.09.526 · Cited by 15
Off-policy synchronous iteration IRL method for multi-player zero-sum games with input constraints
Ren, H., Zhang, H., Mu, Y., & Duan, J. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.10.075 · Cited by 15
Design of Observer-Based Control With Residual Generator Using Actor–Critic Reinforcement Learning
Qian, L., Zhao, X., Liu, P., Zhang, Z., & Lv, Y. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3215671 · Cited by 13
Incremental reinforcement learning and optimal output regulation under unmeasurable disturbances
Zhao, J., Yang, C., Gao, W., & Park, J. H. (2023) · Automatica · doi:10.1016/j.automatica.2023.111468 · Cited by 14
γ-Iterative Dual Heuristic Dynamic Programming for Nonlinear Critical Surfaces With Strong Constraints
Zhang, H., Yan, Y., & Sun, J. (2024) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2024.3355909 · Cited by 14
Model Reference Tracking Control Solutions for a Visual Servo System Based on a Virtual State from Unknown Dynamics
Lala, T., Chirla, D.-P., & Rădac, M. (2021) · Energies · doi:10.3390/en15010267 · Cited by 14
Output‐feedback robust control of systems with uncertain dynamics via data‐driven policy learning
Zhao, J., & Lv, Y. (2022) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.6374 · Cited by 14
Adaptive event-triggered control of a uncertain linear discrete time system using measured input and output data
Sahoo, A., Xu, H., & Jagannathan, S. (2013) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2013.6580726 · Cited by 14
Stackelberg games for model-free continuous-time stochastic systems based on adaptive dynamic programming
Liu, X., Ge, Y., & Li, Y. (2019) · Applied Mathematics and Computation · doi:10.1016/j.amc.2019.124568 · Cited by 14
Adaptive optimal output feedback tracking control for unknown discrete‐time linear systems using a combined reinforcement Q‐learning and internal model method
Sun, W., Zhao, G., & Peng, Y. (2019) · IET Control Theory and Applications · doi:10.1049/iet-cta.2019.0560 · Cited by 14
A Safety-Certified Policy Iteration Algorithm for Control of Constrained Nonlinear Systems
Yazdani, N. M., Moghaddam, R. K., Kiumarsi, B., & Modares, H. (2020) · IEEE Control Systems Letters · doi:10.1109/lcsys.2020.2990632 · Cited by 14
Intelligent adaptive optimal control using incremental model-based global dual heuristic programming subject to partial observability
Sun, B., & Kampen, E.-J. V. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107153 · Cited by 14
Data‐based optimal Denial‐of‐Service attack scheduling against robust control based on Q‐learning
An, L., & Yang, G. (2019) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.4666 · Cited by 14
Performance Optimization and Fault-Tolerance of Highly Dynamic Systems Via Q-Learning With an Incrementally Attached Controller Gain System
Xu, Y., Zhao, Z., & Yin, S. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3155876 · Cited by 13
S EADS
Fu, X., Cai, H., Li, W., & Li, L. (2020) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3379345 · Cited by 13
Data-based adaptive critic design for discrete-time zero-sum games using output feedback
Cui, L., Zhang, H., Zhang, X., & Luo, Y. (2011) · IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning · doi:10.1109/adprl.2011.5967351 · Cited by 13
Optimal trajectory Output Tracking control with a Q-learning algorithm
Vamvoudakis, K. G. (2016) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2016.7526571 · Cited by 13
Self-citation
Data-driven adaptive optimal output-feedback control of a 2-DOF helicopter
Gao, W., & Jiang, Z. (2016) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2016.7525294 · Cited by 13
Adaptive optimal output regulation via output-feedback: An adaptive dynamic programing approach
Gao, W., & Jiang, Z. (2016) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2016.7799168 · Cited by 13
Model‐free optimal tracking control for discrete‐time system with delays using reinforcement Q ‐learning
Liu, Y., & Yu, R. (2018) · Electronics Letters · doi:10.1049/el.2017.3238 · Cited by 13
Data-driven adaptive optimal control of linear uncertain systems with unknown jumping dynamics
Zhang, M., Gan, M., Chen, J., & Jiang, Z. (2019) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2019.05.020 · Cited by 13
Adaptive and optimal output feedback control of linear systems: An adaptive dynamic programming approach
Gao, W., Jiang, Y., Jiang, Z.-P., & Chai, T. (2014) · Proceeding of the 11th World Congress on Intelligent Control and Automation · doi:10.1109/wcica.2014.7053043 · Cited by 35
Model-Free H ∞ Output Feedback Control of Road Sensing in Vehicle Active Suspension Based on Reinforcement Learning
Wang, G., Li, K., Liu, S., & Jing, H. (2023) · Journal of Dynamic Systems Measurement and Control · doi:10.1115/1.4062342 · Cited by 12
An adaptive adjustment strategy for bolt posture errors based on an improved reinforcement learning algorithm
Luo, W., Zhang, J., Feng, P., Liu, H., Yu, D., & Wu, Z. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01906-x · Cited by 12
Optimal output tracking control of linear discrete-time systems with unknown dynamics by adaptive dynamic programming and output feedback
Cai, X. S., Wang, C., Liu, S., Chen, G., & Wang, G. (2022) · International Journal of Systems Science · doi:10.1080/00207721.2022.2085343 · Cited by 12
Optimal Tracking Control of Uncertain Systems
Kiumarsi, B., Modares, H., & Lewis, F. L. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-805246-4.00005-7 · Cited by 13
Self-citation
Output Feedback Reinforcement Learning Control for the Continuous-Time Linear Quadratic Regulator Problem
Rizvi, S. A. A., & Lin, Z. (2018) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.2018.8431290 · Cited by 12
Experience replay–based output feedback Q‐learning scheme for optimal output tracking control of discrete‐time linear systems
Rizvi, S. A. A., & Lin, Z. (2019) · International Journal of Adaptive Control and Signal Processing · doi:10.1002/acs.2981 · Cited by 12
Switching Flight Control for Incremental Model-Based Dual Heuristic Dynamic Programming
Li, H., Sun, L., Tan, W., Jia, B., & Liu, X. (2020) · Journal of Guidance Control and Dynamics · doi:10.2514/1.g004519 · Cited by 12
A novel method for aeroengine performance model reconstruction based on CDAE model
Lin, L., He, W., Guo, F., Fu, S., Tong, C., & Zu, L. (2023) · Advanced Engineering Informatics · doi:10.1016/j.aei.2023.101909 · Cited by 12
Policy Iteration-Based Learning Design for Linear Continuous-Time Systems Under Initial Stabilizing OPFB Policy
Zhang, C., Chen, C., Lewis, F. L., & Xie, S. (2024) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2024.3418190 · Cited by 11
Self-citation
Fuzzy reinforcement learning based control of linear systems with input saturation
Liu, K., Ban, X., & Xie, S. (2025) · ISA Transactions · doi:10.1016/j.isatra.2024.12.045 · Cited by 11
Optimal antisynchronization control for unknown multiagent systems with deep deterministic policy gradient approach
Zhang, C., Ji, L., Yang, S., & Li, H. (2022) · Information Sciences · doi:10.1016/j.ins.2022.12.008 · Cited by 11
A review of the applications and hotspots of reinforcement learning
Hou, J. Y., Li, H., Hu, J., Zhao, C., Guo, Y., Li, S., & Pan, Q. (2017) · IEEE International Conference on Unmanned Systems (ICUS) · doi:10.1109/icus.2017.8278398 · Cited by 11
Stability analysis of heuristic dynamic programming algorithm for nonlinear systems
Feng, T., Zhang, H., Luo, Y., & Zhang, J. (2014) · Neurocomputing · doi:10.1016/j.neucom.2014.08.046 · Cited by 11
Data-driven partially observable dynamic processes using adaptive dynamic programming
Zhong, X., Ni, Z., Tang, Y., & He, H. (2014) · IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning · doi:10.1109/adprl.2014.7010628 · Cited by 11
Cooperative attack strategy design via H−/H∞ scheme for linear cyber‐physical systems
Wang, X., & Yang, G. (2019) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.4747 · Cited by 11
Adaptive Output Synchronization With Designated Convergence Rate of Multiagent Systems Based on Off-Policy Reinforcement Learning
Huang, C., Chen, C., Xie, K., Li, Z., & Xie, S. (2024) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2024.3387395 · Cited by 10
Q-Learning Methods for LQR Control of Completely Unknown Discrete-Time Linear Systems
Fan, W., & Xiong, J. (2024) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2024.3434533 · Cited by 11
Output feedback fault-tolerant Q-learning for discrete-time linear systems with actuator faults
Rafiee, S., Kankashvar, M., & Bolandi, H. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109355 · Cited by 10
Stochastic H2/H∞ off-policy reinforcement learning tracking control for linear discrete-time systems with multiplicative noises
Yin, Y., Luo, S., & Deng, F. (2024) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2024.107349 · Cited by 10
Output Feedback Optimal Tracking Control Using Reinforcement Q-Learning
Rizvi, S. A. A., & Lin, Z. (2018) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.2018.8430997 · Cited by 10
Data Driven Distributed Bipartite Consensus Tracking for Nonlinear Multiagent Systems via Iterative Learning Control
Zhao, H., Peng, L., & Yu, H. (2020) · IEEE Access · doi:10.1109/access.2020.3014496 · Cited by 10
A Finite Horizon Markov Decision Process Based Reinforcement Learning Control of a Rapid Thermal Processing system
Pradeep, D. J., & Noel, M. M. (2018) · Journal of Process Control · doi:10.1016/j.jprocont.2018.06.002 · Cited by 10
Data-driven adaptive optimal control for stochastic systems with unmeasurable state
Zhang, M., Gan, M., & Chen, J. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.12.001 · Cited by 10
Optimal Control of a Two-Wheeled Self-Balancing Robot by Reinforcement Q-learning
Guo, L., Rizvi, S. A. A., & Lin, Z. (2020) · International Conference on Control and Automation · doi:10.1109/icca51439.2020.9264485 · Cited by 10
Optimal tracking control of switched systems applied in grid-connected hybrid generation using reinforcement learning
Sun, J., Zhang, H., Wang, Y., & Fu, M. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-05696-2 · Cited by 10
Data Driven Linear Quadratic Gaussian Control Design
Putri, A. N., Machbub, C., Mahayana, D., & Hidayat, E. (2023) · IEEE Access · doi:10.1109/access.2023.3254879 · Cited by 12
Event-triggered adaptive dynamic programming for continuous-time nonlinear system using measured input-output data
Zhong, X., Ni, Z., & He, H. (2015) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2015.7280471 · Cited by 10
Robust adaptive dynamic programming for nonlinear control design
Jiang, Y., & Jiang, Z. (2012) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2012.6426987 · Cited by 10
Data-Driven Nonlinear Near-Optimal Regulation Based on Multi-Dimensional Taylor Network Dynamic Programming
Sun, Q., Zhang, C., Jiang, N.-Y., Yu, J.-J., & Xu, L. (2020) · IEEE Access · doi:10.1109/access.2020.2975391 · Cited by 10
Online System Identification and Optimal Control for Mission-Critical IoT Systems Over MIMO Fading Channels
Tang, M., Cai, S., & Lau, V. K. N. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3175965 · Cited by 10
$F$ -Discrepancy for Efficient Sampling in Approximate Dynamic Programming
Cervellera, C., & Macciò, D. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2453123 · Cited by 9
Suboptimal reduced control of unknown nonlinear singularly perturbed systems via reinforcement learning
Liu, X., Yang, C., Zhou, L., Fu, J., & Dai, W. (2021) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.5624 · Cited by 9
Value iteration for LQR control of unknown stochastic-parameter linear systems
Fan, W., & Xiong, J. (2024) · Systems & Control Letters · doi:10.1016/j.sysconle.2024.105731 · Cited by 9
Online solution to the linear quadratic tracking problem of continuous-time systems using reinforcement learning
Modares, H., & Lewis, F. L. (2013) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2013.6760477 · Cited by 9
Self-citation
Finite-horizon LQ control for unknown discrete-time linear systems via extremum seeking
Frihauf, P., Krstić, M., & Başar, T. (2012) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2012.6426052 · Cited by 9
Constrained Differential Games for Secure Decision-Making Against Stealthy Attacks
Fotiadis, F., Kanellopoulos, A., & Vamvoudakis, K. G. (2020) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc45564.2020.9147555 · Cited by 9
Self-citation
Convergence of the standard RLS method andUDUTfactorisation of covariance matrix for solving the algebraic Riccati equation of the DLQR via heuristic approximate dynamic programming
Rêgo, P. H. M., Neto, J. V. D. F., & Ferreira, E. F. M. (2013) · International Journal of Systems Science · doi:10.1080/00207721.2013.844283 · Cited by 9
Adaptive dynamic programming for discrete-time LQR optimal tracking control problems with unknown dynamics
Liu, Y., Luo, Y., & Zhang, H. (2014) · IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning · doi:10.1109/adprl.2014.7010636 · Cited by 9
Data-driven distributed output consensus control for multi-agent systems with unknown internal state
Zhang, C., Ji, L., Yang, S., Guo, X., & Li, H. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128868 · Cited by 8
Optimal Output Feedback Learning Control for Continuous-Time Linear Quadratic Regulation
Xie, K., Guay, M., Lu, M., Wang, S., & Deng, F. Y. (2025) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2025.3532182 · Cited by 8
Robust Data-Driven Predictive Control for Unknown Linear Systems With Bounded Disturbances
Hu, K., & Liu, T. (2025) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2025.3560697 · Cited by 8
Linear Quadratic Control of Unknown Nonlinear Systems Using Model-Free Reinforcement Learning
Zhao, D., Feng, M., Xian, W., Qin, J., She, J., & Yan, S. (2025) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2025.3581264 · Cited by 8
Robust data-driven predictive control for unknown linear time-invariant systems
Hu, K., & Liu, T. (2024) · Systems & Control Letters · doi:10.1016/j.sysconle.2024.105914 · Cited by 8
Robust adaptive dynamic programming for optimal nonlinear control design
Jiang, Y., & Jiang, Z. (2013) · Asian Control Conference (ASCC) · doi:10.1109/ascc.2013.6606031 · Cited by 8
Off-policy integral reinforcement learning algorithm in dealing with nonzero sum game for nonlinear distributed parameter systems
Ren, H., Dai, J., Zhang, H., & Zhang, K. (2020) · Transactions of the Institute of Measurement and Control · doi:10.1177/0142331220932634 · Cited by 8
A new approach of optimal control for a class of continuous-time chaotic systems by an online ADP algorithm
Song, R., Xiao, W., & Wei, Q. (2014) · Chinese Physics B · doi:10.1088/1674-1056/23/5/050504 · Cited by 8
A Novel Approach for Sampling in Approximate Dynamic Programming Based on $F$ -Discrepancy
Cervellera, C., & Macciò, D. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2660533 · Cited by 8
Trans-layer model learning: A hierarchical modeling strategy for real-time reliability evaluation of complex systems
Tan, Z., Zhong, S., & Lin, L. (2018) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2018.09.016 · Cited by 8
LQ control of unknown discrete‐time linear systems—A novel approach and a comparison study
Li, N., Kolmanovsky, I. V., & Girard, A. (2018) · Optimal Control Applications and Methods · doi:10.1002/oca.2477 · Cited by 8
A novel data-driven model-free synchronization protocol for discrete-time multi-agent systems via TD3 based algorithm
Liu, Z., Li, Y., Ge, L., & Chen, Z. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111430 · Cited by 7
Neural-network-based optimal control for a class of nonlinear cdiscrete-time systems with control constraints using the citerative GDHP algorithm
Liu, D., Wang, D., & Zhao, D. (2011) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2011.6033199 · Cited by 7
Anti-lock braking systems data-driven control using Q-learning
Rădac, M., Precup, R., & Roman, R. (2017) · IEEE International Symposium on Industrial Electronics (ISIE) · doi:10.1109/isie.2017.8001283 · Cited by 7
Data-Driven Model-Free Model-Reference Nonlinear Virtual State Feedback Control from Input-Output Data
Rădac, M., Precup, R., Hedrea, E., & Mituleţu, I.-C. (2018) · Mediterranean Conference on Control and Automation (MED) · doi:10.1109/med.2018.8442894 · Cited by 7
Safe Reinforcement Learning Control for Water Distribution Networks
Val, J., Wiśniewski, R., & Kallesøe, C. S. (2021) · IEEE Conference on Control Technology and Applications (CCTA) · doi:10.1109/ccta48906.2021.9659138 · Cited by 7
Input-output data based tracking control under DoS attacks
Yin, L., Xu, L., Zhu, H., Zhu, Y., & Wu, C. (2023) · International Journal of Control · doi:10.1080/00207179.2023.2221745 · Cited by 7
Optimal control for a class of nonlinear systems with state delay based on Adaptive Dynamic Programming with ε-error bound
Lin, X., Cao, N., & Lin, Y. (2013) · IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning · doi:10.1109/adprl.2013.6615005 · Cited by 7
Reinforcement Learning based Online Adaptive Flight Control for the Cessna Citation II(PH-LAB) Aircraft
Konatala, R., Kampen, E.-J. V., & Looye, G. (2021) · AIAA SciTech Forum · doi:10.2514/6.2021-0883 · Cited by 6
Solving optimal predictor-feedback control using approximate dynamic programming
Wang, H., Zhao, F., Zhang, Z., Xu, J., & Li, X. (2024) · Automatica · doi:10.1016/j.automatica.2024.111848 · Cited by 6
Successive over relaxation for model-free LQR control of discrete-time Markov jump systems
Fan, W., & Xiong, J. (2024) · Automatica · doi:10.1016/j.automatica.2024.111919 · Cited by 6
An Incremental Approximate Dynamic Programming Flight Controller Based on Output Feedback
Zhou, Y., Kampen, E.-J. V., & Chu, Q. P. (2016) · AIAA Guidance, Navigation, and Control Conference and Exhibit · doi:10.2514/6.2016-0360 · Cited by 6
Learning Model-Free Reference Tracking Control with Affordable Systems
Rădac, M., & Borlea, A.-B. (2022) · Intelligent systems reference library · doi:10.1007/978-3-031-09928-1_10 · Cited by 6
Constrained event-driven policy iteration design for nonlinear discrete time systems
Liu, L., Song, R., & Xia, L. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.01.060 · Cited by 6
A novel adaptive cooperative attack design against cyber-physical systems via mixed H∞/ H − index
Zhang, T., & Ye, D. (2018) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2018.09.028 · Cited by 6
Intelligent Nonlinear Adaptive Flight Control using Incremental Approximate Dynamic Programming
Dias, P. M., Zhou, Y., & Kampen, E.-J. V. (2019) · AIAA SciTech Forum · doi:10.2514/6.2019-2339 · Cited by 6

Authors: Lewis, F. L., & Vamvoudakis, K. G.
Affiliations: The University of Texas at Arlington (US)
ORCID: Lewis, F L
Vamvoudakis, K G
Venue: IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2011-02
Volume: 41
Issue: 1
Pages: 14-25
ISSN: 1083-4419 (ISSN-L)
1941-0492
DOI: 10.1109/tsmcb.2010.2043839
OpenAlex: W2151966330
Cited by: 544
References: 48
Citations by year: 2026: 22 · 2025: 49 · 2024: 46 · 2023: 43 · 2022: 37 · 2021: 49 · 2020: 53 · 2019: 34 · 2018: 52 · 2017: 26 · 2016: 35 · 2015: 19 · 2014: 33 · 2013: 18 · 2012: 19
FWCI: 15.12
Type: journal article
Language: en
Open access: closed
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Locations: IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) (publishedVersion)
PubMed (publishedVersion)
Topics: Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
Keywords: approximate dynamic programming, partially observable Markov decision processes, output feedback control, linear dynamical systems, policy iteration, value iteration, Q-learning, observability index
MeSH terms: Algorithms, Artificial Intelligence, Feedback, Learning, Markov Chains, Reinforcement, Psychology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks