Cited by 544Adaptive Dynamic Programming ControlReinforcement Learning in RoboticsAdvanced Control Systems Optimization

The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.

Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data

Lewis, F L, Vamvoudakis, K G·2011·IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)

DOI 10.1109/tsmcb.2010.2043839

Open this paper in CoChat — free account

Abstract

Approximate dynamic programming (ADP) is a class of reinforcement learning methods that have shown their importance in a variety of applications, including feedback control of dynamical systems. ADP generally requires full information about the system internal states, which is usually not available in practical situations. In this paper, we show how to implement ADP methods using only measured input/output data from the system. Linear dynamical systems with deterministic behavior are considered herein, which are systems of great interest in the control system community. In control system theory, these types of methods are referred to as output feedback (OPFB). The stochastic equivalent of the systems dealt with in this paper is a class of partially observable Markov decision processes. We develop both policy iteration and value iteration algorithms that converge to an optimal controller that requires only OPFB. It is shown that, similar to Q -learning, the new methods have the important advantage that knowledge of the system dynamics is not needed for the implementation of these learning algorithms or for the OPFB control. Only the order of the system, as well as an upper bound on its "observability index," must be known. The learned OPFB controller is in the form of a polynomial autoregressive moving-average controller that has equivalent performance with the optimal state variable feedback gain.

Integrity signals

Registry

Crossref · 10.1109/tsmcb.2010.2043839

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Learning from Delayed Rewards
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,674
Neuro-dynamic Programming
(2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,601
Optimal Control
Lewis, F. L. (1986) · Cited by 2,957
Self-citation
Beyond regression : new fools for prediction and analysis in the behavioral sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,739
Approximate Dynamic Programming
Powell, W. B. (2011) · Wiley series in probability and statistics · doi:10.1002/9781118029176 · Cited by 2,359
Kronecker products and matrix calculus in system theory
Brewer, J. (1978) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1978.1084534 · Cited by 1,920
Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,722
Self-citation
Reinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,591
Self-citation

Flagged first, then by citations

Show all 48 references

Research works printed but not linked: 3

sutton, 1998, Reinforcement LearningAn Introduction
werbos, 1992, Handbook of Intelligent Control
1992, Handbook of Intelligent Control

Sorted by citations

Items 101–200 of 544

Output-Feedback Robust Control of Uncertain Systems via Online Data-Driven Learning
Na, J., Zhao, J., Gao, G., & Li, Z. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3007414 · Cited by 50
Sampled‐data‐based adaptive optimal output‐feedback control of a 2‐degree‐of‐freedom helicopter
Gao, W., Huang, M., Jiang, Z., & Chai, T. (2016) · IET Control Theory and Applications · doi:10.1049/iet-cta.2015.0977 · Cited by 49
Finite-horizon LQ control for unknown discrete-time linear systems via extremum seeking
Frihauf, P., Krstić, M., & Başar, T. (2013) · European Journal of Control · doi:10.1016/j.ejcon.2013.05.015 · Cited by 47
Nonlinear and Adaptive Suboptimal Control of Connected Vehicles: A Global Adaptive Dynamic Programming Approach
Gao, W., & Jiang, Z. (2016) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-016-0395-3 · Cited by 47
Data-Driven Model-Free Tracking Reinforcement Learning Control with VRFT-based Adaptive Actor-Critic
Rădac, M., & Precup, R. (2019) · Applied Sciences · doi:10.3390/app9091807 · Cited by 47
Optimized Impedance Adaptation of Robot Manipulator Interacting With Unknown Environment
Liu, X., Ge, S. S., Zhao, F., & Mei, X. (2020) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2020.2971944 · Cited by 47
Tracking control optimization scheme of continuous-time nonlinear system via online single network adaptive critic design method
Zhang, K., Zhang, H., Xiao, G., & Su, H. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.04.008 · Cited by 46
Adaptive optimal output regulation of linear discrete-time systems based on event-triggered output-feedback
Zhao, F., Gao, W., Liu, T., & Jiang, Z. (2021) · Automatica · doi:10.1016/j.automatica.2021.110103 · Cited by 46
Output‐feedback H∞quadratic tracking control of linear systems using reinforcement learning
Moghadam, R., & Lewis, F. L. (2017) · International Journal of Adaptive Control and Signal Processing · doi:10.1002/acs.2830 · Cited by 44
Self-citation
Data-driven optimal tracking control for discrete-time systems with delays using adaptive dynamic programming
Liu, Y., Zhang, H., Yu, R., & Qu, Q. (2018) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2018.06.013 · Cited by 44
A Framework for Autonomous Impedance Regulation of Robots Based on Imitation Learning and Optimal Control
Wu, Y., Zhao, F., Tao, T., & Ajoudani, A. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3033260 · Cited by 44
Data-driven output-feedback fault-tolerant control for unknown dynamic systems with faults changing system dynamics
Wang, J., & Yang, G. (2016) · Journal of Process Control · doi:10.1016/j.jprocont.2016.04.004 · Cited by 43
Data-Driven H ∞ Optimal Output Feedback Control for Linear Discrete-Time Systems Based on Off-Policy Q -Learning
Zhang, L., Fan, J., Xue, W., Lopez, V. G., Li, J., Chai, T., & Lewis, F. L. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3112457 · Cited by 41
Self-citation
A neural network based online learning and control approach for Markov jump systems
Zhong, X., He, H., Zhang, H., & Wang, Z. (2014) · Neurocomputing · doi:10.1016/j.neucom.2014.01.060 · Cited by 41
Incremental model based online heuristic dynamic programming for nonlinear adaptive tracking control with partial observability
Zhou, Y., Kampen, E.-J. V., & Chu, Q. P. (2020) · Aerospace Science and Technology · doi:10.1016/j.ast.2020.106013 · Cited by 41
Data-Driven Design and Optimization of Feedback Control Systems for Industrial Applications
Zhang, Y., Yang, Y. Z., Ding, S. X., & Li, L. (2014) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2301757 · Cited by 39
ADP based optimal tracking control for a class of linear discrete-time system with multiple delays
Liu, Y., Zhang, H., Luo, Y., & Han, J. (2016) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2016.03.012 · Cited by 40
Evolving intelligent system for the modelling of nonlinear systems with dead-zone input
Rubio, J. D. J., & Pérez-Cruz, J. H. (2013) · Applied Soft Computing · doi:10.1016/j.asoc.2013.03.018 · Cited by 39
Nonfragile Output Feedback Tracking Control for Markov Jump Fuzzy Systems Based on Integral Reinforcement Learning Scheme
Wang, J., Wu, J., Cao, J., Chadli, M., & Shen, H. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3203795 · Cited by 39
Data-based Suboptimal Neuro-control Design with Reinforcement Learning for Dissipative Spatially Distributed Processes
Luo, B., Wu, H., & Li, H. (2014) · Industrial & Engineering Chemistry Research · doi:10.1021/ie4031743 · Cited by 39
Incremental Approximate Dynamic Programming for Nonlinear Adaptive Tracking Control with Partial Observability
Zhou, Y., Kampen, E.-J. V., & Chu, Q. P. (2018) · Journal of Guidance Control and Dynamics · doi:10.2514/1.g003472 · Cited by 39
Stochastic linear quadratic optimal control for model-free discrete-time systems based on Q-learning algorithm
Wang, T., Zhang, H., & Luo, Y. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.04.018 · Cited by 38
Optimal dynamic output feedback control of unknown linear continuous-time systems by adaptive dynamic programming
Xie, K., Zheng, Y., Jiang, Y., Lan, W., & Yu, X. P. (2024) · Automatica · doi:10.1016/j.automatica.2024.111601 · Cited by 37
Output feedback reinforcement Q-learning control for the discrete-time linear quadratic regulator problem
Rizvi, S. A. A., & Lin, Z. (2017) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2017.8263836 · Cited by 37
Multilayer perception based reinforcement learning supervisory control of energy systems with application to a nuclear steam supply system
Dong, Z., Huang, X., Dong, Y., & Zhang, Z. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.114193 · Cited by 37
Robust controller design of continuous-time nonlinear system using neural network
Zhong, X., He, H., & Prokhorov, D. V. (2013) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2013.6707098 · Cited by 35
Design and HIL Realization of an Online Adaptive Dynamic Programming Approach for Real-Time Economic Operations of Household Energy Systems
Yuan, J., Yu, S. S., Zhang, G., Lim, C. P., Trinh, H. M., & Zhang, Y. (2021) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2021.3107447 · Cited by 34
Online Policy Learning-Based Output-Feedback Optimal Control of Continuous-Time Systems
Zhao, J., Lv, Y., Zeng, Q., & Wan, L. (2022) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2022.3211832 · Cited by 33
Optimal Transmission Power Scheduling of Networked Control Systems Via Fuzzy Adaptive Dynamic Programming
An, L., & Yang, G. (2020) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2020.2983658 · Cited by 33
Approximate Dynamic Programming for Optimal Stationary Control With Control-Dependent Noise
Jiang, Y., & Jiang, Z.-P. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2165729 · Cited by 33
A Novel Mean-Field-Game-Type Optimal Control for Very Large-Scale Multiagent Systems
Zhou, Z., & Xu, H. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3028267 · Cited by 32
Nonlinear control of a boost converter using a robust regression based reinforcement learning algorithm
Pradeep, D. J., Noel, M. M., & Arun, N. (2016) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2016.02.007 · Cited by 32
Fault Diagnosis of Energy Networks Based on Improved Spatial–Temporal Graph Neural Network With Massive Missing Data
Zhang, J., Cheng, Y., & He, X. Y. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3281394 · Cited by 31
Optimal tracking control for linear discrete-time systems using reinforcement learning
Kiumarsi, B., Lewis, F. L., Sistani, M. B. N., & Karimpour, A. (2013) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2013.6760476 · Cited by 32
Self-citation
Adaptive Dynamic Programming and Cooperative Output Regulation of Discrete-time Multi-agent Systems
Gao, W., Liu, Y., Odekunle, A., Yu, Y., & Lu, P. (2018) · International Journal of Control Automation and Systems · doi:10.1007/s12555-017-0635-8 · Cited by 31
Integral reinforcement learning off-policy method for solving nonlinear multi-player nonzero-sum games with saturated actuator
Ren, H., Zhang, H., Wen, Y., & Liu, C. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.01.033 · Cited by 31
Integral reinforcement learning-based optimal output feedback control for linear continuous-time systems with input delay
Gao, W., Luo, B., & Xue, S. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.06.073 · Cited by 31
Output Feedback Q-Learning for Linear-Quadratic Discrete-Time Finite-Horizon Control Problems
Calafiore, G. C., & Possieri, C. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3010304 · Cited by 30
Online integral reinforcement learning control for an uncertain highly flexible aircraft using state and output feedback
Peng, C., & Ma, J. (2020) · Aerospace Science and Technology · doi:10.1016/j.ast.2020.106442 · Cited by 30
Virtual State Feedback Reference Tuning and Value Iteration Reinforcement Learning for Unknown Observable Systems Control
Rădac, M., & Borlea, A.-I. (2021) · Energies · doi:10.3390/en14041006 · Cited by 30
Bringing new technologies and approaches to the operation and control of chemical process systems
Rawlings, J. B., & Maravelias, C. T. (2019) · AIChE Journal · doi:10.1002/aic.16615 · Cited by 28
Optimized Interaction Control for Robot Manipulator Interacting With Flexible Environment
Liu, X., Ge, S. S., Zhao, F., & Mei, X. (2020) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2020.3047919 · Cited by 31
Model-free H ∞ tracking control for de-oiling hydrocyclone systems via off-policy reinforcement learning
Li, S., Durdevic, P., & Yang, Z. (2021) · Automatica · doi:10.1016/j.automatica.2021.109862 · Cited by 29
Prescribed performance incremental adaptive optimal fault-tolerant control for nonlinear systems with actuator faults
Zhang, S., Huang, C., Ji, K., & Zhang, H. (2021) · ISA Transactions · doi:10.1016/j.isatra.2021.03.011 · Cited by 29
Data‐driven design of two‐degree‐of‐freedom controllers using reinforcement learning techniques
Zhang, Y., Ding, S. X., Yang, Y. Z., & Li, L. (2014) · IET Control Theory and Applications · doi:10.1049/iet-cta.2014.0156 · Cited by 29
Data-driven adaptive dynamic programming for partially observable nonzero-sum games viaQ-learning method
Wang, W., Chen, X., Fu, H., & Wu, M. (2019) · International Journal of Systems Science · doi:10.1080/00207721.2019.1599463 · Cited by 29
Data-Driven Homotopic Reinforcement Learning-Based Adaptive Optimal Control for Markov Jump Nonlinear Systems
Wang, H., Cheng, J. S., He, S., & Tang, S. (2025) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2023.3325658 · Cited by 28
Data-Efficient Multirobot, Multitask Transfer Learning for Trajectory Tracking
Pereida, K., Helwa, M. K., & Schoellig, A. P. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2795653 · Cited by 28
Adaptive model-free fault-tolerant control based on integral reinforcement learning for a highly flexible aircraft with actuator faults
Ma, J., & Peng, C. (2021) · Aerospace Science and Technology · doi:10.1016/j.ast.2021.107204 · Cited by 28
Predictive cruise control of connected and autonomous vehicles via reinforcement learning
Gao, W., Odekunle, A., Chen, Y., & Jiang, Z. (2018) · IET Control Theory and Applications · doi:10.1049/iet-cta.2018.6031 · Cited by 27
Q-Learning-Based Adaptive Power Control in Wireless RF Energy Harvesting Heterogeneous Networks
Zhang, R., Xiong, K., Guo, W., Yang, X. C., Fan, P., & Letaief, K. B. (2020) · IEEE Systems Journal · doi:10.1109/jsyst.2020.3015386 · Cited by 27
Reinforcement learning-based asymptotic cooperative tracking of a class multi-agent dynamic systems using neural networks
Cui, L., Wang, X., & Zhang, Y. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.06.066 · Cited by 27
Multiperson zero‐sum differential games for a class of uncertain nonlinear systems
Liu, D., & Wei, Q. (2012) · International Journal of Adaptive Control and Signal Processing · doi:10.1002/acs.2349 · Cited by 27
Data-Driven Output-Feedback Fault-Tolerant Tracking Control Method and Its Application to a DC Servo System
Wang, J., & Yang, G. (2019) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2019.2906666 · Cited by 26
Reinforcement Learning based on MPC/MHE for Unmodeled and Partially Observable Dynamics
Esfahani, H. N., Kordabad, A. B., & Gros, S. (2021) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc50511.2021.9483399 · Cited by 26
Space manipulator optimal impedance control using integral reinforcement learning
Wu, H. X., Hu, Q., Shi, Y., Zheng, J., Sun, K., & Wang, J. (2023) · Aerospace Science and Technology · doi:10.1016/j.ast.2023.108388 · Cited by 26
Data-driven real-time home energy management system based on adaptive dynamic programming
Yuan, J., Zeng, X., Zhou, J., Li, J., Lv, J., Chen, R., Chen, K., Yang, W., & Zhang, Y. (2024) · Electric Power Systems Research · doi:10.1016/j.epsr.2024.111055 · Cited by 25
Learning Without External Reward [Research Frontier]
He, H., & Zhong, X. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840727 · Cited by 25
Robust control design for multi-player nonlinear systems with input disturbances via adaptive dynamic programming
Qu, Q., Zhang, H., Luo, C., & Yu, R. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.11.054 · Cited by 25
Data-based and secure switched cyber–physical systems
Zhai, L., & Vamvoudakis, K. G. (2020) · Systems & Control Letters · doi:10.1016/j.sysconle.2020.104826 · Cited by 25
Self-citation
Performance Optimization of Maglev Train’s Electromagnetic Levitation System: Control Structure and Algorithm
Xu, Y., Zhao, Z., Long, Z., & Wen, T. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3262865 · Cited by 25
Self-Learning Takagi–Sugeno Fuzzy Control With Application to Semicar Active Suspension Model
Fang, H., Tu, Y., He, S., Wang, H., Sun, C., & Cheng, S. S. (2023) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2023.3290041 · Cited by 25
Batch-to-Batch Finite-Horizon LQ Control for Unknown Discrete-Time Linear Systems Via Stochastic Extremum Seeking
Liu, S.-J., Krstić, M., & Başar, T. (2016) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2016.2620379 · Cited by 25
Inverse Q-Learning Using Input–Output Data
Lian, B., Xue, W., Lewis, F. L., & Davoudi, A. A. (2023) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2023.3338197 · Cited by 24
Self-citation
Adaptive Critic Design for Pure-Feedback Discrete-Time MIMO Systems Preceded by Unknown Backlashlike Hysteresis
Tang, L. D., Liu, Y., & Chen, C. L. P. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2805689 · Cited by 24
Data-driven MIMO model-free reference tracking control with nonlinear state-feedback and fractional order controllers
Rădac, M., & Precup, R. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2018.09.035 · Cited by 24
Robust Adaptive Dynamic Programming
Jiang, Y., & Jiang, Z. (2012) · doi:10.1002/9781118453988.ch13 · Cited by 24
Extended incremental nonlinear dynamic inversion for optical flow control of micro air vehicles
Zhou, Y., Ho, H. W., & Chu, Q. P. (2021) · Aerospace Science and Technology · doi:10.1016/j.ast.2021.106889 · Cited by 22
Heuristic Dynamic Programming Using Echo State Network For Multivariable Tracking Control Of Wastewater Treatment Process
Bo, Y.-C., & Jun‐Fei, Q. (2014) · Asian Journal of Control · doi:10.1002/asjc.994 · Cited by 23
Data-Driven Fault Estimation and Control for Unknown Discrete-Time Systems via Multiobjective Optimization Method
Li, X., & Wang, N. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3010222 · Cited by 22
Hierarchical Cognitive Control for Unknown Dynamic Systems Tracking
Rădac, M., & Lala, T. (2021) · Mathematics · doi:10.3390/math9212752 · Cited by 22
Adaptive optimal control for a wastewater treatment plant based on a data-driven method
Qiao, J., Bo, Y.-C., Chai, W., & Han, H. (2013) · Water Science & Technology · doi:10.2166/wst.2013.087 · Cited by 22
Value iteration, adaptive dynamic programming, and optimal control of nonlinear systems
Bian, T., & Jiang, Z. (2016) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2016.7798777 · Cited by 22
Online reinforcement learning for a class of partially unknown continuous‐time nonlinear systems via value iteration
Su, H., Zhang, H., Zhang, K., & Gao, W. (2017) · Optimal Control Applications and Methods · doi:10.1002/oca.2391 · Cited by 22
Linear optimal tracking control: An adaptive dynamic programming approach
Gao, W., & Jiang, Z. (2015) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2015.7172106 · Cited by 21
Model-Free Output Consensus Control for Partially Observable Heterogeneous Multivehicle Systems
Sun, Y., Chen, X., Wang, W., Fu, H., & Wu, M. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2981654 · Cited by 21
Output‐feedback Q‐learning for discrete‐time linear H ∞ tracking control: A Stackelberg game approach
Ren, Y., Wang, Q., & Duan, Z. (2022) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.6169 · Cited by 21
Synchronous Reinforcement Learning-Based Control for Cognitive Autonomy
Vamvoudakis, K. G., & Kokolakis, N.-M. T. (2020) · Foundations and Trends® in Systems and Control · doi:10.1561/2600000022 · Cited by 21
Self-citation
Adaptive Optimal Discrete-Time Output-Feedback Using an Internal Model Principle and Adaptive Dynamic Programming
Wang, Z., Wang, Y., & Kowalczuk, Z. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123759 · Cited by 21
Learning Secure Control Design for Cyber-Physical Systems Under False Data Injection Attacks
Fei, C., Shen, J., Qiu, H., Zhang, Z., & Xing, W. (2024) · IEEE Transactions on Industrial Cyber-Physical Systems · doi:10.1109/ticps.2024.3373715 · Cited by 20
Model-free optimal controller design for continuous-time nonlinear systems by adaptive dynamic programming based on a precompensator
Zhang, J., Zhang, H., Liu, Z., & Wang, Y. (2015) · ISA Transactions · doi:10.1016/j.isatra.2014.08.018 · Cited by 20
Nearly data-based optimal control for linear discrete model-free systems with delays via reinforcement learning
Zhang, J., Zhang, H., Wang, B., & Cai, T. (2014) · International Journal of Systems Science · doi:10.1080/00207721.2014.941147 · Cited by 20
Robust approximate dynamic programming and global stabilization with nonlinear dynamic uncertainties
Jiang, Y., & Jiang, Z. (2011) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2011.6160279 · Cited by 20
Model-free control of unknown nonlinear systems using an iterative learning concept: theoretical development and experimental validation
Madadi, E., & Söffker, D. (2018) · Nonlinear Dynamics · doi:10.1007/s11071-018-4415-7 · Cited by 20
Reinforcement-Learning-Based Adaptive Optimal Flight Control with Output Feedback and Input Constraints
Sun, B., & Kampen, E.-J. V. (2021) · Journal of Guidance Control and Dynamics · doi:10.2514/1.g005715 · Cited by 20
Online identifier–actor–critic algorithm for optimal control of nonlinear systems
Lin, H., Wei, Q., & Liu, D. (2016) · Optimal Control Applications and Methods · doi:10.1002/oca.2259 · Cited by 20
Detection of actuator faults for continuous-time systems with intermittent state feedback
Fotiadis, F., & Vamvoudakis, K. G. (2021) · Systems & Control Letters · doi:10.1016/j.sysconle.2021.104938 · Cited by 19
Self-citation
Determining node duty cycle using Q-learning and linear regression for WSN
Huang, H. Y., Kim, K. T., & Youn, H. Y. (2020) · Frontiers of Computer Science · doi:10.1007/s11704-020-9153-6 · Cited by 19
Lifelong Learning-Based Optimal Trajectory Tracking Control of Constrained Nonlinear Affine Systems Using Deep Neural Networks
Ganie, I. A., & Jagannathan, S. (2024) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2024.3405354 · Cited by 19
Efficient Sensors Selection for Traffic Flow Monitoring: An Overview of Model-Based Techniques Leveraging Network Observability
Fabris, M., Ceccato, R., & Zanella, A. (2025) · Sensors · doi:10.3390/s25051416 · Cited by 19
H∞ Control of Nonaffine Aerial Systems Using Off-policy Reinforcement Learning
Kiumarsi, B., Kang, W., & Lewis, F. L. (2016) · Unmanned Systems · doi:10.1142/s2301385016400069 · Cited by 19
Self-citation
Output-Feedback Control of Unknown Linear Discrete-Time Systems With Stochastic Measurement and Process Noise via Approximate Dynamic Programming
Wang, J., & Yang, G. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2726004 · Cited by 19
Vision-Based Nonlinear Incremental Control for a Morphing Wing With Mechanical Imperfections
Sun, B., Mkhoyan, T., Kampen, E.-J. V., Breuker, R. D., & Wang, X. (2022) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2022.3175679 · Cited by 18
Output-Based Secure Control Under False Data Injection Attacks
Xu, L., Zhu, H., Guo, K., Gao, Y., & Wu, C. (2024) · IEEE Transactions on Industrial Cyber-Physical Systems · doi:10.1109/ticps.2024.3359154 · Cited by 18
Optimal bipartite consensus control for heterogeneous unknown multi-agent systems via reinforcement learning
Meng, H., Pang, D., Cao, J., Guo, Y., & Niazi, A. U. K. (2024) · Applied Mathematics and Computation · doi:10.1016/j.amc.2024.128785 · Cited by 18
Data-driven output-feedback fault-tolerant L 2 control of unknown dynamic systems
Wang, J., & Yang, G. (2016) · ISA Transactions · doi:10.1016/j.isatra.2016.04.014 · Cited by 18
Stochastic Optimal Control for Robot Manipulation Skill Learning Under Time-Varying Uncertain Environment
Liu, X., Liu, Z., & Huang, P. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3211440 · Cited by 17
Extremum seeking for optimal control problems with unknown time‐varying systems and unknown objective functions
Scheinker, A., & Scheinker, D. (2020) · International Journal of Adaptive Control and Signal Processing · doi:10.1002/acs.3097 · Cited by 16
Bridging Reinforcement Learning and Iterative Learning Control: Autonomous Motion Learning for Unknown, Nonlinear Dynamics
Meindl, M., Lehmann, D., & Seel, T. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.793512 · Cited by 16
Optimal Control for Continuous-Time Unknown Nonlinear Affine Systems: A Q -Learning Approach
Yu, S., Zhang, H., Ming, Z., & Sun, J. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3327264 · Cited by 16

Authors: Lewis, F. L., & Vamvoudakis, K. G.
Affiliations: The University of Texas at Arlington (US)
ORCID: Lewis, F L
Vamvoudakis, K G
Venue: IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2011-02
Volume: 41
Issue: 1
Pages: 14-25
ISSN: 1083-4419 (ISSN-L)
1941-0492
DOI: 10.1109/tsmcb.2010.2043839
OpenAlex: W2151966330
Cited by: 544
References: 48
Citations by year: 2026: 22 · 2025: 49 · 2024: 46 · 2023: 43 · 2022: 37 · 2021: 49 · 2020: 53 · 2019: 34 · 2018: 52 · 2017: 26 · 2016: 35 · 2015: 19 · 2014: 33 · 2013: 18 · 2012: 19
FWCI: 15.12
Type: journal article
Language: en
Open access: closed
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Locations: IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) (publishedVersion)
PubMed (publishedVersion)
Topics: Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
Keywords: approximate dynamic programming, partially observable Markov decision processes, output feedback control, linear dynamical systems, policy iteration, value iteration, Q-learning, observability index
MeSH terms: Algorithms, Artificial Intelligence, Feedback, Learning, Markov Chains, Reinforcement, Psychology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks