Cited by 544Adaptive Dynamic Programming ControlReinforcement Learning in RoboticsAdvanced Control Systems Optimization

The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.

Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data

Lewis, F L, Vamvoudakis, K G·2011·IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)

DOI 10.1109/tsmcb.2010.2043839

Open this paper in CoChat — free account

Abstract

Approximate dynamic programming (ADP) is a class of reinforcement learning methods that have shown their importance in a variety of applications, including feedback control of dynamical systems. ADP generally requires full information about the system internal states, which is usually not available in practical situations. In this paper, we show how to implement ADP methods using only measured input/output data from the system. Linear dynamical systems with deterministic behavior are considered herein, which are systems of great interest in the control system community. In control system theory, these types of methods are referred to as output feedback (OPFB). The stochastic equivalent of the systems dealt with in this paper is a class of partially observable Markov decision processes. We develop both policy iteration and value iteration algorithms that converge to an optimal controller that requires only OPFB. It is shown that, similar to Q -learning, the new methods have the important advantage that knowledge of the system dynamics is not needed for the implementation of these learning algorithms or for the OPFB control. Only the order of the system, as well as an upper bound on its "observability index," must be known. The learned OPFB controller is in the form of a polynomial autoregressive moving-average controller that has equivalent performance with the optimal state variable feedback gain.

Integrity signals

Registry

Crossref · 10.1109/tsmcb.2010.2043839

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Learning from Delayed Rewards
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,674
Neuro-dynamic Programming
(2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,601
Optimal Control
Lewis, F. L. (1986) · Cited by 2,957
Self-citation
Beyond regression : new fools for prediction and analysis in the behavioral sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,739
Approximate Dynamic Programming
Powell, W. B. (2011) · Wiley series in probability and statistics · doi:10.1002/9781118029176 · Cited by 2,359
Kronecker products and matrix calculus in system theory
Brewer, J. (1978) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1978.1084534 · Cited by 1,920
Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,722
Self-citation
Reinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,591
Self-citation

Flagged first, then by citations

Show all 48 references

Research works printed but not linked: 3

sutton, 1998, Reinforcement LearningAn Introduction
werbos, 1992, Handbook of Intelligent Control
1992, Handbook of Intelligent Control

Sorted by citations

Items 301–400 of 544

Application of deep reinforcement learning to networked control systems with uncertain network delays
Ikemoto, J., & Ushio, T. (2020) · Nonlinear Theory and Its Applications IEICE · doi:10.1587/nolta.11.480 · Cited by 6
Adaptive output‐feedback quadratic tracking control of continuous‐time systems via value iteration with its application
Peng, Y., Meng, Q., & Sun, W. (2020) · IET Control Theory and Applications · doi:10.1049/iet-cta.2020.0255 · Cited by 6
Robust Control of Uncertain Linear Systems Based on Reinforcement Learning Principles
Xu, D., Wang, Q., & Li, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2894594 · Cited by 6
Online adaptive optimal control for bilinear systems
Luo, B., & Wu, H. (2012) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2012.6314866 · Cited by 6
Model‐free distributed optimal control for general discrete‐time linear systems using reinforcement learning
Feng, X., Zhao, Z., & Yang, W. D. (2024) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.7280 · Cited by 5
A Homotopy Method for Continuous-Time Model-Free LQR Control Based on Policy Iteration
Fan, W., & Xiong, J. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2025.125132 · Cited by 5
Advancements and challenges of machine learning and deep learning in autonomous control of nuclear reactors
Hsieh, H.-Y., & Tsvetkov, P. V. (2025) · Annals of Nuclear Energy · doi:10.1016/j.anucene.2025.111643 · Cited by 6
On the effect of probing noise in optimal control LQR via Q-learning using adaptive filtering algorithms
Yánez, W. J. L., & de Souza, F. D. C. (2022) · European Journal of Control · doi:10.1016/j.ejcon.2022.100633 · Cited by 6
Safe adaptive output‐feedback optimal control of a class of linear systems
Mahmud, S. M. N., Abudia, M., Nivison, S., Bell, Z. I., & Kamalapurkar, R. (2024) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.7334 · Cited by 5
Continuous shared control in prosthetic hand grasp tasks by Deep Deterministic Policy Gradient with Hindsight Experience Replay
Gao, Z., Tang, R., Chen, L., Huang, Q., & He, J. (2020) · International Journal of Advanced Robotic Systems · doi:10.1177/1729881420936851 · Cited by 5
Data Driven State Reconstruction of Dynamical System Based on Approximate Dynamic Programming and Reinforcement Learning
da Silva, F. N., & Neto, J. V. D. F. (2021) · IEEE Access · doi:10.1109/access.2021.3080626 · Cited by 5
Control Policy Learning Design for Vehicle Urban Positioning via BeiDou Navigation
Qin, Y., Zhang, C., Chen, C., Xie, S., & Lewis, F. L. (2024) · Journal of Systems Science and Complexity · doi:10.1007/s11424-024-3357-z · Cited by 5
Self-citation
Data-driven virtual reference feedback tuning and reinforcement Q-learning for model-free position control of an aerodynamic system
Rădac, M., Precup, R., & Roman, R. (2016) · Mediterranean Conference on Control and Automation (MED) · doi:10.1109/med.2016.7535876 · Cited by 5
A subspace predictive control method for partially unknown systems with parameter learning event-triggered law
Li, Z., & Yang, G. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.04.050 · Cited by 5
Predictive Cruise Control of Connected and Autonomous Vehicles: An Adaptive Dynamic Programming Approach
Odekunle, A., Gao, W., Anayor, C., Wang, X., & Chen, Y. (2018) · IEEE SoutheastCon · doi:10.1109/secon.2018.8479020 · Cited by 5
Matrix Manifold-Based Performance Monitoring of Automatic Control Systems
Xu, Y., Yu, H., & Zhao, Z. (2022) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2022.3167864 · Cited by 5
Adaptive optimal control for linear stochastic systems with additive noise
Bian, T., & Zhong-Ping, J. (2015) · Chinese Control Conference (CCC) · doi:10.1109/chicc.2015.7260102 · Cited by 5
Data-based approximate policy iteration for nonlinear continuous-time optimal control designPreprint
Luo, B., Wu, H., Huang, T., & Liu, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1311.0396 · Cited by 5
Robust On-Line ADP-based Solution of a Class of Hierarchical Nonlinear Differential GamePreprint
Satouri, M., Kebriaei, H., Razminia, A., & Yazdanpanah, M. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.11414 · Cited by 5
Output feedback linear quadratic regulator design through one-shot Q-function learning
Asl, A. P., & Akbari, A. (2025) · International Journal of Control · doi:10.1080/00207179.2025.2497894 · Cited by 4
Output-feedback learning-based adaptive optimal control of nonlinear systems
Gao, W., Wang, Y., & Vamvoudakis, K. G. (2026) · Automatica · doi:10.1016/j.automatica.2026.112875 · Cited by 4
Self-citation
Online Output-Feedback Optimal Control of Linear Systems Based on Data-Driven Adaptive Learning
Zhao, J., Na, J., Gao, G., Han, S., Chen, Q., & Wang, S. (2020) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2020.12.2201 · Cited by 4
Adaptive Optimal Control of CVCF Inverters With Uncertain Load: An Adaptive Dynamic Programming Approach
Wang, Z., & Yu, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3090815 · Cited by 4
Learning-based near-optimal tracking control for industrial processes with slow and fast modes
Li, J., & Yang, M. (2023) · ISA Transactions · doi:10.1016/j.isatra.2023.06.021 · Cited by 4
Reinforcement Learning for Partially Observable Linear Gaussian Systems Using Batch Dynamics of Noisy Observations
Yaghmaie, F. A., Modares, H., & Gustafsson, F. (2024) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2024.3385680 · Cited by 4
Water Age Control for Water Distribution Networks via Safe Reinforcement Learning
Val, J., Wiśniewski, R., Kallesøe, C. S., & Tsouvalas, A. (2024) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2024.3426300 · Cited by 4
Value iteration and adaptive optimal control for linear continuous-time systems
Bian, T., & Jiang, Z. (2015) · IEEE Conference on Cybernetics and Intelligent Systems · doi:10.1109/iccis.2015.7274547 · Cited by 4
Robust adaptive dynamic programming for continuous-time linear stochastic systems
Bian, T., & Jiang, Z. (2014) · The proceedings of the IEEE Conference on Control Applications/The proceedings of the ... IEEE Conference on Control Applications · doi:10.1109/isic.2014.6967601 · Cited by 4
Optimal Digital Control with Uncertain Network Delay of Linear Systems Using Reinforcement Learning
Fujita, T., & Ushio, T. (2016) · IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences · doi:10.1587/transfun.e99.a.454 · Cited by 4
Adaptive Optimal Output Regulation of Discrete-time Linear Systems subject to Input Time-delay
Gao, W., & Jiang, Z. (2018) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.2018.8431175 · Cited by 4
Optimal Control for Water Distribution Networks with Unknown Dynamics
Val, J., Wiśniewski, R., & Kallesøe, C. S. (2020) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2020.12.075 · Cited by 4
Reward inference of discrete-time expert's controllers: A complementary learning approach
Perrusquía, A., & Guo, W. (2023) · Information Sciences · doi:10.1016/j.ins.2023.02.079 · Cited by 4
A novel stable value iteration-based approximate dynamic programming algorithm for discrete-time nonlinear systems
Qu, Y., Wang, A., & Lin, S. (2018) · Chinese Physics B · doi:10.1088/1674-1056/27/1/010203 · Cited by 4
Reinforcement learning-based optimal control considering L computation time delay of linear discrete-time systems
Fujita, T., & Ushio, T. (2014) · IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning · doi:10.1109/adprl.2014.7010632 · Cited by 4
Learning-based Event-triggered Adaptive Optimal Output Regulation of Linear Discrete-time Systems
Zhao, F., Gao, W., Liu, T., & Jiang, Z. (2021) · IEEE Data Driven Control and Learning Systems Conference (DDCLS) · doi:10.1109/ddcls52934.2021.9455453 · Cited by 4
Reinforcement learning: Application to PWR's core power level control
Kirgni, H. B. (2025) · Progress in Nuclear Energy · doi:10.1016/j.pnucene.2025.105644 · Cited by 3
Output-Feedback Control of Linear Continuous-Time Systems Using Discounted Inverse Reinforcement Learning
Wu, H., Hu, Q., Zheng, J., Shao, X., Liu, Y., & Li, D. (2026) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2026.3651519 · Cited by 3
Adaptive optimal control of discrete-time systems by reinforcement learning: damping coefficients-based stabilizing policy iterations
Li, D., & Dong, J. (2026) · Science China Information Sciences · doi:10.1007/s11432-025-4737-8 · Cited by 3
Output Feedback H∞Control of Unknown Discrete-time Linear Systems: Off-policy Reinforcement Learning
Tooranjipour, P., & Kiumarsi, B. (2021) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc45484.2021.9683057 · Cited by 3
Policy Iteration for Multiplicative Noise Output Feedback Control
Gravell, B., Gargiani, M., Lygeros, J., & Summers, T. H. (2022) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc51059.2022.9993098 · Cited by 3
Optimal Control of Unknown Discrete-Time Linear Systems with Additive Noise
Yang, X., & Liu, S. (2023) · Journal of Systems Science and Complexity · doi:10.1007/s11424-023-1352-4 · Cited by 3
Reinforcement Learning Based Controllers for Dynamical Systems
Singh, M., Rayguru, M. M., & Kaushik, G. (2023) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt56998.2023.10308258 · Cited by 3
Swarm-intelligence-based value iteration for optimal regulation of continuous-time nonlinear systems
Wang, D., Hu, Q., Liu, A., & Qiao, J. (2025) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2025.101913 · Cited by 3
An adaptive dynamic programming algorithm to solve optimal control of uncertain nonlinear systems
Cui, X., Luo, Y., & Zhang, H. (2014) · IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning · doi:10.1109/adprl.2014.7010643 · Cited by 3
Incremental approximate dynamic programming for nonlinear flight control design
Zhou, Y., Kampen, E.-J. V., & Chu, Q. P. (2015) · Research Repository (Delft University of Technology) · Cited by 3
Cooperative and Adaptive Optimal Output Regulation of Discrete-Time Multi-Agent Systems Using Reinforcement Learning
Gao, W., Liu, Y., Odekunle, A., Jiang, Z., Yu, Y., & Lu, P. (2018) · IEEE International Conference on Real-time Computing and Robotics (RCAR) · doi:10.1109/rcar.2018.8621852 · Cited by 3
Detection of Sensor Attacks in Uncertain Stochastic Linear Systems
Bhowmick, C., & Jagannathan, S. (2019) · IEEE Conference on Control Technology and Applications (CCTA) · doi:10.1109/ccta.2019.8920410 · Cited by 3
Optimal preview controller design for multirate systems based on adaptive dynamic programming
Wang, D., Ye, J., Suixiang, G., & Yang, S. (2021) · Chinese Control Conference (CCC) · doi:10.23919/ccc52363.2021.9549329 · Cited by 3
Real-Time Reinforcement Learning Control in Poor Experimental Conditions
Val, J., Wiśniewski, R., & Kallesøe, C. S. (2021) · European Control Conference (ECC) · doi:10.23919/ecc54610.2021.9654896 · Cited by 3
Crowd Prediction and Autonomous Navigation with Partial Observations
Li, K., Shan, M., Worrall, S., & Nebot, E. (2022) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc55140.2022.9922257 · Cited by 3
Reinforcement learning control with function approximation via multivariate simplex splines
Feng, Y., Zhou, Y., Ho, H. W., & Isa, N. A. M. (2023) · International Journal of Adaptive Control and Signal Processing · doi:10.1002/acs.3579 · Cited by 3
Derivative Feedback Control Using Reinforcement Learning
Zaheer, M. H., Yoon, S. Y., & Rizvi, S. A. A. (2023) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc49753.2023.10383943 · Cited by 3
Optimal fuzzy output feedback tracking control for unmanned surface vehicles systems
Song, W., Zuo, Y., & Tong, S. (2024) · Ocean Engineering · doi:10.1016/j.oceaneng.2024.117034 · Cited by 3
Bridging the Gap Between Reinforcement Learning and Nonlinear Output-Feedback Control
Gao, W., Jiang, Z., & Chai, T. (2024) · Chinese Control Conference (CCC) · doi:10.23919/ccc63176.2024.10661387 · Cited by 3
Stochastic Adaptive Dynamic Programming for Robust Optimal Control Design
Bian, T., & Jiang, Z. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-805246-4.00007-0 · Cited by 3
A simulation-based approach to develop a holonic robotic cell
Steed, C. A. (2019) · Industrial Robot the international journal of robotics research and application · doi:10.1108/ir-07-2018-0149 · Cited by 3
Tuning heuristics and convergence analysis of reinforcement learning algorithm for online data-based optimal control design
da Silva, F. N., & Neto, J. V. D. F. (2020) · Research Society and Development · doi:10.33448/rsd-v9i2.2128 · Cited by 3
A learning-based optimal tracking controller for continuous linear systems with unknown dynamics: Theory and case study
Zhang, J., Wang, Q., & Wang, T. (2020) · Measurement and Control · doi:10.1177/0020294020915213 · Cited by 3
Adaptive Online Data-Driven Tracking Control for Highly Flexible Aircrafts With Partial Observability
Peng, C., & Ma, J. (2020) · IEEE Access · doi:10.1109/access.2020.3031891 · Cited by 3
RL-based optimal networked control considering network delay of discrete-time linear systems
Fujita, T., & Ushio, T. (2015) · European Control Conference (ECC) · doi:10.1109/ecc.2015.7330910 · Cited by 3
Networked Control of Nonlinear Systems under Partial Observation Using Continuous Deep Q-Learning
Ikemoto, J., & Ushio, T. (2019) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc40024.2019.9029214 · Cited by 3
Quantized measurements in Q-learning based model-free optimal control
Tiistola, S., Ritala, R. K., & Vilkko, M. K. (2020) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2020.12.2219 · Cited by 3
A Decentralized LQR Output Feedback Control for Aero-Engines
Ji, X., Li, J., Ren, J., & Wu, Y. (2023) · Actuators · doi:10.3390/act12040164 · Cited by 3
A deep reinforcement learning control framework for a partially observable system: experimental validation on a rotary flexible link system
Kumar, V. J., & Elumalai, V. K. (2025) · International Journal of Systems Science · doi:10.1080/00207721.2025.2468870 · Cited by 2
Dynamic Output Feedback Linear Quadratic Control for CPSs Under Sparse Attacks
Zhang, M., & Lin, C. (2025) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2025.3555018 · Cited by 2
Optimal output feedback event-triggered tracking control for Takagi-Sugeno fuzzy systems
Song, W., Zuo, Y., & Tong, S. (2025) · Fuzzy Sets and Systems · doi:10.1016/j.fss.2025.109565 · Cited by 2
Multiplayer Cascaded Policy Iteration for Nash Differential Games
Chen, Y., Chen, C., Lewis, F. L., & Xie, S. (2025) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2025.3605273 · Cited by 2
Self-citation
Data-driven adaptive optimal output tracking with assured convergence rate
Yang, S., Zhao, H., Ogura, M., & Peng, L. (2025) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2025.108086 · Cited by 2
Reinforcement Learning for Robot Optimal Impedance Control: Theoretical Feasibility and Extensibility Analysis
Wu, H. X., Hu, Q., Zheng, J., Shao, X., Liu, Y., & Li, D. (2025) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2025.3607988 · Cited by 2
Prescribed-rate target tracking for time-delayed systems using output measurements
Zhang, C., Chen, C., Lewis, F. L., Xie, K., & Xie, S. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108223 · Cited by 2
Self-citation
Adaptive optimal control for nonlinear networked systems under encoding-decoding mechanism: A generalized policy iteration approach
Li, X., Li, X., & Han, H. (2025) · Information Sciences · doi:10.1016/j.ins.2025.122890 · Cited by 2
Input–output data-driven sensor selection for cyber-physical systems
Fotiadis, F., & Vamvoudakis, K. G. (2026) · Automatica · doi:10.1016/j.automatica.2026.112829 · Cited by 2
Self-citation
Safety Aware Continual Reinforcement Learning-Based Output Tracking Control of Nonlinear Continuous-Time Systems
Ganie, I. A., & Jagannathan, S. (2026) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2025.3647584 · Cited by 2
Adaptive Dynamic Programming and Its Application to Economic Dispatch in Microgrid: A Brief Overview
Chen, Z., Deng, Q., & Chen, K. (2022) · Journal of Advances in Applied & Computational Mathematics · doi:10.15377/2409-5761.2022.09.2 · Cited by 2
$H_{\infty}$ Tracking Control of Unknown Discrete- Time Linear Systems via Output-Data-Driven Off-policy Q-learning Algorithm
Zhang, K., Liu, X., Zhang, L., Chen, Q., & Peng, Y. (2022) · Chinese Control Conference (CCC) · doi:10.23919/ccc55666.2022.9901898 · Cited by 2
Fixed-Point Control of Airships Based on a Characteristic Model: A Data-Driven Approach
Chen, Y., Shen, S., Hu, Z., & Huang, L. (2023) · Mathematics · doi:10.3390/math11020310 · Cited by 2
A Nonlinear Critical Surface With Input Constraints on Costate-Adaptive Dynamic Programming
Zhao, G., Sun, J., Yan, Y., & Zhang, H. (2023) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2023.3266431 · Cited by 2
Learning-based Prescribed Rate Design for Output Regulation of Discrete-time Systems
Zhang, C., Chen, C., & Xie, S. (2023) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc58219.2023.10326503 · Cited by 2
Data‐driven based optimal output feedback control with low computation cost
Yu, X., Yu, J., Zhang, Y., Wu, J., Yan, W. M., & Ou, L. (2024) · International Journal of Adaptive Control and Signal Processing · doi:10.1002/acs.3832 · Cited by 2
Action-dependent heuristic dynamic programming for hybrid-order systems based on dynamic event-triggered mechanism
Li, J. Q., Li, H., Zheng, L., Ran, L., & Ji, L. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128171 · Cited by 2
Destination Floors-Oriented Group Elevators Dispatching Algorithm Based on Q-Learning and Simulated Annealing Method
Liang, R., Peng, Y., & Zhang, K. (2024) · IEEE Data Driven Control and Learning Systems Conference (DDCLS) · doi:10.1109/ddcls61622.2024.10606744 · Cited by 2
Safety Control for Cyber–Physical Systems Under False Data Injection Attacks
Xu, L., Zhu, Y., Li, Z., & Zhang, Q. (2025) · Electronics · doi:10.3390/electronics14061103 · Cited by 2
Learning-Based State Estimation for Automated Lane-Changing
Ha, W. Y., Chakraborty, S., Lin, X., Özbay, K. M. A., & Jiang, Z. (2024) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc58415.2024.10919550 · Cited by 2
Resilient Learning-Based Control for Partially Observable Systems Under DoS Attacks
Chakraborty, S., & Jiang, Z.-P. (2025) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2025.12.166 · Cited by 2
Data-Driven Based Optimal Output-Feedback Control of Continuous-Time Systems
Li, Z., Wu, T., Na, J., Zhao, J., Gao, G., & Herrmann, G. (2018) · International Conference on Modelling, Identification and Control · doi:10.1109/icmic.2018.8529962 · Cited by 2
Data-driven optimal control with reduced output measurements
Modares, H., Gan, O. P., Zhu, L., Lewis, F. L., & Yue, B. (2014) · World Congress on Intelligent Control and Automation · doi:10.1109/wcica.2014.7052989 · Cited by 2
Self-citation
Fixed Final Time Optimal Adaptive Control of Linear Discrete-Time Systems in Input-Output form
Zhao, Q., Xu, H., & Jagannathan, S. (2013) · Journal of Artificial Intelligence and Soft Computing Research · doi:10.2478/jaiscr-2014-0012 · Cited by 2
A data-driven methodology for solving the control strategy of descriptor systems
Zhang, D., Li, M., & Li, J. (2013) · Asian Control Conference (ASCC) · doi:10.1109/ascc.2013.6606163 · Cited by 2
Semi-global Feedback Control System Design in an Unknown Plane at an Unknown Destination
AKIBA, H., Nakamura, H., & Nakamura, N. (2014) · Transactions of the Society of Instrument and Control Engineers · doi:10.9746/sicetr.50.775 · Cited by 2
A New Approach for a Class of Continuous-Time Chaotic Systems Optimal Control by Online ADP Algorithm
Wei, Q., Song, R., Li, B., & Lin, X. (2017) · Studies in systems, decision and control · doi:10.1007/978-981-10-4080-1_8 · Cited by 2
Data-Driven Design of Feedback-Feedforward Control Systems for Dynamic Processes
Hua, C., Shardt, Y. A. W., Ding, S. X., & Wang, Y. (2017) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2017.08.2211 · Cited by 2
Autonomous Navigation in Partially Observable Environments Using Hierarchical Q-Learning
Zhou, Y., Kampen, E.-J. V., & Chu, Q. P. (2016) · Research Repository (Delft University of Technology) · Cited by 2
Comparison of Two Performance Optimization Approaches for Data-Driven Design of Fault-Tolerant Control Systems
Xie, H., Hu, C., Zhang, Y., Yang, X., & Li, Y. (2018) · IEEE International Symposium on Industrial Electronics (ISIE) · doi:10.1109/isie.2018.8433657 · Cited by 2
Output Feedback Reinforcement Q-learning for Optimal Quadratic Tracking Control of Unknown Discrete-Time Linear Systems and Its Application
Zhao, G., Sun, W., Cai, H. G., & Peng, Y. (2018) · International Conference on Control, Automation, Robotics and Vision (ICARCV) · doi:10.1109/icarcv.2018.8581252 · Cited by 2
Intelligent Adaptive Control Using LADP and IADP Applied to F-16 Aircraft with Imperfect Measurements
Cárdenas, J. I. D. A., Sun, B., & Kampen, E.-J. V. (2021) · AIAA SciTech Forum · doi:10.2514/6.2021-1119 · Cited by 2
An N-step Look Ahead Algorithm Using Mixed (On and Off) Policy Reinforcement Learning
Kuchibhotla, V., Harshitha, P., & Goyal, S. (2020) · International Conference on Intelligent Sustainable Systems (ICISS) · doi:10.1109/iciss49785.2020.9315959 · Cited by 2
Reinforcement Learning Control for Water Distribution Networks with Periodic Disturbances
Val, J., Wiśniewski, R., & Kallesøe, C. S. (2021) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc50511.2021.9482787 · Cited by 2
A Reinforcement Learning-Based Control Approach for Unknown Nonlinear Systems with Persistent Adversarial Inputs
Zhong, X., & He, H. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534429 · Cited by 2
Off-Policy Actor-Critic Structure for Optimal Control of Unknown Systems with Disturbances
Song, R., Wei, Q., & Li, Q. (2018) · Studies in systems, decision and control · doi:10.1007/978-981-13-1712-5_9 · Cited by 2
Model-Free Design of Linear Quadratic Regulator
Rizvi, S. A. A., & Lin, Z. (2022) · Control engineering · doi:10.1007/978-3-031-15858-2_2 · Cited by 2

Authors: Lewis, F. L., & Vamvoudakis, K. G.
Affiliations: The University of Texas at Arlington (US)
ORCID: Lewis, F L
Vamvoudakis, K G
Venue: IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2011-02
Volume: 41
Issue: 1
Pages: 14-25
ISSN: 1083-4419 (ISSN-L)
1941-0492
DOI: 10.1109/tsmcb.2010.2043839
OpenAlex: W2151966330
Cited by: 544
References: 48
Citations by year: 2026: 22 · 2025: 49 · 2024: 46 · 2023: 43 · 2022: 37 · 2021: 49 · 2020: 53 · 2019: 34 · 2018: 52 · 2017: 26 · 2016: 35 · 2015: 19 · 2014: 33 · 2013: 18 · 2012: 19
FWCI: 15.12
Type: journal article
Language: en
Open access: closed
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Locations: IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) (publishedVersion)
PubMed (publishedVersion)
Topics: Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
Keywords: approximate dynamic programming, partially observable Markov decision processes, output feedback control, linear dynamical systems, policy iteration, value iteration, Q-learning, observability index
MeSH terms: Algorithms, Artificial Intelligence, Feedback, Learning, Markov Chains, Reinforcement, Psychology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks