Cited by 544Adaptive Dynamic Programming ControlReinforcement Learning in RoboticsAdvanced Control Systems Optimization

The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.

Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data

Lewis, F L, Vamvoudakis, K G·2011·IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)

DOI 10.1109/tsmcb.2010.2043839

Open this paper in CoChat — free account

Abstract

Approximate dynamic programming (ADP) is a class of reinforcement learning methods that have shown their importance in a variety of applications, including feedback control of dynamical systems. ADP generally requires full information about the system internal states, which is usually not available in practical situations. In this paper, we show how to implement ADP methods using only measured input/output data from the system. Linear dynamical systems with deterministic behavior are considered herein, which are systems of great interest in the control system community. In control system theory, these types of methods are referred to as output feedback (OPFB). The stochastic equivalent of the systems dealt with in this paper is a class of partially observable Markov decision processes. We develop both policy iteration and value iteration algorithms that converge to an optimal controller that requires only OPFB. It is shown that, similar to Q -learning, the new methods have the important advantage that knowledge of the system dynamics is not needed for the implementation of these learning algorithms or for the OPFB control. Only the order of the system, as well as an upper bound on its "observability index," must be known. The learned OPFB controller is in the form of a polynomial autoregressive moving-average controller that has equivalent performance with the optimal state variable feedback gain.

Integrity signals

Registry

Crossref · 10.1109/tsmcb.2010.2043839

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Learning from Delayed Rewards
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,674
Neuro-dynamic Programming
(2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,601
Optimal Control
Lewis, F. L. (1986) · Cited by 2,957
Self-citation
Beyond regression : new fools for prediction and analysis in the behavioral sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,739
Approximate Dynamic Programming
Powell, W. B. (2011) · Wiley series in probability and statistics · doi:10.1002/9781118029176 · Cited by 2,359
Kronecker products and matrix calculus in system theory
Brewer, J. (1978) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1978.1084534 · Cited by 1,920
Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,722
Self-citation
Reinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,591
Self-citation

Flagged first, then by citations

Show all 48 references

Research works printed but not linked: 3

sutton, 1998, Reinforcement LearningAn Introduction
werbos, 1992, Handbook of Intelligent Control
1992, Handbook of Intelligent Control

Sorted by citations

Items 401–500 of 544

Linear Quadratic Gaussian using Kalman Network and Reinforcement Learning for Discrete-Time System
Putri, A. N., Machbub, C., Mahayana, D., & Hidayat, E. (2022) · International Conference on System Engineering and Technology · doi:10.1109/icset57543.2022.10010824 · Cited by 2
Two-step reinforcement learning for model-free redesign of nonlinear optimal regulator
Minami, M., Masumoto, Y., Okawa, Y., Sasaki, T., & Hori, Y. (2023) · SICE Journal of Control Measurement and System Integration · doi:10.1080/18824889.2023.2278753 · Cited by 2
Sliding Mode Control Based Output Feedback Adaptive Dynamic Programming Algorithm for Uncertain Wheel Inverted Pendulum Systems
Dao, P. N., & Quang, N. H. (2021) · Advances in intelligent systems and computing · doi:10.1007/978-981-15-7527-3_92 · Cited by 2
Extremum Seeking Optimal Controls of Unknown SystemsPreprint
Scheinker, A., & Scheinker, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.05181 · Cited by 2
D3PI: Data-Driven Distributed Policy Iteration for Homogeneous Interconnected Systems.Preprint
Alemzadeh, S., Talebi, S., & Mesbahi, M. (2021) · arXiv (Cornell University) · Cited by 2
Multi-agent optimization and learning methods for sustainable, smart and resilient power distribution systems and microgrids
Zhang, Q. (2022) · doi:10.31274/td-20240329-526 · Cited by 2
Model-Free H∞ Disturbance Rejection and Linear Quadratic Zero-Sum Games
Rizvi, S. A. A., & Lin, Z. (2022) · Control engineering · doi:10.1007/978-3-031-15858-2_3 · Cited by 1
Design and Experimental Validation of a Model-Free Controller for Beam Stabilization in Adaptive Optics Systems
Guo, S., Cheng, T., Yang, K., Kong, L., Su, C., Wang, S., & Yang, P. (2024) · IEEE photonics journal · doi:10.1109/jphot.2024.3471827 · Cited by 1
Incremental value iteration for optimal output regulation of linear systems with unknown exosystems
Jing, C., Wang, C., Liang, D., Xu, Y., & Hao, L. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129579 · Cited by 1
Input-Output Data-Driven Sensor Selection
Fotiadis, F., & Vamvoudakis, K. G. (2024) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc56724.2024.10886647 · Cited by 1
Self-citation
Fault-tolerant Q-learning for discrete-time linear systems with actuator and sensor faults using input-output measured data
Kankashvar, M., Rafiee, S., & Bolandi, H. (2025) · Franklin Open · doi:10.1016/j.fraope.2025.100259 · Cited by 1
Data-driven learning for output-feedback robust control of systems with unmatched uncertainties
Jia, B., Zhao, J., Huang, Y., Lv, Y., Zeng, Q., & Wan, L. (2025) · International Journal of Systems Science · doi:10.1080/00207721.2025.2520346 · Cited by 1
Information-State-Based Reinforcement Learning for the Control of Partially Observed Nonlinear Systems
Goyal, R., Mohamed, M. N. G., Wang, R., Sharma, A., & Chakravorty, S. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3593259 · Cited by 1
Model-Free Q-Learning for Output Feedback Nash Strategy of Decentralized Nonzero-Sum Games
Zhang, Q., Wang, H., Peng, K., & Zhang, H. (2025) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2025.3600349 · Cited by 1
Reinforcement Learning Based Adaptive Output Feedback Control of Uncertain Nonlinear Systems
Zhou, N., Liang, X., Deng, W., Yao, J., Li, D., & Ha, L. (2025) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.70194 · Cited by 1
A multi-event-triggered control framework for nonzero-sum differential games of linear systems
Kaynama, S., & Montazeri, M. (2025) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2025.108363 · Cited by 1
A Review on Resilient Reinforcement Learning for Linear Cyber-Physical Systems
Chakraborty, S., Gao, W., & Jiang, Z.-P. (2026) · Journal of Systems Science and Complexity · doi:10.1007/s11424-026-5502-3 · Cited by 2
Robotic Systems Tracking Error Measurement: A Data-Driven Output-Feedback Control Method
Zhao, J., Sun, Q., & Jia, B. (2026) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2026.3693455 · Cited by 1
Near optimal event-triggered control of nonlinear continuous-time systems using input and output data
Xu, H., & Jagannathan, S. (2014) · World Congress on Intelligent Control and Automation · doi:10.1109/wcica.2014.7052993 · Cited by 1
Adaptive dynamic programming for nonlinear nonaffine systems
Bian, T., Jiang, Y., & Jiang, Z. (2014) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2014.7039949 · Cited by 1
Robust adaptive dynamic programming for continuous-time linear and nonlinear systems
Jiang, Y. (2014) · Cited by 1
Linear Quadratic Gaussian Control for UDP-Like Systems
Lin, H. S., Su, H., Shi, P. J., Shu, Z., & Wu, Z. (2016) · Studies in systems, decision and control · doi:10.1007/978-3-319-44212-9_6 · Cited by 1
Multi input-multi output tank system data-driven model reference control
Rădac, M., Precup, R., & Roman, R. (2017) · International Conference on Control and Automation · doi:10.1109/icca.2017.8003211 · Cited by 1
Multiple Actor-Critic Optimal Control via ADP
Song, R., Wei, Q., & Li, Q. (2018) · Studies in systems, decision and control · doi:10.1007/978-981-13-1712-5_5 · Cited by 1
Asymmetric constrained control scheme design with discrete output feedback in unknown robot–environment interaction system
Yu, X., Luo, H., Shi, S., Yan, W. Q., & Ou, L. (2022) · Robotica · doi:10.1017/s0263574722001138 · Cited by 1
Model-Free Optimal Tracking Control and Multi-Agent Synchronization
Rizvi, S. A. A., & Lin, Z. (2022) · Control engineering · doi:10.1007/978-3-031-15858-2_6 · Cited by 1
Data-driven Reinforcement Learning for Linear Quadratic Control of Unknown Systems over Lossy Channels
Zhang, L., Xiao, L., Wang, J., & Chen, M. Z. Q. (2022) · IEEE International Conference on Automation, Electronics and Electrical Engineering (AUTEEE) · doi:10.1109/auteee56487.2022.9994506 · Cited by 1
Unconstrained feedback controller design using Q-learning from noisy process data
Kumar, P., & Rawlings, J. B. (2023) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2023.108325 · Cited by 1
Learning-Based Optimal Impedance Control for Space Manipulator Contact Tasks
Wu, H., Sun, K., Hu, Q., Shi, Y., Zheng, J., & Wang, J. (2023) · International Conference on Control, Automation and Robotics (ICCAR) · doi:10.1109/iccar57134.2023.10151722 · Cited by 1
A Model-Free Iteration Algorithm for Markov Jump Linear Systems Based on Gauss-Seidel Method
Fan, W., & Xiong, J. (2023) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc49753.2023.10383557 · Cited by 1
The Adaptive Optimal Output Feedback Tracking Control of Unknown Discrete-Time Linear Systems Using a Multistep Q-Learning Approach
Dong, X., Lin, Y., Suo, X., Wang, X., & Sun, W. (2024) · Mathematics · doi:10.3390/math12040509 · Cited by 1
Data-Driven Structured Policy Iteration for Homogeneous Distributed Systems
Alemzadeh, S., Talebi, S., & Mesbahi, M. (2024) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2024.3366038 · Cited by 2
Reinforcement Learning With Adjustable Convergence Rate for Data-Based Nonlinear Control
Tang, G., Wang, D., Li, X., Ren, J., & Liu, N. (2024) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc62350.2024.10587338 · Cited by 1
Output feedback control of anti‐linear systems using adaptive dynamic programming
Yu, L., & Wang, H. R. (2024) · Optimal Control Applications and Methods · doi:10.1002/oca.3203 · Cited by 1
Adaptive Optimal Output-Feedback Control of Discrete-Time Systems based on Hybrid Iteration
Liu, S., Gao, W., & Jiang, Z. (2024) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc56724.2024.10886087 · Cited by 1
Performance-based fault diagnosis using model-free reinforcement learning
Ma, H., Li, L., & Yang, Y. Z. (2025) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2025.107854 · Cited by 1
Lyapunov based optimal control of a class of nonlinear systems
Zargarzadeh, H. (2012) · Cited by 1
The adaptive dynamic programming based on the data - A total least squares approach
Zhang, B., Da-qing, Z., Yi-fa, Y., & Gang-gang, Z. (2014) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc.2014.6852807 · Cited by 1
Improving model reference control performance using model-free VRFT and Q-learning
Rădac, M., & Precup, R. (2016) · International Conference on System Theory, Control and Computing (ICSTCC) · doi:10.1109/icstcc.2016.7790632 · Cited by 1
Data-driven optimal control for a class of unknown continuous-time nonlinear system using a novel ADP method
Zhang, K., Zhang, H., Jiang, H., & Liu, C. (2016) · International Conference on Intelligent Control and Information Processing · doi:10.1109/icicip.2016.7885887 · Cited by 1
Reinforcement Learning-Based Adaptive Optimal Control for Partially Unknown Systems Using Differentiator
Guo, X., Yan, W., & Cui, R. (2018) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.2018.8431133 · Cited by 1
Off-Policy Integral Reinforcement Learning for Semi-Global Constrained Output Regulation of Continuous-Time Linear Systems
Yang, Y., Chen, X., Yin, Y., & Wunsch, D. C. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489343 · Cited by 1
Improved Q-Learning Method for Linear Discrete-Time Systems
Chen, J., Wang, J., & Huang, J. (2020) · Processes · doi:10.3390/pr8030368 · Cited by 1
Data-Driven Fault-Tolerant Control Schemes
Ding, S. X. (2020) · Springer eBooks · doi:10.1007/978-3-662-62004-5_22 · Cited by 1
Data- Driven Process Monitoring Schemes for Industrial Processes based on Performance Degradation Prediction
Li, L. (2020) · Youth Academic Annual Conference of Chinese Association of Automation (YAC) · doi:10.1109/yac51587.2020.9337606 · Cited by 1
Reinforcement Learning for Optimal Adaptive Control of Time Delay Systems
Rizvi, S. A. A., Wei, Y., & Lin, Z. (2021) · Studies in systems, decision and control · doi:10.1007/978-3-030-60990-0_8 · Cited by 1
Adaptive spacecraft attitude control with incremental approximate dynamic programming
Zhou, Y., Kampen, E.-J. V., & Chu, Q. P. (2017) · Research Repository (Delft University of Technology) · Cited by 1
Model-Free Perception-Based Control via Q-Learning with an Application to Heat-Seeking Missile Guidance
Kovalik, W. S., Zhai, L., & Vamvoudakis, K. G. (2021) · IEEE Conference on Control Technology and Applications (CCTA) · doi:10.1109/ccta48906.2021.9658723 · Cited by 1
Self-citation
Online Learning Algorithms for Optimal Control and Dynamic Games
Vamvoudakis, K. G., & Lewis, F. L. (2012) · doi:10.1002/9781118453988.ch16 · Cited by 1
Self-citation
Adaptive Dynamic Programming for Uncertain Linear Systems
Jiang, Y., & Jiang, Z. (2017) · Wiley eBooks · doi:10.1002/9781119132677.ch2 · Cited by 1
A Supplementary Condition for the Convergence of the Control Policy during Adaptive Dynamic ProgrammingPreprint
Bao, X., Mao, Z., & Sharma, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.07743 · Cited by 1
Generalized State-Feedback Controller Synthesis for Underactuated Systems through Bayesian OptimizationPreprint
Solís, M. A., & Thomas, S. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.17158 · Cited by 1
Neural Control and Approximate Dynamic Programming
Lewis, F. L., & Vamvoudakis, K. G. (2014) · Encyclopedia of Systems and Control · doi:10.1007/978-1-4471-5102-9_224-2 · Cited by 1
Self-citation
Bridging Reinforcement Learning and Iterative Learning Control: Autonomous Reference Tracking for Unknown, Nonlinear DynamicsPreprint
Meindl, M., Lehmann, D., & Seel, T. (2021) · doi:10.36227/techrxiv.15022497 · Cited by 1
Neural Control and Approximate Dynamic Programming
Lewis, F. L. (2014) · Encyclopedia of Systems and Control · doi:10.1007/978-1-4471-5102-9_224-1
Self-citation
Multiple data-based ADP structures to solve the infinite horizon optimal control problem
Song, R., & Wei, Q. (2015) · International Conference on Information Science and Technology · doi:10.1109/icist.2015.7289023
Higher-level application of Adaptive Dynamic Programming/Reinforcement Learning - a next phase for controls and system identification?
Lendaris, G. G. (2011) · IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning · doi:10.1109/adprl.2011.5967395
Optimal Control for a Class of Nonlinear Discrete- Time Systems With Control Constraints Using the Iterative GDHP Algorithm
Liu, D., Wang, D., & Zhao, D. (2011)
A New Approach to Robust and Optimal Nonlinear Control Design
Jiang, Z., & Jiang, Y. (2013) · IASTED International Conference on Modelling, Identification and Control (MIC) · doi:10.2316/p.2013.799-029
Adaptive Critic Design with Local Gaussian Process Models
Wang, W., Chen, X., & Jian-xin, H. (2016) · Journal of Advanced Computational Intelligence and Intelligent Informatics · doi:10.20965/jaciii.2016.p1135
On-Line Adaptive Dynamic Programming for Feedback Control
Zhong, X. (2017) · Digital Commons - URI (University of Rhode Island) · doi:10.23860/diss-zhong-xiangnan-2017
Efficient Multi-Task and Multi-Robot Transfer with Continued Learning.
Pereida, K., Helwa, M. K., & Schoellig, A. P. (2017)
Derivation of integrated state equation for combined outputs-inputs vector of discrete-time linear time-invariant system and its application to reinforcement learning
Sasaki, T., Uchibe, E., Iwane, H., Yanami, H., Anai, H., & Doya, K. (2017) · Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE) · doi:10.23919/sice.2017.8105538
A Theoretical Difficulty in Approximate Dynamic Programming with Input ConstraintsPreprint
Bao, X., Mao, Z., & Sharma, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06424
Model-free semi-global output regulation for discrete-time linear systems subject to input amplitude saturation
Yang, Y., Ding, D., Yin, Y., & Wunsch, D. C. (2018) · Youth Academic Annual Conference of Chinese Association of Automation (YAC) · doi:10.1109/yac.2018.8406363
Online optimal consensus control based on PI algorithm for nonlinear multi-agent system with completely unknown dynamics
Sun, J., & Li, J. (2018) · Chinese Automation Congress · doi:10.1109/cac.2018.8623176
Power Management of Networked Microgrids using A Reinforcement Learning ApproachPreprint
Zhang, Q., Dehghanpour, K., Wang, Z., & Huang, Q. (2018) · arXiv (Cornell University)
Reinforcement Learning, Intelligent Control and their Applications in Connected and Autonomous Vehicles
Odekunle, A. (2019)
IRL Method for Time-Continuous Two-Player Nonzero Sum Game of Unknown System with Constrained-Input
Ren, H., Zhang, H., Wen, Y., & Mu, Y. (2019) · IEEE Data Driven Control and Learning Systems Conference (DDCLS) · doi:10.1109/ddcls.2019.8908924
Extremum Seeking for Creating Optimal Feedback Controls of Unknown Systems by Tuning Basis Functions
Scheinker, A., & Scheinker, D. (2020) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc45564.2020.9147549
Neuro-Inspired Control
Lewis, F. L., & Vamvoudakis, K. G. (2020) · Encyclopedia of Systems and Control · doi:10.1007/978-1-4471-5102-9_224-3
Self-citation
A prescribed performance adaptive optimal control scheme for flying-wing aircraft
Huang, C., & Zhang, S. (2020) · International Conference on Unmanned Aircraft Systems (ICUAS) · doi:10.1109/icuas48674.2020.9213966
Continuous Deep Q-Learning with Simulator for Stabilization of Uncertain Discrete-Time SystemsPreprint
Ikemoto, J., & Ushio, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.05640
Reinforcement Learning Based Adaptive Optimal Strategy in Robotic Control Systems
Dao, P. N., & Quang, N. H. (2021) · Advances in intelligent systems and computing · doi:10.1007/978-981-15-7527-3_90
Output Feedback Adaptive Reinforcement Learning Tracking Control for Wheel Inverted Pendulum Systems
Hoang, A. D., Quang, N. H., Dao, P. N., & Nguyen, T. T. (2021) · Advances in intelligent systems and computing · doi:10.1007/978-981-15-7527-3_51
Higher-order Polynomial Signal Tracking Control of Unknown Systems using Off-policy Integral Reinforcement Learning
Cheng, W., & Li, J. (2020) · International Conference on Control, Automation, Robotics and Vision (ICARCV) · doi:10.1109/icarcv50220.2020.9305458
Bridging Reinforcement Learning and Iterative Learning Control: Autonomous Reference Tracking for Unknown, Nonlinear DynamicsPreprint
Meindl, M., Lehmann, D., & Seel, T. (2021) · doi:10.36227/techrxiv.15022497.v1
Game Theoretical Reinforcement Learning for Robust H∞ Tracking Control of Discrete-Time Linear Systems with Unknown Dynamics
Wu, H., Li, S., Durdevic, P., & Yang, Z. (2021) · Intelligent Autonomous Systems · doi:10.1109/icoias53694.2021.00058
Safety aware model-based reinforcement learning for optimal control of a class of output-feedback nonlinear systemsPreprint
Mahmud, S. M. N., Abudia, M., Nivison, S., Bell, Z. I., & Kamalapurkar, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.00271
Aprendizagem profunda por reforço combinada com controlador recursivo robusto para controle lateral baseado em visão computacional
de Morais, G. A. P. (2020) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/d.18.2020.tde-25082020-092618
One-shot Input-Output History Feedback Controller Design for Unknown Linear Systems
Hirai, T., & Sadamoto, T. (2021) · Transactions of the Institute of Systems Control and Information Engineers · doi:10.5687/iscie.34.235
Adaptive control and signal processing literature survey (No. 24)
(2011) · International Journal of Adaptive Control and Signal Processing · doi:10.1002/acs.1252
Recent advances on micro-control for near-critical complex systems
Zhao, D., & Dong, Y. (2022) · Control Theory and Technology · doi:10.1007/s11768-022-00082-2
Safety-critical Policy Iteration Algorithm for Control under Model Uncertainty
Yazdani, N. M., Moghaddam, R. K., & Olyaei, M. H. (2022) · Artificial Intelligence Advances · doi:10.30564/aia.v4i1.4361
Introduction
Jiang, Y., & Jiang, Z. (2017) · Wiley eBooks · doi:10.1002/9781119132677.ch1
Neuro-inspired Control
Lewis, F. L., & Vamvoudakis, K. G. (2021) · Encyclopedia of Systems and Control · doi:10.1007/978-3-030-44184-5_224
Self-citation
Global Linear Convergence of Online Reinforcement Learning for Partially Observable Systems
Hirai, T., & Sadamoto, T. (2022) · European Control Conference (ECC) · doi:10.23919/ecc55457.2022.9838197
Neural Control and Approximate Dynamic Programming
Lewis, F. L., & Vamvoudakis, K. G. (2015) · Encyclopedia of Systems and Control · doi:10.1007/978-1-4471-5058-9_224
Self-citation
Model-Free Control of Time Delay Systems
Rizvi, S. A. A., & Lin, Z. (2022) · Control engineering · doi:10.1007/978-3-031-15858-2_5
Introduction to Optimal Control and Reinforcement Learning
Rizvi, S. A. A., & Lin, Z. (2022) · Control engineering · doi:10.1007/978-3-031-15858-2_1
Model Uncertainty-aware Adaptive Controller Design with Online Parameter Identification
Çatak, A., Cetin, A. T., & Koyuncu, E. (2023) · AIAA SciTech Forum · doi:10.2514/6.2023-0490
Performance-Based Fault Detection for Model-Free Automatic Control Systems
Liu, R., Xiu, X., & Tian, E. (2022) · China Automation Congress (CAC) · doi:10.1109/cac57257.2022.10055496
Output Feedback Control for Deterministic Unknown Dynamics Discrete-Time System using Deep Recurrent Q-NetworksPreprint
Novitarini, A. (2023) · doi:10.36227/techrxiv.23578710.v1
Output Feedback Control for Deterministic Unknown Dynamics Discrete-Time System using Deep Recurrent Q-NetworksPreprint
Novitarini, A. (2023) · doi:10.36227/techrxiv.23578710
Background on Reinforcement Learning and Optimal Control
Li, J., Lewis, F. L., & Fan, J. (2023) · Advances in industrial control · doi:10.1007/978-3-031-28394-9_1
Self-citation
Policy Iteration-Output Feedback Adaptive Dynamic Programming Tracking Control for a Two-Wheeled Self Balancing Robot
Cao, T. T., Nguyễn, V. Q., Duc, H. A. N., Nguyen, Q. P., & Dao, P. N. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-99-4725-6_71
Optimal Output Tracking for Unknown Linear Discrete-Time Systems Based on Adaptive Dynamic Programming and Output Feedback
Fan, K., Cai, X. S., Wu, J., Wu, X. Y., & Xiao, Q. (2023) · Lecture notes in electrical engineering · doi:10.1007/978-981-99-6882-4_72
Output Feedback Speed Control for a Wankel Rotary Engine via Q-Learning
Chen, A. S., Herrmann, G., Burgess, S. C., & Brace, C. J. (2023) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2023.10.1014
Output Feedback Control for Deterministic Unknown Dynamics Discrete-Time System using Deep Recurrent Q-NetworksPreprint
Novitarini, A. (2023) · doi:10.36227/techrxiv.23578710.v2
Output Feedback Control for Deterministic Unknown Dynamics Discrete-Time System Using Deep Recurrent Q-Networks
Putri, A. N., Hidayat, E., Mahayana, D., & Machbub, C. (2023) · IEEE Access · doi:10.1109/access.2023.3342201

Authors: Lewis, F. L., & Vamvoudakis, K. G.
Affiliations: The University of Texas at Arlington (US)
ORCID: Lewis, F L
Vamvoudakis, K G
Venue: IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2011-02
Volume: 41
Issue: 1
Pages: 14-25
ISSN: 1083-4419 (ISSN-L)
1941-0492
DOI: 10.1109/tsmcb.2010.2043839
OpenAlex: W2151966330
Cited by: 544
References: 48
Citations by year: 2026: 22 · 2025: 49 · 2024: 46 · 2023: 43 · 2022: 37 · 2021: 49 · 2020: 53 · 2019: 34 · 2018: 52 · 2017: 26 · 2016: 35 · 2015: 19 · 2014: 33 · 2013: 18 · 2012: 19
FWCI: 15.12
Type: journal article
Language: en
Open access: closed
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Locations: IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) (publishedVersion)
PubMed (publishedVersion)
Topics: Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
Keywords: approximate dynamic programming, partially observable Markov decision processes, output feedback control, linear dynamical systems, policy iteration, value iteration, Q-learning, observability index
MeSH terms: Algorithms, Artificial Intelligence, Feedback, Learning, Markov Chains, Reinforcement, Psychology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks