Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design

Reinforcement Learning: An Introduction

Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works

Reinforcement Learning: An Introduction

Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works

Dynamic Programming

Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588 · see citing works

Technical Note: Q-Learning

Watkins, C., & Dayan, P. (1992) · Machine-mediated learning · doi:10.1023/a:1022676722315 · Cited by 14,277 · see citing works

Functional analysis

Rudin, W. (1973) · Cited by 13,708 · see citing works

Real and Complex Analysis.

Garreau, G. A., & Rudin, W. (1987) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348852 · Cited by 10,941 · see citing works

Untitled work

(1995) · Cited by 10,558 · see citing works

A Stochastic Approximation Method

Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643 · see citing works

Q-learning

Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083 · see citing works

Markov decision processes: Discrete stochastic dynamic programming

Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037 · see citing works

Convergence of Probability Measures

Kingmán, J. F. C., & Billingsley, P. (1969) · Revue de l Institut International de Statistique / Review of the International Statistical Institute · doi:10.2307/1402124 · Cited by 6,930 · see citing works

Introduction to Reinforcement Learning

Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855 · see citing works

Learning to predict by the methods of temporal differences

Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · see citing works

Adaptive Control

Åström, K. J., & Wittenmark, B. (1989) · Cited by 5,009 · see citing works

Neuro-dynamic Programming

(2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,601 · see citing works

Neuronlike adaptive elements that can solve difficult learning control problems

Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,238 · see citing works

Stochastic Differential Equations: Theory and Applications.

Whittle, P., & Arnold, L. (1976) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344856 · Cited by 2,391 · see citing works

Adaptive Control Processes: A Guided Tour.

Page, E. S., & Bellman, R. (1962) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2343225 · Cited by 2,208 · see citing works

Stochastic Approximation and Recursive Algorithms and Applications

Kushner, H. J., & Yin, G. G. (2003) · Springer eBooks · doi:10.1007/b97441 · Cited by 1,912 · see citing works

Markov Decision Processes: Discrete Stochastic Dynamic Programming

Baxter, L. A., & Puterman, M. L. (1995) · Technometrics · doi:10.2307/1269932 · Cited by 1,889 · see citing works

Reinforcement learning and adaptive dynamic programming for feedback control

Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,569 · see citing works

Robust stabilization of uncertain linear systems: quadratic stabilizability and H/sup infinity / control theory

Khargonekar, P. P., Petersen, I. R., & Zhou, K. (1990) · IEEE Transactions on Automatic Control · doi:10.1109/9.50357 · Cited by 1,535 · see citing works

Analysis of recursive stochastic algorithms

Ljung, L. (1977) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1977.1101561 · Cited by 1,485 · see citing works

Least squares stationary optimal control and the algebraic Riccati equation

Willems, J. C. (1971) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1971.1099831 · Cited by 1,440 · see citing works

Principles of sensorimotor learning

Wolpert, D. M., Diedrichsen, J., & Flanagan, J. R. (2011) · Nature Reviews Neuroscience · doi:10.1038/nrn3112 · Cited by 1,391 · see citing works

Convergence of Probability Measures

Snell, J. L. (1970) · Technometrics · doi:10.1080/00401706.1970.10488647 · Cited by 1,355 · see citing works

On an iterative technique for Riccati equation computations

Kleinman, D. L. (1968) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1968.1098829 · Cited by 1,231 · see citing works

Algebraic Riccati Equations

Lancaster, P., & Rodman, L. (1995) · doi:10.1093/oso/9780198537953.001.0001 · Cited by 1,218 · see citing works

Adaptive critic designs

Prokhorov, D. V., & Wunsch, D. C. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.623201 · Cited by 1,181 · see citing works

Algebraic Riccati Equations

(2006) · Birkhäuser-Verlag eBooks · doi:10.1007/3-7643-7350-4_14 · Cited by 1,125 · see citing works

Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers

Lewis, F. L., Vrabie, D., & Vamvoudakis, K. G. (2012) · IEEE Control Systems · doi:10.1109/mcs.2012.2214134 · Cited by 1,107 · see citing works

Real and complex analysis, 3rd ed.

Rudin, W. (1987) · McGraw-Hill, Inc. eBooks · Cited by 1,103 · see citing works

The central nervous system stabilizes unstable dynamics by learning optimal impedance

Burdet, E., Osu, R., Franklin, D. W., Milner, T. E., & Kawato, M. (2001) · Nature · doi:10.1038/35106566 · Cited by 1,097 · see citing works

Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics

Jiang, Y., & Jiang, Z. (2012) · Automatica · doi:10.1016/j.automatica.2012.06.096 · Cited by 1,079 · see citing works

Neuro-Dynamic Programming

(2010) · Springer eBooks · doi:10.1007/978-0-387-30164-8_588 · Cited by 978 · see citing works

Adaptive optimal control for continuous-time linear systems based on policy iteration

Vrabie, D., Pastravanu, O. C., Abu-Khalaf, M., & Lewis, F. L. (2008) · Automatica · doi:10.1016/j.automatica.2008.08.017 · Cited by 953 · see citing works

Limit Theorems for Stochastic Processes

Skorokhod, A. V. (1956) · Theory of Probability and Its Applications · doi:10.1137/1101022 · Cited by 897 · see citing works

Handbook of Learning and Approximate Dynamic Programming

Si, J., Barto, A. G., Powell, W. B., & Wunsch, D. C. (2004) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470544785 · Cited by 863 · see citing works

Adaptive dynamic programming

Murray, J. J., Cox, C. J., Lendaris, G. G., & Saeks, R. (2002) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2002.801727 · Cited by 677 · see citing works

Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation

Beard, R. W., Saridis, G. N., & Wen, J. T.-Y. (1997) · Automatica · doi:10.1016/s0005-1098(97)00128-3 · Cited by 663 · see citing works

Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints

Zhang, H., Luo, Y., & Liu, D. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2009.2027233 · Cited by 653 · see citing works

Asynchronous Stochastic Approximation and Q-Learning

Tsitsiklis, J. N. (1994) · Machine Learning · doi:10.1023/a:1022689125041 · Cited by 597 · see citing works

Stochastic Differential Equations: Theory and Applications

Baxendale, P. H., & Lototsky, S. V. (2007) · Interdisciplinary mathematical sciences · doi:10.1142/6453 · Cited by 592 · see citing works

Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data

Lewis, F. L., & Vamvoudakis, K. G. (2011) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2010.2043839 · Cited by 544 · see citing works

Robust stabilization of linear systems with norm-bounded time-varying uncertainty

Zhou, K., & Khargonekar, P. P. (1988) · Systems & Control Letters · doi:10.1016/0167-6911(88)90034-5 · Cited by 496 · see citing works

Asynchronous stochastic approximation and Q-learning

Tsitsiklis, J. N. (1994) · Machine Learning · doi:10.1007/bf00993306 · Cited by 453 · see citing works

Linear Quadratic Tracking Control of Partially-Unknown Continuous-Time Systems Using Reinforcement Learning

Modares, H., & Lewis, F. L. (2014) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2014.2317301 · Cited by 451 · see citing works

Evidence for the Flexible Sensorimotor Strategies Predicted by Optimal Feedback Control

Liu, D., & Todorov, E. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.1110-06.2007 · Cited by 447 · see citing works

Stochastic Approximation and Its Applications

Chen, H.-F. (2003) · Kluwer Academic Publishers eBooks · doi:10.1007/b101987 · Cited by 437 · see citing works

Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems

Jiang, Y., & Jiang, Z. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2294968 · Cited by 415 · see citing works

Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming

Wang, D., Liu, D., Wei, Q., Zhao, D., & Jin, N. (2012) · Automatica · doi:10.1016/j.automatica.2012.05.049 · Cited by 395 · see citing works

Dynamic Programming and Optimal Control, Vol. II

Bertsekas, D. P. (1976) · Cited by 310 · see citing works

Transient stability enhancement and voltage regulation of power systems

Wang, Y., Hill, D. J., Middleton, R. H., et al. (1993) · IEEE Transactions on Power Systems · doi:10.1109/59.260819 · Cited by 310 · see citing works

Multivariable adaptive control: A survey

Tao, G. (2014) · Automatica · doi:10.1016/j.automatica.2014.10.015 · Cited by 299 · see citing works

Adaptive Dynamic Programming for Finite-Horizon Optimal Control of Discrete-Time Nonlinear Systems With $\varepsilon$-Error Bound

Wang, F., Jin, N., Liu, D., & Wei, Q. (2010) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2076370 · Cited by 294 · see citing works

Optimal Adaptive Control and Differential Games by Reinforcement Learning Principles

Vrabie, D., Vamvoudakis, K. G., & Lewis, F. L. (2012) · Institution of Engineering and Technology eBooks · doi:10.1049/pbce081e · Cited by 288 · see citing works

Approximate policy iteration: a survey and some new methods

Bertsekas, D. P. (2011) · Journal of Control Theory and Applications · doi:10.1007/s11768-011-1005-3 · Cited by 273 · see citing works

Integral Reinforcement Learning for Linear Continuous-Time Zero-Sum Games With Completely Unknown Dynamics

Li, H., Liu, D., & Wang, D. (2014) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2014.2300532 · Cited by 231 · see citing works

Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses

Xu, H., Jagannathan, S., & Lewis, F. L. (2012) · Automatica · doi:10.1016/j.automatica.2012.03.007 · Cited by 230 · see citing works

Neural-Network-Based Optimal Control for a Class of Unknown Discrete-Time Nonlinear Systems Using Globalized Dual Heuristic Programming

Liu, D., Wang, D., Zhao, D., Wei, Q., & Jin, N. (2012) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2012.2198057 · Cited by 222 · see citing works

Adaptive dynamic programming and optimal control of nonlinear nonaffine systems

Bian, T., Jiang, Y., & Jiang, Z. (2014) · Automatica · doi:10.1016/j.automatica.2014.08.023 · Cited by 220 · see citing works

Stability of Stochastic Approximation under Verifiable Conditions

Andrieu, C., Moulines, É., & Priouret, P. (2005) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012902417267 · Cited by 198 · see citing works

Functional significance of stiffness in adaptation of multijoint arm movements to stable and unstable dynamics

Franklin, D. W., Burdet, E., Osu, R., Kawato, M., & Milner, T. E. (2003) · Experimental Brain Research · doi:10.1007/s00221-003-1443-3 · Cited by 189 · see citing works

Data-Driven Adaptive Optimal Control of Connected Vehicles

Gao, W., Jiang, Z., & Özbay, K. M. A. (2016) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2016.2597279 · Cited by 175 · see citing works

Robust adaptive dynamic programming for linear and nonlinear systems: An overview

Jiang, Z., & Jiang, Y. (2013) · European Journal of Control · doi:10.1016/j.ejcon.2013.05.017 · Cited by 173 · see citing works

Optimal Control of Nonlinear Continuous-Time Systems in Strict-Feedback Form

Zargarzadeh, H., Dierks, T., & Jagannathan, S. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2441712 · Cited by 168 · see citing works

Asynchronous Stochastic Approximations

Borkar, V. S. (1998) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012995282784 · Cited by 167 · see citing works

Decentralized Adaptive Optimal Control of Large-Scale Systems With Application to Power Systems

Bian, T., Jiang, Y., & Jiang, Z. (2014) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2345343 · Cited by 166 · see citing works

Multiple Actor-Critic Structures for Continuous-Time Optimal Control Using Input-Output Data

Song, R., Lewis, F. L., Wei, Q., Zhang, H., Jiang, Z., & Levine, D. S. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2399020 · Cited by 140 · see citing works

A review of the matrix Riccati equation

Kučera, V. (1973) · Czech digital mathematics library · Cited by 136 · see citing works

Revisiting Approximate Dynamic Programming and its Convergence

Heydari, A. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2314612 · Cited by 134 · see citing works

Robust controller design for uncertain systems with time delays: LMI approach

Jeung, E. T., Oh, D.-C., Kim, J., et al. (1996) · Automatica · doi:10.1016/0005-1098(96)00055-6 · Cited by 124 · see citing works

Adaptive continuous-time linear quadratic Gaussian control

Duncan, T. E., Guo, L., & Pasik-Duncan, B. (1999) · IEEE Transactions on Automatic Control · doi:10.1109/9.788532 · Cited by 124 · see citing works

Model-Free Dual Heuristic Dynamic Programming

Ni, Z., He, H., Zhong, X., & Prokhorov, D. V. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2424971 · Cited by 108 · see citing works

Construction of Suboptimal Control Sequences

Leake, R. J., & Liu, R.-W. (1967) · SIAM Journal on Control · doi:10.1137/0305004 · Cited by 106 · see citing works

On the stability of linear stochastic systems

Kleinman, D. L. (1969) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1969.1099206 · Cited by 97 · see citing works

Phase Portrait of the Matrix Riccati Equation

Shayman, M. A. (1986) · SIAM Journal on Control and Optimization · doi:10.1137/0324001 · Cited by 96 · see citing works

Biological Learning and Control: How the Brain Builds Representations, Predicts Events, and Makes Decisions

Shadmehr, R., & Mussa-Ivaldi, S. (2012) · Cited by 95 · see citing works

Adaptive Dynamic Programming for Stochastic Systems With State and Control Dependent Noise

Bian, T., Jiang, Y., & Jiang, Z. (2016) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2016.2550518 · Cited by 90 · see citing works

Abstract dynamic programming

Pardalos, P. M. (2013) · Optimization methods & software · doi:10.1080/10556788.2013.858876 · Cited by 86 · see citing works

Optimal Adaptive Control of Linear-Quadratic-Gaussian Systems

Kumar, P. (1983) · SIAM Journal on Control and Optimization · doi:10.1137/0321009 · Cited by 80 · see citing works

Abstract Dynamic Programming

Sargent, T. J., & Stachurski, J. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009540780.010 · Cited by 61 · see citing works

Stochastic Approximation for Nonexpansive Maps: Application to Q -Learning Algorithms

Abounadi, J., Bertsekas, D. P., & Borkar, V. S. (2002) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012998346621 · Cited by 61 · see citing works

Robust controller design for uncertain systems with variable time delay

García‐Sanz, M., Guillén, J. C., & Ibarrola, J. (2001) · Control Engineering Practice · doi:10.1016/s0967-0661(01)00040-5 · Cited by 41 · see citing works

Adaptive dynamic programming as a theory of sensorimotor control

Jiang, Y., & Jiang, Z. (2014) · Biological Cybernetics · doi:10.1007/s00422-014-0613-7 · Cited by 38 · see citing works

Approximate Dynamic Programming for Optimal Stationary Control With Control-Dependent Noise

Jiang, Y., & Jiang, Z.-P. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2165729 · Cited by 33 · see citing works

A junction-by-junction feedback-based strategy with convergence analysis for dynamic traffic assignment

Liu, T., Lu, X., & Jiang, Z. (2015) · Science China Information Sciences · doi:10.1007/s11432-015-5444-1 · Cited by 25 · see citing works

Adaptive optimal control for linear stochastic systems with additive noise

Bian, T., & Zhong-Ping, J. (2015) · Chinese Control Conference (CCC) · doi:10.1109/chicc.2015.7260102 · Cited by 5 · see citing works

Learning-Based Adaptive Optimal Tracking Control of Strict-Feedback Nonlinear Systems

Gao, W., & Jiang, Z.-P. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2761718 · Cited by 176 · see citing works

Data-driven control of hydraulic servo actuator: An event-triggered adaptive dynamic programming approach

Djordjevic, V., Tao, H., Song, X., He, S., Gao, W., & Stojanovic, V. (2023) · Mathematical Biosciences and Engineering · doi:10.3934/mbe.2023376 · Cited by 107 · see citing works

Reinforcement Learning for Approximate Optimal Control

Dixon, W. E. (2020) · Encyclopedia of Systems and Control · doi:10.1007/978-1-4471-5102-9_100063-1

Approximate Dynamic Programming and Reinforcement Learning - Algorithms, Analysis and an Application

Lakshminarayanan, C. (2018)

Value Iteration is Optic Composition

Hedges, J., & Sakamoto, R. R. (2023) · Electronic Proceedings in Theoretical Computer Science · doi:10.4204/eptcs.380.24 · Cited by 2 · see citing works

UAV perimeter patrol operations optimization using efficient Dynamic Programming

Krishnamoorthy, K., Pachter, M. N., Chandler, P., et al. (2011) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2011.5990603 · Cited by 7 · see citing works

Dynamic Programming Principle for Backward Doubly Stochastic Recursive Optimal Control Problem and Sobolev Weak Solution of The Stochastic Hamilton-Bellman EquationPreprint

Li, Y., Matoussi, A., Wei, L., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.05426