The DOI matches its Crossref record; 144 of the 144 references it cites were found in a registry or an index.
Reinforcement Learning and Dynamic Programming Using Function Approximators
Open accessAbstract
From household appliances to applications in robotics, engineered systems involving complex dynamics can only be as effective as the algorithms that control them. While Dynamic Programming (DP) has provided researchers with a way to optimally solve decision and control problems involving complex dynamic systems, its practical value was limited by algorithms that lacked the capacity to scale up to realistic problems. However, in recent years, dramatic developments in Reinforcement Learning (RL), the model-free counterpart of DP, changed our understanding of what is possible. Those developments led to the creation of reliable methods that can be applied even when a mathematical model of the system is unavailable, allowing researchers to solve challenging control problems in engineering, as well as in a variety of other disciplines, including economics, medicine, and artificial intelligence. Reinforcement Learning and Dynamic Programming Using Function Approximators provides a comprehensive and unparalleled exploration of the field of RL and DP. With a focus on continuous-variable problems, this seminal text details essential developments that have substantially altered the field over the past decade. In its pages, pioneering experts provide a concise introduction to classical RL and DP, followed by an extensive presentation of the state-of-the-art and novel methods in RL and DP with approximation. Combining algorithm development with theoretical guarantees, they elaborate on their work with illustrative examples and insightful comparisons. Three individual chapters are dedicated to representative algorithms from each of the major classes of techniques: value iteration, policy iteration, and policy search. The features and performance of these algorithms are highlighted in extensive experimental studies on a range of control applications. The recent development of applications involving complex systems has led to a surge of interest in RL and DP methods and the subsequent need for a quality resource on the subject. For graduate students and others new to the field, this book offers a thorough introduction to both the basics and emerging methods. And for those researchers and practitioners working in the fields of optimal and adaptive control, machine learning, artificial intelligence, and operations research, this resource offers a combination of practical algorithms, theoretical analysis, and comprehensive examples that they will be able to adapt and apply to their own work. Access the authors' website at www.dcsc.tudelft.nl/rlbook/ for additional material, including computer code used in the studies and information concerning new developments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Genetic algorithms in search, optimization, and machine learning
(1989) · Choice Reviews Online · doi:10.5860/choice.27-0936 · Cited by 49,010
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
Artificial intelligence: a modern approach
(1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 21,974
Fuzzy identification of systems and its applications to modeling and control
Takagi, T., & Sugeno, M. (1985) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1985.6313399 · Cited by 19,592
An Introduction to Support Vector Machines and Other Kernel-based Learning Methods
Cristianini, N., & Shawe-Taylor, J. (2000) · doi:10.1017/cbo9780511801389 · Cited by 13,855
A tutorial on support vector regression
Smola, A. J., & Schölkopf, B. (2004) · Statistics and Computing · doi:10.1023/b:stco.0000035301.49549.88 · Cited by 13,387
(1995) · Cited by 10,558
Geurts, P., Ernst, D., & Wehenkel, L. (2006) · Machine Learning · doi:10.1007/s10994-006-6226-1 · Cited by 9,300
Self-citation(2006) · Springer Series in Operations Research and Financial Engineering · doi:10.1007/978-0-387-40065-5 · Cited by 9,008
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Numerical Recipes: The Art of Scientific Computing
Ziegel, E. (1987) · Technometrics · doi:10.1080/00401706.1987.10488304 · Cited by 6,702
Advances in kernel methods: support vector learning
Schölkopf, B., Burges, C. J. C., & Smola, A. J. (1999) · doi:10.5555/299094 · Cited by 5,789
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
(1999) · Cited by 5,147
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149
Abe, S. (2001) · Springer eBooks · doi:10.1007/978-1-4471-0285-4 · Cited by 3,721
Predictive control with constraints
Camacho, E. F. (2003) · Automatica · doi:10.1016/s0005-1098(03)00056-6 · Cited by 3,530
An Introduction to Support Vector Machines and Other Kernel‐based Learning Methods
(2001) · Kybernetes · doi:10.1108/k.2001.30.1.103.6 · Cited by 3,308
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Baxter, L. A. (1995) · Technometrics · doi:10.1080/00401706.1995.10484354 · Cited by 2,415
Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations
Ascher, U. M., & Petzold, L. R. (1998) · doi:10.1137/1.9781611971392 · Cited by 2,403
A Comprehensive Survey of Multiagent Reinforcement Learning
Busoniu, L., Babuska, R., & De Schutter, B. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2007.913919 · Cited by 2,235
Konda, V. R., & Tsitsiklis, J. N. (2002) · Cited by 2,202
Fundamentals of Artificial Neural Networks
Hassoun, M. H. (1996) · Proceedings of the IEEE · doi:10.1109/jproc.1996.503146 · Cited by 1,887
Fuzzy sets and fuzzy logic: theory and applications
(1996) · Choice Reviews Online · doi:10.5860/choice.33-2786 · Cited by 1,871
Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping
Ng, A. Y., Harada, D., & Russell, S. (1999) · Cited by 1,744
Digital control of dynamic systems
Anderson, B. (1981) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1981.1163502 · Cited by 1,690
Stochastic optimal control : the discrete time case
Bertsekas, D. P., & Shreve, S. E. (2007) · Cited by 1,654
Optimal control theory: An introduction
Levine, W. (1972) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1972.1100008 · Cited by 1,623
On-line Q-learning Using Connectionist Systems
Rummery, G. A. (1994) · Cited by 1,477
Neuro-Dynamic Programming: NDP
Bertsekas, D. P. (2001) · Encyclopedia of Optimization · doi:10.1007/0-306-48332-7_333 · Cited by 1,466
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition
Dietterich, T. G. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.639 · Cited by 1,435
Tree-Based Batch Mode Reinforcement Learning
Ernst, D., Geurts, P., & Wehenkel, L. A. (2005) · Journal of Machine Learning Research · doi:10.5555/1046920.1088690 · Cited by 1,352
Self-citationFuzzy Logic: Intelligence, Control, and Information
Yen, J., & Langari, R. (1998) · Medical Entomology and Zoology · Cited by 1,337
On the Convergence of Pattern Search Algorithms
Torczon, V. (1997) · SIAM Journal on Optimization · doi:10.1137/s1052623493250780 · Cited by 1,336
Sutton, R. S. (1990) · Machine Learning Proceedings 1990 · doi:10.1016/b978-1-55860-141-3.50030-4 · Cited by 1,314
Corriou, J.-P. (2004) · Process Control · doi:10.1007/978-1-4471-3848-8_16 · Cited by 1,271
Cooperative Multi-Agent Learning: The State of the Art
Panait, L., & Luke, S. (2005) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-005-2631-2 · Cited by 1,237
An analysis of temporal-difference learning with function approximation
Tsitsiklis, J. N., & Van Roy, B. (1997) · IEEE Transactions on Automatic Control · doi:10.1109/9.580874 · Cited by 1,194
Prokhorov, D. V., & Wunsch, D. C. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.623201 · Cited by 1,186
Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding
Sutton, R. S. (1995) · Cited by 1,072
Reinforcement Learning in Continuous Time and Space
Doya, K. (2000) · Neural Computation · doi:10.1162/089976600300015961 · Cited by 960
Adaptive optimal control for continuous-time linear systems based on policy iteration
Vrabie, D., Pastravanu, O., Abu-Khalaf, M., & Lewis, F. L. (2009) · Automatica · doi:10.1016/j.automatica.2008.08.017 · Cited by 949
Learning and tuning fuzzy logic controllers through reinforcements
Berenji, H. R., & Khedkar, P. (1992) · IEEE Transactions on Neural Networks · doi:10.1109/72.159061 · Cited by 942
Residual Algorithms: Reinforcement Learning with Function Approximation
Baird, L. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50013-x · Cited by 918
Kakade, S. M. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 775
Riedmiller, M. (2005) · Lecture Notes in Computer Science · doi:10.1007/11564096_32 · Cited by 757
Approximate dynamic programming: solving the curses of dimensionality
Pardalos, P. M. (2009) · Optimization Methods and Software · doi:10.1080/10556780802583108 · Cited by 666
Natural Exponential Families with Quadratic Variance Functions
Morris, C. N. (1982) · The Annals of Statistics · doi:10.1214/aos/1176345690 · Cited by 617
Linear Least-Squares algorithms for temporal difference learning
Bradtke, S. J., & Barto, A. G. (1996) · Machine Learning · doi:10.1007/bf00114723 · Cited by 599
Big Omicron and big Omega and big Theta
Knuth, D. E. (1976) · ACM SIGACT News · doi:10.1145/1008328.1008329 · Cited by 592
Natural actor–critic algorithms
Bhatnagar, S., Sutton, R. S., Ghavamzadeh, M., & Lee, M. (2009) · Automatica · doi:10.1016/j.automatica.2009.07.008 · Cited by 559
Fast gradient-descent methods for temporal-difference learning with linear function approximation
Sutton, R. S., Maei, H. R., Precup, D., Bhatnagar, S., Silver, D., Szepesvári, C., & Wiewiora, E. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553501 · Cited by 496
Finite-Time Bounds for Fitted Value Iteration
Munos, R., & Szepesvári, C. (2008) · SZTAKI Publication Repository (Hungarian Academy of Sciences) · doi:10.5555/1390681.1390708 · Cited by 474
Asynchronous stochastic approximation and Q-learning
Tsitsiklis, J. N. (1994) · Machine Learning · doi:10.1007/bf00993306 · Cited by 453
Stable Function Approximation in Dynamic Programming
Gordon, G. J. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50040-2 · Cited by 446
Kernel Methods for Pattern Analysis
Breneman, J. (2005) · Technometrics · doi:10.1198/tech.2005.s264 · Cited by 426
If multi-agent learning is the answer, what is the question?
Shoham, Y., Powers, R., & Grenager, T. (2007) · Artificial Intelligence · doi:10.1016/j.artint.2006.02.006 · Cited by 410
Fuzzy inference system learning by reinforcement methods
Jouffe, L. (1998) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/5326.704563 · Cited by 384
FOR LEAST-SQUARES POLICY ITERATION
Schwab, D. (2016) · Cited by 372
Anytime Point-Based Approximations for Large POMDPs
Pineau, J., Gordon, G., & Thrun, S. (2006) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2078 · Cited by 370
Kernel-Based Reinforcement Learning
(2017) · Encyclopedia of Machine Learning and Data Mining · doi:10.1007/978-1-4899-7687-1_100235 · Cited by 367
Dynamic Programming and Suboptimal Control: A Survey from ADP to MPC*
Bertsekas, D. P. (2005) · European Journal of Control · doi:10.3166/ejc.11.310-334 · Cited by 366
PEGASUS: A Policy Search Method for Large MDPs and POMDPs
Ng, A. Y., & Jordan, M. I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3878 · Cited by 360
Reinforcement learning with Gaussian processes
Engel, Y., Mannor, S., & Meir, R. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102377 · Cited by 335
Learning to Drive a Bicycle Using Reinforcement Learning and Shaping
Randløv, J., & Alstrøm, P. (1998) · Cited by 324
Computationally Feasible Bounds for Partially Observed Markov Decision Processes
Lovejoy, W. S. (1991) · Operations Research · doi:10.1287/opre.39.1.162 · Cited by 307
Reinforcement Learning with Soft State Aggregation
Singh, S. P., Jaakkola, T., & Jordan, M. I. (1994) · Cited by 274
Kernel-Based Least Squares Policy Iteration for Reinforcement Learning
Xu, X., Hu, D., & Lu, X. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2007.899161 · Cited by 268
Moore, A. W., & Atkeson, C. G. (1995) · Machine Learning · doi:10.1023/a:1022656217772 · Cited by 259
The role of exploration in learning control
Thrun, S. (1992) · Cited by 258
Experiments with Reinforcement Learning in Problems with Continuous State and Action Spaces
Santamaria, J. C., Sutton, R. S., & Ram, A. (1997) · Adaptive Behavior · doi:10.1177/105971239700600201 · Cited by 258
Specific therapy regimes could lead to long-term immunological control of HIV
Wodarz, D., & Nowak, M. A. (1999) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.96.25.14464 · Cited by 250
Stochastic Learning and Optimization - A Sensitivity-Based Approach
Cao, X. (2008) · IFAC Proceedings Volumes · doi:10.3182/20080706-5-kr-1001.00589 · Cited by 250
Robot shaping: developing autonomous agents through learning
Dorigo, M., & Colombetti, M. (1994) · Artificial Intelligence · doi:10.1016/0004-3702(94)90047-7 · Cited by 238
Point-Based Value Iteration for Continuous POMDPs
Porta, J. M., Vlassis, N. A., Spaan, M. T. J., & Poupart, P. (2006) · DIGITAL.CSIC (Spanish National Research Council (CSIC)) · doi:10.13039/501100000780 · Cited by 234
Evolutionary Function Approximation for Reinforcement Learning
Whiteson, S., & Stone, P. (2006) · Cited by 231
Dynamic Multidrug Therapies for HIV: Optimal and STI Control Approaches
M. Adams, B., 1. Center for Research in Scientific Computation, N. C. S. U., T. Banks, H., Kwon, H.-D., T. Tran, H., & 2. Center for Research in Scientific Computation, N. C. S. U. (2004) · Mathematical Biosciences and Engineering · doi:10.3934/mbe.2004.1.223 · Cited by 228
A Generalization Error for Q-Learning.
Murphy, S. A. (2005) · PubMed · Cited by 216
Reinforcement Learning Versus Model Predictive Control: A Comparison on a Power System Problem
Ernst, D., Glavic, M., Capitanescu, F., & Wehenkel, L. (2009) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2008.2007630 · Cited by 214
Self-citationPredictive State Representations: A New Theory for Modeling Dynamical Systems
Singh, S. P., James, M., & Rudary, M. R. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1207.4167 · Cited by 201
Near-optimal Regret Bounds for Reinforcement Learning
Auer, P. L., Jaksch, T., & Ortner, R. (2008) · Cited by 197
Gaussian process dynamic programming
Deisenroth, M. P., Rasmussen, C. E., & Peters, J. (2009) · Neurocomputing · doi:10.1016/j.neucom.2008.12.019 · Cited by 192
Fitted Q-iteration in continuous action-space MDPs
Antos, A., Szepesvári, C., & Munos, R. (2007) · HAL · Cited by 188
Basis Function Adaptation in Temporal Difference Reinforcement Learning
Menache, I., Mannor, S., & Shimkin, N. (2005) · Annals of Operations Research · doi:10.1007/s10479-005-5732-z · Cited by 184
Discovering Hierarchy in Reinforcement Learning with HEXQ
Hengst, B. (2002) · Cited by 181
Regularization and feature selection in least-squares temporal difference learning
Kolter, J. Z., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553442 · Cited by 176
Bayes meets bellman: the Gaussian process approach to temporal difference learning
Engel, Y., Mannor, S., & Meir, R. (2003) · Cited by 174
An actor-critic algorithm for constrained Markov decision processes
Borkar, V. S. (2005) · Systems & Control Letters · doi:10.1016/j.sysconle.2004.08.007 · Cited by 172
Tuning Bandit Algorithms in Stochastic Environments
Audibert, J.-Y., Munos, R., & Szepesvári, C. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-75225-7_15 · Cited by 171
Adaptive aggregation methods for infinite horizon dynamic programming
Bertsekas, D. P., & Castanon, D. A. (1989) · IEEE Transactions on Automatic Control · doi:10.1109/9.24227 · Cited by 171
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
Jaakkola, T., Jordan, M. I., & Singh, S. P. (1993) · Defense Technical Information Center (DTIC) · doi:10.21236/ada276517 · Cited by 165
Automatic basis function construction for approximate dynamic programming and reinforcement learning
Keller, P. W., Mannor, S., & Precup, D. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143901 · Cited by 164
Parr, R., Li, L. C., Taylor, G., Painter-Wakefield, C., & Littman, M. L. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390251 · Cited by 164
Solving the Vehicle Routing Problem with Stochastic Demands using the Cross-Entropy Method
Chepuri, K., & Homem‐de‐Mello, T. (2005) · Annals of Operations Research · doi:10.1007/s10479-005-5729-7 · Cited by 160
Self-Improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching
Lin, L.-J. (1992) · Reinforcement Learning · doi:10.1007/978-1-4615-3618-5_5 · Cited by 160
Adaptive Critic Learning Techniques for Engine Torque and Air–Fuel Ratio Control
Liu, D., Javaherian, H., Kovalenko, O., & Huang, T. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.922019 · Cited by 154
Sutton, R. S., Maei, H. R., & Szepesvári, C. (2008) · neural information processing systems · Cited by 152
Lyapunov design for safe reinforcement learning
Perkins, T. J., & Barto, A. G. (2003) · doi:10.5555/944919.944955 · Cited by 148
Q-decomposition for reinforcement learning agents
Russell, S., & Zimdars, A. L. (2003) · International Conference on Machine Learning · Cited by 146
Tree based discretization for continuous state space reinforcement learning
Uther, W., & Veloso, M. (1998) · Cited by 145
The cross entropy method for fast policy search
Mannor, S., Rubinstein, R. Y., & Gat, Y. (2003) · Cited by 144
Convergence properties of the cross-entropy method for discrete optimization
Costa, A. N., Jones, O. D., & Kroese, D. P. (2006) · Operations Research Letters · doi:10.1016/j.orl.2006.11.005 · Cited by 143
Reinforcement learning as classification: leveraging modern classifiers
Lagoudakis, M. G., & Parr, R. (2003) · Cited by 134
Online Optimization in X-Armed Bandits
Bubeck, S., Stoltz, G., Szepesvári, C., & Munos, R. (2008) · Cited by 132
Efficient Non-linear Control Through Neuroevolution
Gomez, F. J., Schmidhuber, J., & Miikkulainen, R. P. (2006) · Lecture notes in computer science · doi:10.1007/11871842_64 · Cited by 129
Neural reinforcement learning for behaviour synthesis
Touzet, C. F. (1997) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(97)00042-0 · Cited by 121
Reinforcement learning for a biped robot based on a CPG-actor-critic method
Nakamura, Y., Mori, T., Sato, M.-A., & Ishii, S. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.01.002 · Cited by 119
Reinforcement Learning with Function Approximation Converges to a Region
Gordon, G. J. (2000) · Cited by 113
Clinical data based optimal STI strategies for HIV: a reinforcement learning approach
Ernst, D., Stan, G.-B. V., Gonçalves, J. M., & Wehenkel, L. A. (2006) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2006.377527 · Cited by 111
Self-citationFarahmand, A. M., Ghavamzadeh, M., Mannor, S., & Szepesvári, C. (2008) · Cited by 110
Finite time bounds for sampling based fitted value iteration
Szepesvári, C., & Munos, R. (2005) · International Conference on Machine Learning · doi:10.1145/1102351.1102462 · Cited by 108
Issues on Stability of ADP Feedback Controllers for Dynamical Systems
Balakrishnan, S. N., Ding, J., & Lewis, F. L. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.926599 · Cited by 108
Convergence Results for Some Temporal Difference Methods Based on Least Squares
Yu, H., & Bertsekas, D. P. (2009) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2009.2022097 · Cited by 105
Rust, A. G., Adams, R. G., George, S., & Bolouri, H. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44597-8 · Cited by 100
(1993) · Choice Reviews Online · doi:10.5860/choice.31-1560 · Cited by 100
Evaluation of Policy Gradient Methods and Variants on the Cart-Pole Benchmark
Riedmiller, M., Peters, J., & Schaal, S. (2007) · IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning · doi:10.1109/adprl.2007.368196 · Cited by 91
Reinforcement Learning: An OverviewPreprint
Murphy, K. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.05265 · Cited by 91
Optimistic Planning of Deterministic Systems
Hren, J.-F., & Munos, R. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-89722-4_12 · Cited by 87
Sparse Distributed Memories for On-Line Value-Based Reinforcement Learning
Ratitch, B., & Precup, D. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30115-8_33 · Cited by 76
Interpolation-based Q-learning
Szepesvári, C., & Smart, W. D. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015445 · Cited by 76
Reinforcement Learning in the Multi-Robot Domain
Matarić, M. J. (1997) · Springer eBooks · doi:10.1007/978-1-4757-6451-2_4 · Cited by 76
Batch reinforcement learning in a complex domain
Kalyanakrishnan, S., & Stone, P. (2007) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/1329125.1329241 · Cited by 75
Least-Squares Methods in Reinforcement Learning for Control
Lagoudakis, M. G., Parr, R., & Littman, M. L. (2002) · Lecture notes in computer science · doi:10.1007/3-540-46014-4_23 · Cited by 72
Reinforcement Learning with Replacing Eligibility Traces
Singh, S. P., & Sutton, R. S. (2007) · Springer eBooks · doi:10.1007/978-0-585-33656-5_7 · Cited by 66
Berenji, H. R., & Vengerov, D. (2003) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2003.814834 · Cited by 63
Improved Temporal Difference Methods with Linear Function Approximation
Si, J., Barto, A. G., Powell, W. B., & Wunsch, D. (2009) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470544785.ch9 · Cited by 61
Fuzzy modeling with multivariate membership functions: gray-box identification and control design
Abonyi, J., Babuška, R., & Szeifert, F. (2001) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/3477.956037 · Cited by 61
Incremental least-squares temporal difference learning
Geramifard, A. (2007) · doi:10.7939/r3-fkn7-mk03 · Cited by 53
Multidimensional Triangulation and Interpolation for Reinforcement Learning
Davies, S. (1996) · Cited by 53
Feature-based methods for large scale dynamic programming
Tsitsiklis, J. N., & Roy, B. V. (2002) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.1995.478954 · Cited by 38
Matheny, M. E., Resnic, F. S., Arora, N., & Lucila Ohno-machado, A. (2007) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2007.05.008 · Cited by 36
Convergence and Divergence in Standard and Averaging Reinforcement Learning
Wiering, M. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30115-8_44 · Cited by 32
Antos, A., Szepesvári, C., & Munos, R. (2006) · Lecture notes in computer science · doi:10.1007/11776420_42 · Cited by 23
Approximate dynamic programming using support vector regression
Bethke, B. M., How, J. P., & Ozdaglar, A. (2008) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.2008.4739322 · Cited by 21
Pérez-Uribe, A. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44597-8_37 · Cited by 18
Fuzzy Q-Learning with an adaptive representation
Waldock, A., & Carse, B. (2008) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzzy.2008.4630449 · Cited by 15
Hierarchical Average Reward Reinforcement Learning
GhavamzadehMohammad, & MahadevanSridhar (2007) · doi:10.5555/1314498.1390329 · Cited by 15
Sequential Decision Making Based on Direct Search
Schmidhuber, J. (2000) · Lecture notes in computer science · doi:10.1007/3-540-44565-x_10 · Cited by 14
Fuzzy Partition Optimization for Approximate Fuzzy Q-iteration
Buşoniu, L., Ernst, D., Schutter, B. D., & Babuška, R. (2008) · IFAC Proceedings Volumes · doi:10.3182/20080706-5-kr-1001.00949 · Cited by 8
Self-citationApproximate Policy Iteration for Closed-Loop Learning of Visual Tasks
Jodogne, S., Briquet, C., & Piater, J. (2006) · Lecture notes in computer science · doi:10.1007/11871842_23 · Cited by 6
Q-Learning in Simulated Robotic Soccer - Large State Spaces and Incomplete Information.
Tuyls, K., Maes, S., & Manderick, B. (2002) · Cited by 3
Cited by
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160
(2012) · IEEE Control Systems · doi:10.1109/mcs.2012.2214134 · Cited by 1,105
Algorithms for Reinforcement Learning
Szepesvári, C. (2010) · Synthesis Lectures on Artificial Intelligence and Machine Learning · doi:10.1007/978-3-031-01551-9 · Cited by 1,049
A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients
Grondman, I., Busoniu, L., Lopes, G. A. D., & Babuska, R. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2012.2218595 · Cited by 1,042
Self-citationApplications of deep learning and reinforcement learning to biological data
Mahmud, M., Kaiser, M. S., Hussain, A., & Vassanelli, S. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2790388 · Cited by 765
Metadata
- Authors
- Busoniu, L., Babuska, R., De Schutter, B., & Ernst, D.
- Venue
- CRC Press eBooks
- Publisher
- CRC Press
- Published
- 2017-07-28
- OpenAlex
- W1626155273
- Cited by
- 919
- References
- 144
- Type
- reference entry
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.