The DOI matches its Crossref record; 174 of the 174 references it cites were found in a registry or an index.
Algorithms for Reinforcement Learning
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
(1995) · Cited by 10,558
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,680
Finite-time Analysis of the Multiarmed Bandit Problem
Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002) · Machine Learning · doi:10.1023/a:1013689704352 · Cited by 6,033
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
(1999) · Cited by 5,147
(1991) · Springer eBooks · doi:10.1007/978-3-7091-2840-4 · Cited by 5,095
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1023/a:1022676722315 · Cited by 3,907
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,515
Dynamic Programming and Markov Processes.
Freimer, M., & Howard, R. A. (1961) · Journal of the American Statistical Association · doi:10.2307/2282268 · Cited by 3,378
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119
Bandit Based Monte-Carlo Planning
Kocsis, L., & Szepesvári, C. (2006) · Lecture Notes in Computer Science · doi:10.1007/11871842_29 · Cited by 2,855
Self-citationNatural Gradient Works Efficiently in Learning
Amari, S.-I. (1998) · Neural Computation · doi:10.1162/089976698300017746 · Cited by 2,681
Markov games as a framework for multi-agent reinforcement learning
Littman, M. L. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50027-1 · Cited by 2,614
Applied nonparametric regression
(1991) · Choice Reviews Online · doi:10.5860/choice.29-0364 · Cited by 2,580
A theory of cerebellar function
Albus, J. S. (1971) · Mathematical Biosciences · doi:10.1016/0025-5564(71)90051-4 · Cited by 2,539 · cited 1×
“Another choice that leads to binary features is tile coding (originally called CMAC, Albus, 1971, 1981).”
BackgroundIntroduction to Nonparametric Estimation
Tsybakov, A. B. (2009) · Springer Series in Statistics · doi:10.1007/b13794 · Cited by 2,486
Applied Nonparametric Regression
Härdle, W. (1990) · Cambridge University Press eBooks · doi:10.1017/ccol0521382483 · Cited by 2,457
Asymptotically efficient adaptive allocation rules
Lai, T. L., & Robbins, H. (1985) · Advances in Applied Mathematics · doi:10.1016/0196-8858(85)90002-8 · Cited by 2,406
Approximate Dynamic Programming: Solving the Curses of Dimensionality
Powell, W. B. (2011) · Wiley Series in Probability and Statistics · doi:10.1002/9781118029176 · Cited by 2,297
Konda, V. R., & Tsitsiklis, J. N. (2002) · Cited by 2,202
Some aspects of the sequential design of experiments
Robbins, H. (1952) · Bulletin of the American Mathematical Society · doi:10.1090/s0002-9904-1952-09620-8 · Cited by 2,184
Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning)
Rasmussen, C. E., & Williams, C. K. I. (2005) · The MIT Press eBooks · Cited by 1,829
Acceleration of Stochastic Approximation by Averaging
Polyak, B. T., & Juditsky, A. B. (1992) · SIAM Journal on Control and Optimization · doi:10.1137/0330046 · Cited by 1,827 · cited 1×
“In fact, a simple method, called iterate-averaging due to Polyak and Juditsky (1992), is known to achieve the best possible asymptotic rate of convergence.”
MethodA Distribution-Free Theory of Nonparametric Regression
Györfi, L., Kohler, M., Krzyżak, A., & Walk, H. (2002) · Springer Series in Statistics · doi:10.1007/b97848 · Cited by 1,738
Stochastic optimal control : the discrete time case
Bertsekas, D. P., & Shreve, S. E. (2007) · Cited by 1,654
Approximate Dynamic Programming: Solving the Curses of Dimensionality
Powell, W. B. (2007) · Wiley Series in Probability and Statistics · doi:10.1002/9780470182963 · Cited by 1,566
Self-improving reactive agents based on reinforcement learning, planning and teaching
Lin, L.-J. (1992) · Machine Learning · doi:10.1007/bf00992699 · Cited by 1,564 · cited 2×
“A non-exhaustive list includes the use of adaptive step-sizes (Sutton, 1992; George and Powell, 2006), normalizing the updates (Bradtke, 1994) or reusing previous samples (Lin, 1992).”
MethodOn-line Q-learning Using Connectionist Systems
Rummery, G. A. (1994) · Cited by 1,477
Tree-Based Batch Mode Reinforcement Learning
Ernst, D., Geurts, P., & Wehenkel, L. A. (2005) · Journal of Machine Learning Research · doi:10.5555/1046920.1088690 · Cited by 1,352
An analysis of temporal-difference learning with function approximation
Tsitsiklis, J. N., & Van Roy, B. (1997) · IEEE Transactions on Automatic Control · doi:10.1109/9.580874 · Cited by 1,194
The Tradeoffs of Large-Scale Learning
Bottou, L., & Bousquet, O. (2011) · Optimization for Machine Learning · doi:10.7551/mitpress/8996.003.0015 · Cited by 1,128
Introduction to Empirical Processes and Semiparametric Inference
Kosorok, M. R. (2008) · Springer Series in Statistics · doi:10.1007/978-0-387-74978-5 · Cited by 1,089
Multi-Armed Bandit Allocation Indices.
Bather, J. A., & Gittins, J. C. (1990) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982809 · Cited by 998
Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (UAI 2010)
Adams, R. P., Dahl, G. E., & Murray, I. (2010) · Cited by 966
Reinforcement Learning and Dynamic Programming Using Function Approximators
Busoniu, L., Babuska, R., De Schutter, B., & Ernst, D. (2017) · CRC Press eBooks · doi:10.1201/9781439821091 · Cited by 919
Residual Algorithms: Reinforcement Learning with Function Approximation
Baird, L. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50013-x · Cited by 918
Shapiro, A. (2003) · Handbooks in Operations Research and Management Science · doi:10.1016/s0927-0507(03)10006-0 · Cited by 916 · cited 3×
“Using the terminology of stochastic programming, LSTD can be seen to use sample average approximation (Shapiro, 2003).”
MethodNear-Optimal Reinforcement Learning in Polynomial Time
Kearns, M., & Singh, S. (2002) · Machine Learning · doi:10.1023/a:1017984413808 · Cited by 792
TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play
Tesauro, G. (1994) · Neural Computation · doi:10.1162/neco.1994.6.2.215 · Cited by 787
Near-optimal Regret Bounds for Reinforcement Learning
Jaksch, T., Ortner, R., & Auer, P. L. (2010) · Cited by 784
Kakade, S. M. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 775
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
Jaakkola, T., Jordan, M. I., & Singh, S. P. (1994) · Neural Computation · doi:10.1162/neco.1994.6.6.1185 · Cited by 770
Riedmiller, M. (2005) · Lecture Notes in Computer Science · doi:10.1007/11564096_32 · Cited by 757
Temporal credit assignment in reinforcement learning
Sutton, R. S. (1984) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 749
Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach
Boyan, J. A., & Littman, M. L. (1994) · Cited by 738
Peters, J., & Schaal, S. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.11.026 · Cited by 738
Konda, V. R., & Tsitsiklis, J. N. (2003) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012901385691 · Cited by 709
The Linear Programming Approach to Approximate Dynamic Programming
de Farias, D. P., & Van Roy, B. (2003) · Operations Research · doi:10.1287/opre.51.6.850.24925 · Cited by 687
Multi‐Armed Bandit Allocation Indices
Gittins, J., Glazebrook, K., & Weber, R. (2011) · Wiley eBooks · doi:10.1002/9780470980033 · Cited by 684
On the Sample Complexity of Reinforcement Learning
Kakade, S. M. (2003) · OpenGrey (Institut de l'Information Scientifique et Technique) · Cited by 676
Approximately Optimal Approximate Reinforcement Learning
Kakade, S. M., & Langford, J. C. (2002) · Cited by 617
Regression methods for pricing complex American-style options
Tsitsiklis, J. N., & Van Roy, B. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.935083 · Cited by 612
Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms
Singh, S., Jaakkola, T., Littman, M. L., & Szepesvári, C. (2000) · Machine Learning · doi:10.1023/a:1007678930559 · Cited by 609
Self-citationPredictive Representations of State
Littman, M. L., & Sutton, R. S. (2001) · neural information processing systems · Cited by 600
Linear Least-Squares algorithms for temporal difference learning
Bradtke, S. J., & Barto, A. G. (1996) · Machine Learning · doi:10.1007/bf00114723 · Cited by 599
Reinforcement learning with replacing eligibility traces
Singh, S. P., & Sutton, R. S. (1996) · Machine Learning · doi:10.1007/bf00114726 · Cited by 575
Exploration–exploitation tradeoff using variance estimates in multi-armed bandits
Audibert, J.-Y., Munos, R., & Szepesvári, C. (2009) · Theoretical Computer Science · doi:10.1016/j.tcs.2009.01.016 · Cited by 569
Self-citationPolicy gradient reinforcement learning for fast quadrupedal locomotion
Kohl, N., & Stone, P. (2004) · IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004 · doi:10.1109/robot.2004.1307456 · Cited by 564
Borkar, V. S. (2008) · Texts and Readings in Mathematics · doi:10.1007/978-93-86279-38-5 · Cited by 561
Natural actor–critic algorithms
Bhatnagar, S., Sutton, R. S., Ghavamzadeh, M., & Lee, M. (2009) · Automatica · doi:10.1016/j.automatica.2009.07.008 · Cited by 559
Proceedings of the European Conference on Machine Learning
Brazdil, P. (1993) · Cited by 549
An Application of Reinforcement Learning to Aerobatic Helicopter Flight
Abbeel, P., Coates, A., Quigley, M., & Ng, A. Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0006 · Cited by 535
Universal Artificial Intellegence
Hutter, M. (2005) · Texts in Theoretical Computer Science An EATCS Series · doi:10.1007/b138233 · Cited by 523
Generalization in Reinforcement Learning: Safely Approximating the Value Function
Boyan, J. A., & Moore, A. (1994) · Cited by 508
Improving Elevator Performance Using Reinforcement Learning
Crites, R. H., & Barto, A. G. (1995) · Cited by 499
Monte Carlo and Quasi-Monte Carlo Sampling
(2009) · Springer Series in Statistics · doi:10.1007/978-0-387-78165-5 · Cited by 497
Fast gradient-descent methods for temporal-difference learning with linear function approximation
Sutton, R. S., Maei, H. R., Precup, D., Bhatnagar, S., Silver, D., Szepesvári, C., & Wiewiora, E. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553501 · Cited by 496
Self-citationOnline Planning Algorithms for POMDPs
Ross, S., Pineau, J., Paquet, S., & Chaib-draa, B. (2008) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2567 · Cited by 496
Finite-Time Bounds for Fitted Value Iteration
Munos, R., & Szepesvári, C. (2008) · SZTAKI Publication Repository (Hungarian Academy of Sciences) · doi:10.5555/1390681.1390708 · Cited by 474
Self-citation(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303765208377 · Cited by 467
Asynchronous stochastic approximation and Q-learning
Tsitsiklis, J. N. (1994) · Machine Learning · doi:10.1007/bf00993306 · Cited by 453
Albus, J. S. (1981) · DigitalGeorgetown (Georgetown University Library) · Cited by 448
Dynamic Programming and Optimal Control. Volume 1: Dynamic Programming and Optimal Control. Volume 2
Smith, D. K. (1996) · Journal of the Operational Research Society · doi:10.1057/jors.1996.103 · Cited by 448
Stable Function Approximation in Dynamic Programming
Gordon, G. J. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50040-2 · Cited by 446
Feature-based methods for large scale dynamic programming
Tsitsiklis, J. N., & van Roy, B. (1996) · Machine Learning · doi:10.1007/bf00114724 · Cited by 432
Using Randomization to Break the Curse of Dimensionality
Rust, J. (1997) · Econometrica · doi:10.2307/2171751 · Cited by 407
Proceedings of the 26th Annual International Conference on Machine Learning
(2009) · doi:10.1145/1553374 · Cited by 399
Stochastic approximation with two time scales
Borkar, V. S. (1997) · Systems & Control Letters · doi:10.1016/s0167-6911(97)90015-3 · Cited by 389
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/1532443041827907 · Cited by 388
PAC model-free reinforcement learning
Strehl, A. L., Li, L., Wiewiora, E., Langford, J., & Littman, M. L. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143955 · Cited by 384
Likelihood ratio gradient estimation for stochastic systems
Glynn, P. W. (1990) · Communications of the ACM · doi:10.1145/84537.84552 · Cited by 381
A reinforcement learning approach to job-shop scheduling
Zhang, W., & Dietterich, T. G. (1995) · Cited by 378
Random projection trees and low dimensional manifolds
Dasgupta, S., & Freund, Y. (2008) · Proceedings of the fortieth annual ACM symposium on Theory of computing · doi:10.1145/1374376.1374452 · Cited by 375
On Constraint Sampling in the Linear Programming Approach to Approximate Dynamic Programming
de Farias, D. P., & Van Roy, B. (2004) · Mathematics of Operations Research · doi:10.1287/moor.1040.0094 · Cited by 373
Kernel-Based Reinforcement Learning
(2017) · Encyclopedia of Machine Learning and Data Mining · doi:10.1007/978-1-4899-7687-1_100235 · Cited by 367
Dearden, R., Friedman, N., & Russell, S. (1998) · Cited by 363
PEGASUS: A Policy Search Method for Large MDPs and POMDPs
Ng, A. Y., & Jordan, M. I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3878 · Cited by 360
A Bayesian Framework for Reinforcement Learning
Strens, M. J. A. (2000) · International Conference on Machine Learning · Cited by 358
Reinforcement learning with Gaussian processes
Engel, Y., Mannor, S., & Meir, R. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102377 · Cited by 335
Tsitsiklis, J. N., & van Roy, B. (1999) · IEEE Transactions on Automatic Control · doi:10.1109/9.793723 · Cited by 326
Even-Dar, E., & Mansour, Y. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44581-1_39 · Cited by 314
Reinforcement Learning for Humanoid Robotics
Peters, J., Vijayakumar, S., & Schaal, S. (2003) · Cited by 309
The Sample Complexity of Exploration in the Multi-Armed Bandit Problem
Mannor, S., & Tsitsiklis, J. N. (2004) · doi:10.5555/1005332.1005355 · Cited by 307
PAC Bounds for Multi-armed Bandit and Markov Decision Processes
Even-Dar, E., Mannor, S., & Mansour, Y. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-45435-7_18 · Cited by 298
Technical Update: Least-Squares Temporal Difference Learning
Boyan, J. A. (2002) · Machine Learning · doi:10.1023/a:1017936530646 · Cited by 294
The MAXQ Method for Hierarchical Reinforcement Learning
Dietterich, T. G. (1998) · Cited by 285
An analytic solution to discrete Bayesian reinforcement learning
Poupart, P., Vlassis, N., Hoey, J., & Regan, K. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143932 · Cited by 278
Reinforcement Learning with Soft State Aggregation
Singh, S. P., Jaakkola, T., & Jordan, M. I. (1994) · Cited by 274
Kernel-Based Least Squares Policy Iteration for Reinforcement Learning
Xu, X., Hu, D., & Lu, X. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2007.899161 · Cited by 268
Efficient Exploration in Reinforcement Learning
(2012) · SpringerReference · doi:10.1007/springerreference_178956 · Cited by 257
Kernel-Based Reinforcement Learning
Ormoneit, D., & Sen, Ś. (2002) · Machine Learning · doi:10.1023/a:1017928328829 · Cited by 255
Stochastic Learning and Optimization - A Sensitivity-Based Approach
Cao, X. (2008) · IFAC Proceedings Volumes · doi:10.3182/20080706-5-kr-1001.00589 · Cited by 250
Model-Based Bayesian Exploration
Dearden, R., Friedman, N., & André, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6690 · Cited by 221
Antos, A., Szepesvári, C., & Munos, R. (2008) · Machine Learning · doi:10.1007/s10994-007-5038-2 · Cited by 219
Self-citationReinforcement Learning for Dynamic Channel Allocation in Cellular Telephone Systems
Singh, S. P., & Bertsekas, D. P. (1996) · Cited by 214
Fitted Q-iteration in continuous action-space MDPs
Antos, A., Szepesvári, C., & Munos, R. (2007) · HAL · Cited by 188
Self-citationBasis Function Adaptation in Temporal Difference Reinforcement Learning
Menache, I., Mannor, S., & Shimkin, N. (2005) · Annals of Operations Research · doi:10.1007/s10479-005-5732-z · Cited by 184
Least Squares Policy Evaluation Algorithms with Linear Function Approximation
Nedić, A., & Bertsekas, D. P. (2003) · Discrete Event Dynamic Systems · doi:10.1023/a:1022192903948 · Cited by 182
An Approximate Dynamic Programming Algorithm for Large-Scale Fleet Management: A Case Application
Simão, H. P., Day, J., George, A. P., Gifford, T., Nienow, J., & Powell, W. B. (2009) · Transportation Science · doi:10.1287/trsc.1080.0238 · Cited by 182
Local gain adaptation in stochastic gradient descent
Schraudolph, N. N. (1999) · International Conference on Artificial Neural Networks · doi:10.1049/cp:19991170 · Cited by 180
A Unified Analysis of Value-Function-Based Reinforcement-Learning Algorithms
Szepesvári, C., & Littman, M. L. (1999) · Neural Computation · doi:10.1162/089976699300016070 · Cited by 179
Self-citationRegularization and feature selection in least-squares temporal difference learning
Kolter, J. Z., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553442 · Cited by 176
Toward Off-Policy Learning Control with Function Approximation
Maei, H. R., ri, C. S., Bhatnagar, S., & Sutton, R. (2010) · Cited by 175
Consideration of Risk in Reinforcement Learning
Heger, M. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50021-0 · Cited by 174
Asynchronous Stochastic Approximations
Borkar, V. S. (1998) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012995282784 · Cited by 167
Convergent Temporal-Difference Learning with Arbitrary Smooth Function Approximation
Bhatnagar, S., Precup, D., Silver, D., Sutton, R. S., Maei, H. R., & Szepesvári, C. (2009) · neural information processing systems · Cited by 167
Self-citationAutomatic basis function construction for approximate dynamic programming and reinforcement learning
Keller, P. W., Mannor, S., & Precup, D. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143901 · Cited by 164
Parr, R., Li, L. C., Taylor, G., Painter-Wakefield, C., & Littman, M. L. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390251 · Cited by 164
Adaptive stepsizes for recursive estimation with applications in approximate dynamic programming
George, A. P., & Powell, W. B. (2006) · Machine Learning · doi:10.1007/s10994-006-8365-9 · Cited by 164
Approximate Planning in Large POMDPs via Reusable Trajectories
Kearns, M. J., Mansour, Y., & Ng, A. Y. (1999) · neural information processing systems · Cited by 163
Self-Improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching
Lin, L.-J. (1992) · Machine Learning · doi:10.1023/a:1022628806385 · Cited by 157
Efficient Reinforcement Learning Using Recursive Least-Squares Methods
Xu, X., He, H., & Hu, D. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.946 · Cited by 152
Sutton, R. S., Maei, H. R., & Szepesvári, C. (2008) · neural information processing systems · Cited by 152
Self-citationBagnell, J. A., & Schneider, J. (2018) · Figshare · doi:10.1184/r1/6552458 · Cited by 148
Average cost temporal-difference learning
Tsitsiklis, J. N., & Roy, B. V. (1999) · Automatica · doi:10.1016/s0005-1098(99)00099-0 · Cited by 140
Dyna-Style Planning with Linear Function Approximation and Prioritized SweepingPreprint
Sutton, R. S., Szepesvári, C., Geramifard, A., & Bowling, M. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.3285 · Cited by 139
Self-citationAn Upper Bound on the Loss from Approximate Optimal-Value Functions
Singh, S. P., & Yee, R. C. (1994) · Machine Learning · doi:10.1007/bf00993308 · Cited by 135
A theoretical analysis of Model-Based Interval Estimation
Strehl, A. L., & Littman, M. L. (2005) · International Conference on Machine Learning · doi:10.1145/1102351.1102459 · Cited by 135
Risk-Sensitive Optimal Control for Markov Decision Processes with Monotone Cost
Borkar, V. S., & Meyn, S. (2002) · Mathematics of Operations Research · doi:10.1287/moor.27.1.192.334 · Cited by 133
Analyzing feature generation for value-function approximation
Parr, R., Painter-Wakefield, C., Li, L., & Littman, M. L. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273589 · Cited by 133
REGAL: A Regularization based Algorithm for Reinforcement Learning in Weakly Communicating MDPsPreprint
Bartlett, P. L., & Tewari, A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2661 · Cited by 132
Reinforcement learning of local shape in the game of go
Silver, D., Sutton, R. S., & Müller, M. (2007) · UCL Discovery (University College London) · Cited by 122
Reinforcement learning for long-run average cost
Gosavi, A. (2003) · European Journal of Operational Research · doi:10.1016/s0377-2217(02)00874-3 · Cited by 122
(2000) · Applied Physics Letters · doi:10.1162/jmlr.2003.4.6.1107 · Cited by 120
RL-Glue: Language-Independent Software for Reinforcement-Learning Experiments
Tanner, B. K., & White, A. (2009) · Journal of Machine Learning Research · doi:10.5555/1577069.1755857 · Cited by 119
The Asymptotic Convergence-Rate of Q-learning
Szepesvári, C. (1997) · Cited by 111
Self-citationFarahmand, A. M., Ghavamzadeh, M., Mannor, S., & Szepesvári, C. (2008) · Cited by 110
Self-citationExploration in metric state spaces
Kakade, S. M., Kearns, M. J., & Langford, J. C. (2003) · ScholarlyCommons (University of Pennsylvania) · Cited by 107
Model-based reinforcement learning with nearly tight exploration complexity bounds
Szita, I., & ri, C. S. (2010) · Cited by 106
Maei, H. R., & Sutton, R. S. (2010) · Artificial General Intelligence · doi:10.2991/agi.2010.22 · Cited by 106
An intrinsic reward mechanism for efficient exploration
Şimşek, Ö., & Barto, A. G. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143949 · Cited by 97
Performance Loss Bounds for Approximate Value Iteration with State Aggregation
Roy, B. V. (2006) · Mathematics of Operations Research · doi:10.1287/moor.1060.0188 · Cited by 96
Markov Decision Processes with Arbitrary Reward Processes
Yu, J. Y., Mannor, S., & Shimkin, N. (2009) · Mathematics of Operations Research · doi:10.1287/moor.1090.0397 · Cited by 96
Gain Adaptation Beats Least Squares
Sutton, R. S. (2006) · Cited by 92
Temporal Dierences-Based Policy Iteration and Applications in Neuro-Dynamic Programming 1
Bertsekas, D. P., & Ioe, S. (1997) · Cited by 92
Online linear regression and its application to model-based reinforcement learning
Strehl, A. L., & Littman, M. L. (2007) · Cited by 87
Interpolation-based Q-learning
Szepesvári, C., & Smart, W. D. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015445 · Cited by 76
Self-citationSzita, I., & Lörincz, A. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390288 · Cited by 76
Hierarchical POMDP Controller Optimization by Likelihood MaximizationPreprint
Toussaint, M., Charlin, L., & Poupart, P. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.3291 · Cited by 74
A Convergent Form of Approximate Policy Iteration
Perkins, T. J., & Precup, D. (2002) · Cited by 72
Model-Based Bayesian Reinforcement Learning in Large Structured Domains
Ross, S., & Pineau, J. (2012) · PubMed · doi:10.48550/arxiv.1206.3281 · Cited by 69
On-line searching and navigation
Berman, P. (1998) · Lecture notes in computer science · doi:10.1007/bfb0029571 · Cited by 67
Improved Temporal Difference Methods with Linear Function Approximation
Si, J., Barto, A. G., Powell, W. B., & Wunsch, D. (2009) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470544785.ch9 · Cited by 61
Hütter, M. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44795-4_20 · Cited by 61
Learning Representation and Control in Markov Decision Processes: New Frontiers
Sridhar, M. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000003 · Cited by 60
Cross channel optimized marketing by reinforcement learning
Abe, N., Verma, N. K., Apte, C., et al. (2004) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1014052.1016912 · Cited by 56
Multi-resolution Exploration in Continuous Spaces
Nouri, A., & Littman, M. L. (2008) · neural information processing systems · Cited by 56
Reinforcement learning in the presence of rare events
Frank, J., Mannor, S., & Precup, D. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390199 · Cited by 48
Model-Based Exploration in Continuous State Spaces
Jong, N. K., & Stone, P. (2007) · Springer eBooks · doi:10.1007/978-3-540-73580-9_21 · Cited by 43
Incremental dynamic programming for on-line adaptive optimal control
Bradtke, S. J. (1995) · Cited by 37
Learning Exercise Policies for American Options
Li, Y., Szepesvári, C., & Schuurmans, D. (2009) · Cited by 36
Self-citationEstimating Taxi-out times with a reinforcement learning algorithm
Balakrishna, P., Ganesan, R., Sherry, L., et al. (2008) · IEEE/AIAA Digital Avionics Systems Conference (DASC) · doi:10.1109/dasc.2008.4702812 · Cited by 32
Stable Dual Dynamic Programming
Wang, T., Lizotte, D., Bowling, M., & Schuurmans, D. (2008) · Cited by 24
On the Almost Sure Rate of Convergence of Linear Stochastic Approximation Algorithms
Tadić, V. B. (2004) · IEEE Transactions on Information Theory · doi:10.1109/tit.2003.821971 · Cited by 22
Rusmevichientong, P., Salisbury, J. A., Truss, L. T., Roy, B. V., & Glynn, P. W. (2006) · Journal of Revenue and Pricing Management · doi:10.1057/palgrave.rpm.5160011 · Cited by 17
Faster Near-Optimal Reinforcement Learning: Adding Adaptiveness to the E3 Algorithm
Domingo, C. (1999) · Lecture notes in computer science · doi:10.1007/3-540-46769-6_20 · Cited by 7
Learning and exploitation do not conflict under minimax optimality
Szepesvári, C. (1997) · Lecture notes in computer science · doi:10.1007/3-540-62858-4_89 · Cited by 7
Self-citation
Cited by
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
“Uncertainty is important in reinforcement learning (RL) as well (Szepesvári, 2010).”
BackgroundRegret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
Bubeck, S. (2012) · now publishers, Inc. eBooks · doi:10.1561/9781601986276 · Cited by 1,404
Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control
Chu, T., Wang, J., Codeca, L., & Li, Z. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2901791 · Cited by 1,046
A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients
Grondman, I., Busoniu, L., Lopes, G. A. D., & Babuska, R. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2012.2218595 · Cited by 1,042
Lattimore, T., & Szepesvári, C. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108571401 · Cited by 976
Self-citation
Metadata
- Authors
- Szepesvári, C.
- Venue
- Synthesis Lectures on Artificial Intelligence and Machine Learning
- Publisher
- Springer International Publishing
- Published
- 2010
- Volume
- 4
- Issue
- 1
- Pages
- 1-103
- ISSN
- 1939-4608 (ISSN-L)
- 1939-4616
- OpenAlex
- W2073384958
- Cited by
- 1,049
- References
- 174
- FWCI
- 20.97
- Type
- book
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.