The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Markov decision processes: Discrete stochastic dynamic programming
Abstract
From the Publisher: The past decade has seen considerable theoretical and applied research on Markov decision processes, as well as the growing use of these models in ecology, economics, communications engineering, and other fields where outcomes are uncertain and sequential decision-making processes are needed. A timely response to this increased activity, Martin L. Puterman's new work provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models. It discusses all major research directions in the field, highlights many significant applications of Markov decision processes models, and explores numerous important topics that have previously been neglected or given cursory coverage in the literature. Markov Decision Processes focuses primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous-time discrete state models. The book is organized around optimality criteria, using a common framework centered on the optimality (Bellman) equation for presenting results. The results are presented in a theorem-proof format and elaborated on through both discussion and examples, including results that are not available in any other book. A two-state Markov decision process model, presented in Chapter 3, is analyzed repeatedly throughout the book and demonstrates many results an…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Rudin, W. (1973) · Cited by 13,740
Introduction to Stochastic Dynamic Programming.
Berry, D. A., & Ross, S. M. (1986) · Journal of the American Statistical Association · doi:10.2307/2289262 · Cited by 1,130
White, C. C., & White, D. J. (1989) · European Journal of Operational Research · doi:10.1016/0377-2217(89)90348-2 · Cited by 422
Dynamic programming: Models and applications
Denardo, E. V. (1982) · CERN Document Server (European Organization for Nuclear Research) · Cited by 411
Finding Optimal (s, S) Policies Is About As Simple As Evaluating a Single Policy
Zheng, Y., & Federgruen, A. (1991) · Operations Research · doi:10.1287/opre.39.4.654 · Cited by 317
Optimization over Time. Dynamic Programming and Stochastic Control. Volume 1
Thomas, L. C., & Whittle, P. (1983) · Journal of the Operational Research Society · doi:10.1057/jors.1983.56 · Cited by 162
Dynamic Programming and Its Application
Puterman, M. L. (1979) · Academic Press, Inc. eBooks · Cited by 57
Self-citationOn Homogeneous Markov Models with Continuous Time and Finite or Countable State Space
Yushkevic, A. A., & Fainberg, E. A. (1979) · Theory of Probability and Its Applications · doi:10.1137/1124014 · Cited by 34
Finite Dynamic Programming: An Approach to Finite Markov Decision Processes
White, D. J. (1978) · Andalas University Repository (Andalas University) · Cited by 24
Flagged first, then by citations
Cited by
Sorted by citations
Items 701–800 of 1,012
Dynamic Network Slicing for Scalable Fog Computing Systems With Energy Harvesting
Xiao, Y., & Krunz, M. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2871292 · Cited by 72
Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion
Gangapurwala, S., Mitchell, A., & Havoutis, I. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2979656 · Cited by 75
Kumar, A., McMahon, G., Reid, T., & Jain, N. (2020) · IEEE Control Systems · doi:10.1109/mcs.2020.3019151 · Cited by 58
Characterizing optimal empty container reposition policy in periodic-review shuttle service systems
Song, D.-P. (2006) · Journal of the Operational Research Society · doi:10.1057/palgrave.jors.2602150 · Cited by 72
Parallel Rollout for Online Solution of Partially Observable Markov Decision Processes
Chang, H. S., Givan, R. L., & Chong, E. K. P. (2004) · Discrete Event Dynamic Systems · doi:10.1023/b:disc.0000028199.78776.c4 · Cited by 72
Optimal control of a production–inventory system with customer impatience
Benjaafar, S., Gayon, J.-P., & Tepe, S. (2010) · Operations Research Letters · doi:10.1016/j.orl.2010.03.008 · Cited by 72
Approximate dynamic programming for missile defense interceptor fire control
Davis, M. T., Robbins, M. J., & Lunday, B. J. (2016) · European Journal of Operational Research · doi:10.1016/j.ejor.2016.11.023 · Cited by 72
Reinforcement learning using quantum Boltzmann machines
Crawford, D., Levit, A., Ghadermarzy, N., Oberoi, J. S., & Ronagh, P. (2018) · Quantum Information and Computation · doi:10.26421/qic18.1-2-3 · Cited by 117
A Convergent Form of Approximate Policy Iteration
Perkins, T. J., & Precup, D. (2002) · Cited by 72
Kumar, H., Koppel, A., & Ribeiro, A. (2023) · Machine Learning · doi:10.1007/s10994-023-06303-2 · Cited by 108
Remote State Estimation in the Presence of an Active Eavesdropper
Ding, K., Ren, X., Leong, A. S., Quevedo, D. E., & Shi, L. (2020) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2020.2980730 · Cited by 71
Distributed and Centralized Hybrid CSMA/CA-TDMA Schemes for Single-Hop Wireless Networks
Shrestha, B., Hossain, E., & Choi, K. W. (2014) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2014.2327102 · Cited by 71
Fei, Y., Wong, V. W. S., & Leung, V. C. M. (2005) · Mobile Networks and Applications · doi:10.1007/s11036-005-4464-2 · Cited by 71
Meet your expectations with guarantees: Beyond worst-case synthesis in quantitative games
Bruyère, V., Filiot, E., Randour, M., & Raskin, J.-F. (2016) · Information and Computation · doi:10.1016/j.ic.2016.10.011 · Cited by 71
Joint Physical-Layer and System-Level Power Management for Delay-Sensitive Wireless Communications
Mastronarde, N., & van der Schaar, M. (2012) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2012.36 · Cited by 71
Applications of Repeated Games in Wireless Networks: A Survey
Hoang, D. T., Lu, X., Niyato, D. T., Wang, P., Kim, D. I., & Han, Z. (2015) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2015.2445789 · Cited by 70
Holland, J. E., Kochenderfer, M. J., & Olson, W. A. (2013) · Air Traffic Control Quarterly · doi:10.2514/atcq.21.3.275 · Cited by 72
Trip-oriented stochastic optimal energy management strategy for plug-in hybrid electric bus
Du, Y., Zhao, Y., Wang, Q., Zhang, Y., & Huaicheng, X. (2016) · Energy · doi:10.1016/j.energy.2016.09.056 · Cited by 71
Verification and control of partially observable probabilistic systems
Norman, G., Parker, D., & Zou, X. (2017) · Real-Time Systems · doi:10.1007/s11241-017-9269-4 · Cited by 68
Second-Line Agents for Glycemic Control for Type 2 Diabetes: Are Newer Agents Better?
Zhang, Y., McCoy, R. G., Mason, J. E., Smith, S. A., Shah, N. D., & Denton, B. T. (2014) · Diabetes Care · doi:10.2337/dc13-1901 · Cited by 70
Diuk, C., Li, L. C., & Leffler, B. R. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553406 · Cited by 75
SMDP-Based Coordinated Virtual Machine Allocations in Cloud-Fog Computing Systems
Li, Q., Zhao, L., Gao, J., Liang, H., Zhao, L. Y., & Tang, X. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2818680 · Cited by 70
Remote Estimation Over a Packet-Drop Channel With Markovian State
Chakravorty, J., & Mahajan, A. (2019) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2019.2926160 · Cited by 67
On Decision Problems for Probabilistic Büchi Automata
Christel Baier, A., Bertrand, N., & Größer, M. (2008) · Springer eBooks · doi:10.1007/978-3-540-78499-9_21 · Cited by 70
Sensing-Driven Energy Purchasing in Smart Grid Cyber-Physical System
Tham, C., & Luo, T. (2013) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmca.2012.2224337 · Cited by 70
Löhndorf, N., & Minner, S. (2010) · Energy Systems · doi:10.1007/s12667-009-0007-4 · Cited by 70
Inverse Reinforcement Learning through Structured Classification
Klein, E., Geist, M., Piot, B., & Pietquin, O. (2012) · Cited by 70
Markov decision processes in natural resources management: Observability and uncertainty
Williams, B. K. (2009) · Ecological Modelling · doi:10.1016/j.ecolmodel.2008.12.023 · Cited by 70
Optimization methods to solve adaptive management problems
Chadès, I., Nicol, S., Rout, T. M., Péron, M., Dujardin, Y., Pichancourt, J., Hastings, A., & Hauser, C. E. (2016) · Theoretical Ecology · doi:10.1007/s12080-016-0313-0 · Cited by 70
Probabilistic Model Checking: Advances and Applications
Kwiatkowska, M., Norman, G., & Parker, D. (2017) · Springer eBooks · doi:10.1007/978-3-319-57685-5_3 · Cited by 70
Improved ordering of perishables: The value of stock-age information
Haijema, R., & Minner, S. (2019) · International Journal of Production Economics · doi:10.1016/j.ijpe.2018.03.008 · Cited by 70
A reinforcement learning model of selective visual attention
Minut, S., & Mahadevan, S. (2001) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/375735.376414 · Cited by 70
Index policies for the maintenance of a collection of machines by a set of repairmen
Glazebrook, K. D., Mitchell, H. M., & Ansell, P. S. (2004) · European Journal of Operational Research · doi:10.1016/j.ejor.2004.01.036 · Cited by 70
Uncertainty-Aware Action Advising for Deep Reinforcement Learning Agents
Da Silva, F. L., Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6036 · Cited by 69
Song, D., Dong, J., & Xu, J. (2013) · European Journal of Operational Research · doi:10.1016/j.ejor.2013.07.044 · Cited by 69
Real options approach to renewable energy investments in Mongolia
Detert, N., & Kotani, K. (2013) · Energy Policy · doi:10.1016/j.enpol.2012.12.003 · Cited by 69
El-Tantawy, S., & Abdulhai, B. (2012) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2012.6338707 · Cited by 69
Papakonstantinou, K. G., & Shinozuka, M. (2014) · Probabilistic Engineering Mechanics · doi:10.1016/j.probengmech.2014.06.002 · Cited by 69
Semi-markov decision problems and performance sensitivity analysis
Cao, X. (2003) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2003.811252 · Cited by 69
Value functions for RL-based behavior transfer: a comparative study
Taylor, M. E., Stone, P., & Liu, Y. (2005) · Cited by 69
Anticipatory Planning for Human-Robot Teams
Koppula, H. S., Jain, A., & Saxena, A. (2015) · Springer tracts in advanced robotics · doi:10.1007/978-3-319-23778-7_30 · Cited by 69
Lazy approximation for solving continuous finite-horizon MDPs
Li, L., & Littman, M. L. (2005) · Cited by 69
QoS Control Strategies for High-Quality Video Processing
Wüst, C. C., Steffens, L., Verhaegh, W., Bril, R. J., & Hentschel, C. (2005) · Real-Time Systems · doi:10.1007/s11241-005-0502-1 · Cited by 69
Solving generalized semi-Markov decision processes using continuous phase-type distributions
Younes, H. L. S., & Simmons, R. G. (2004) · Cited by 69
Sample Complexity of Multi-task Reinforcement LearningPreprint
Brunskill, E., & Li, L. C. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1309.6821 · Cited by 69
Better-than-Demonstrator Imitation Learning via Automatically-Ranked DemonstrationsPreprint
Brown, D. S., Goo, W., & Niekum, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.03976 · Cited by 70
Maximum Entropy RL (Provably) Solves Some Robust RL Problems
Eysenbach, B., & Levine, S. (2021) · arXiv · doi:10.48550/arxiv.2103.06257 · Cited by 69
Online Learning in Episodic Markovian Decision Processes by Relative Entropy Policy Search
Zimin, A., & Neu, G. (2013) · HAL · Cited by 68
Efficient reinforcement learning for robots using informative simulated priors
Cutler, M., & How, J. P. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139550 · Cited by 68
An Adaptive Response to Uncertainty Generates Positive and Negative Contrast Effects
McNamara, J. M., Fawcett, T. W., & Houston, A. I. (2013) · Science · doi:10.1126/science.1230599 · Cited by 68
Rechargeable sensor activation under temporally correlated events
Jaggi, N., Kar, K., & Krishnamurthy, A. (2007) · Wireless Networks · doi:10.1007/s11276-007-0091-0 · Cited by 68
Probabilistic event structures and domains
Varacca, D., Völzer, H., & Winskel, G. (2006) · Theoretical Computer Science · doi:10.1016/j.tcs.2006.01.015 · Cited by 68
Incremental quantitative verification for Markov decision processes
Kwiatkowska, M., Parker, D., & Hongyang Qu, B. (2011) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn.2011.5958249 · Cited by 67
PUMA: Planning Under Uncertainty with Macro-Actions
He, R., Brunskill, E., & Roy, N. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7749 · Cited by 68
Approximate Planning in POMDPs with Macro-Actions
Theocharous, G., & Kaelbling, L. P. (2003) · Cited by 68
Online learning of shaping rewards in reinforcement learning
Grześ, M., & Kudenko⋆, D. (2010) · Neural Networks · doi:10.1016/j.neunet.2010.01.001 · Cited by 68
A Survey on Model-Based Mission Planning and Execution for Autonomous Spacecraft
Tipaldi, M., & Glielmo, L. (2017) · IEEE Systems Journal · doi:10.1109/jsyst.2017.2720682 · Cited by 67
THE IMPORTANCE OF FUNCTIONAL FORM IN OPTIMAL CONTROL SOLUTIONS OF PROBLEMS IN POPULATION DYNAMICS
Runge, M. C., & Johnson, F. A. (2002) · Ecology · doi:10.1890/0012-9658(2002)083[1357:tioffi]2.0.co;2 · Cited by 68
Inductive Policy Selection for First-Order MDPsPreprint
Yoon, S., Fern, A., & Givan, R. L. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.0614 · Cited by 70
Agnostic System Identification for Model-Based Reinforcement LearningPreprint
Ross, S., & Bagnell, D. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1203.1007 · Cited by 68
Rakhsha, A., Radanović, G., Devidze, R., Zhu, X., & Singla, A. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.12909 · Cited by 73
A Game Theoretic Framework for Model Based Reinforcement LearningPreprint
Rajeswaran, A., Mordatch, I., & Kumar, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.07804 · Cited by 71
Robust Reinforcement Learning on State Observations with Learned Optimal AdversaryPreprint
Zhang, H., Chen, H., Boning, D. S., & Hsieh, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.08452 · Cited by 134
Cognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 72
“Practically, just as standardized software is used across robotics platforms (Quigley, 2009; Macenski et al., 2022), a framework for language agents would consolidate technical investment and improve compatibility.”
BackgroundMethodAverage Age of Information with Hybrid ARQ under a Resource Constraint
Ceran, E. T., Gündüz, D., & György, A. (2018) · OpenMETU (Middle East Technical University) · doi:10.1109/wcnc.2018.8377368 · Cited by 67
Counterfactual state explanations for reinforcement learning agents via generative deep learning
Olson, M., Khanna, R., Neal, L., Li, F., & Wong, W. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103455 · Cited by 80
The dynamic shortest path problem with anticipation
Thomas, B. W., & White, C. C. (2005) · European Journal of Operational Research · doi:10.1016/j.ejor.2005.09.019 · Cited by 67
Optimal Pricing and Admission Control in a Queueing System with Periodically Varying Parameters
Yoon, S., & Lewis, M. E. (2004) · Queueing Systems · doi:10.1023/b:ques.0000035313.20223.3f · Cited by 67
Learning and planning in environments with delayed feedback
Walsh, T. J., Nouri, A., Li, L., & Littman, M. L. (2008) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-008-9056-7 · Cited by 67
Potential-based shaping in model-based reinforcement learning
Asmuth, J., Littman, M. L., & Zinkov, R. (2008) · Cited by 67
Learning strategies in table tennis using inverse reinforcement learning
Muelling, K., Boularias, A., Mohler, B. J., Schölkopf, B., & Peters, J. (2014) · Biological Cybernetics · doi:10.1007/s00422-014-0599-1 · Cited by 72
Joint Inference of Reward Machines and Policies for Reinforcement Learning
Xu, Z., Gavran, I., Ahmad, Y., Majumdar, R., Neider, D., Topcu, U., & Wu, B. (2020) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v30i1.6756 · Cited by 67
An SMDP-Based Prioritized Channel Allocation Scheme in Cognitive Enabled Vehicular Ad Hoc Networks
Li, M., Zhao, L. Y., & Liang, H. (2017) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2676462 · Cited by 66
Deep reinforcement learning-based scheduling for roadside communication networks
Atallah, R. F., Assi, C. M., & Khabbaz, M. (2017) · International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt) · doi:10.23919/wiopt.2017.7959912 · Cited by 66
Error bounds for approximate value iteration
Munos, R. (2005) · Cited by 66
Quantitative Decision Analysis for Sport Fisheries Management
Peterson, J. T., & Evans, J. W. (2003) · Fisheries · doi:10.1577/1548-8446(2003)28[10:qdafsf]2.0.co;2 · Cited by 66
Algorithms and Representations for Reinforcement Learning
Engel, Y. (2005) · Cited by 66
State aggregation in Markov decision processes
Ren, Z., & Krogh, B. H. (2004) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2002.1184960 · Cited by 66
Prediction and Simulation of Human Mobility Following Natural Disasters
Song, X., Zhang, Q., Sekimoto, Y., Shibasaki, R., Yuan, N. J., & Xie, X. (2016) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2970819 · Cited by 65
Wideł, W., Audinot, M., Fila, B., & Pinchinat, S. (2019) · ACM Computing Surveys · doi:10.1145/3331524 · Cited by 65
Improving call admission policies in wireless networks
Ho, C., & Lea, C. (1999) · Wireless Networks · doi:10.1023/a:1019107329380 · Cited by 66
Extensions of Statecharts with probability, time, and stochastic timing
Jansen, D. N. (2003) · doi:10.3990/1.9783952285008 · Cited by 65
Processor-sharing models for integrated-services networks
Queija, R. N. (2000) · Centrum Wiskunde & Informatica (CWI), the national research institute for mathematics and computer science in the Netherlands · doi:10.6100/ir531047 · Cited by 66
Contextual Markov Decision Processes
Hallak, A., Di Castro, D., & Mannor, S. (2015) · arXiv · doi:10.48550/arxiv.1502.02259 · Cited by 65
Scaling Up Robust MDPs using Function Approximation
Tamar, A., Mannor, S., & Xu, H. (2014) · International Conference on Machine Learning · Cited by 65
Optimal Downlink–Uplink Scheduling of Wireless Networked Control for Industrial IoT
Huang, K., Liu, W., Li, Y., Vucetic, B. S., & Savkin, A. V. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2946878 · Cited by 65
MAMBA: A Multi-armed Bandit Framework for Beam Tracking in Millimeter-wave Systems
Aykin, I., Akgun, B., Feng, M., & Krunz, M. (2020) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom41043.2020.9155408 · Cited by 64
Bhandari, A., Scheller-Wolf, A. A., & Harchol‐Balter, M. (2008) · Management Science · doi:10.1287/mnsc.1070.0819 · Cited by 65
Adversary-driven state-based system security evaluation
LeMay, E., Unkenholz, W. L., Parks, D. L., Muehrcke, C., Keefe, K., & Sanders, W. H. (2010) · International Workshop on Security Measurements and Metrics (MetriSec) · doi:10.1145/1853919.1853926 · Cited by 65
On the Computational Complexity of Stochastic Controller Optimization in POMDPs
Vlassis, N., Littman, M. L., & Barber, D. (2012) · ACM Transactions on Computation Theory · doi:10.1145/2382559.2382563 · Cited by 65
Price-Directed Control of a Closed Logistics Queueing Network
Adelman, D. (2007) · Operations Research · doi:10.1287/opre.1070.0408 · Cited by 65
Optimal Generation and Storage Scheduling in the Presence of Renewable Forecast Uncertainties
Gast, N., Tomozei, D.-C., & Boudec, J. L. (2014) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2013.2285395 · Cited by 65
Direct value-approximation for factored MDPs
Schuurmans, D., & Patrascu, R. (2001) · Cited by 65
Overcoming indecision by changing the decision boundary.
Malhotra, G., Leslie, D. S., Ludwig, C. J. H., & Bogacz, R. (2017) · Journal of Experimental Psychology General · doi:10.1037/xge0000286 · Cited by 70
Decentralised Online Planning for Multi-Robot Warehouse Commissioning
Claes, D., Oliehoek, F. A., Baier, H., & Tuyls, K. (2017) · doi:10.5555/3091125.3091198 · Cited by 67
An analysis of Laplacian methods for value function approximation in MDPs
Petrik, M. (2007) · International Joint Conference on Artificial Intelligence · Cited by 65
A Vertical Handoff Decision Algorithm for Heterogeneous Wireless Networks
Stevens‐Navarro, E., Wong, V. W. S., & Lin, Y. (2007) · IEEE Wireless Communications and Networking Conference · doi:10.1109/wcnc.2007.590 · Cited by 65
DERP: A Deep Reinforcement Learning Cloud System for Elastic Resource Provisioning
Bitsakos, C., Konstantinou, I., & Koziris, N. (2018) · IEEE International Conference on Cloud Computing Technology and Science · doi:10.1109/cloudcom2018.2018.00020 · Cited by 68
Probabilistic planning with formal performance guarantees for mobile service robots
Lacerda, B., Faruq, F., Parker, D., & Hawes, N. (2019) · The International Journal of Robotics Research · doi:10.1177/0278364919856695 · Cited by 64
Sun, Q., Wu, Z. S., Gu, W., Zhu, T., Zhong, L., & Gao, T. (2021) · Energy · doi:10.1016/j.energy.2021.120367 · Cited by 63
Metadata
- Authors
- Hazeghi, K., & Puterman, M. L.
- ORCID
- Puterman, Martin L.
- Venue
- Journal of the American Statistical Association
- Publisher
- JSTOR
- Published
- 1995-03
- Volume
- 90
- Issue
- 429
- Pages
- 392
- ISSN
- 0162-1459 (ISSN-L)
- OpenAlex
- W2119567691
- Cited by
- 8,037
- References
- 12
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 86 · 2023: 145 · 2022: 233 · 2021: 616 · 2020: 674 · 2019: 595 · 2018: 480 · 2017: 459 · 2016: 450 · 2015: 507 · 2014: 451 · 2013: 420 · 2012: 392
- Type
- book review
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- Markov decision process, stochastic dynamic programming, discrete-time models, infinite horizon model, finite horizon model, Bellman equation, optimality criteria, average reward criterion, constrained models, sequential decision making under uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
