The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Markov decision processes: Discrete stochastic dynamic programming
Abstract
From the Publisher: The past decade has seen considerable theoretical and applied research on Markov decision processes, as well as the growing use of these models in ecology, economics, communications engineering, and other fields where outcomes are uncertain and sequential decision-making processes are needed. A timely response to this increased activity, Martin L. Puterman's new work provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models. It discusses all major research directions in the field, highlights many significant applications of Markov decision processes models, and explores numerous important topics that have previously been neglected or given cursory coverage in the literature. Markov Decision Processes focuses primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous-time discrete state models. The book is organized around optimality criteria, using a common framework centered on the optimality (Bellman) equation for presenting results. The results are presented in a theorem-proof format and elaborated on through both discussion and examples, including results that are not available in any other book. A two-state Markov decision process model, presented in Chapter 3, is analyzed repeatedly throughout the book and demonstrates many results an…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Rudin, W. (1973) · Cited by 13,740
Introduction to Stochastic Dynamic Programming.
Berry, D. A., & Ross, S. M. (1986) · Journal of the American Statistical Association · doi:10.2307/2289262 · Cited by 1,130
White, C. C., & White, D. J. (1989) · European Journal of Operational Research · doi:10.1016/0377-2217(89)90348-2 · Cited by 422
Dynamic programming: Models and applications
Denardo, E. V. (1982) · CERN Document Server (European Organization for Nuclear Research) · Cited by 411
Finding Optimal (s, S) Policies Is About As Simple As Evaluating a Single Policy
Zheng, Y., & Federgruen, A. (1991) · Operations Research · doi:10.1287/opre.39.4.654 · Cited by 317
Optimization over Time. Dynamic Programming and Stochastic Control. Volume 1
Thomas, L. C., & Whittle, P. (1983) · Journal of the Operational Research Society · doi:10.1057/jors.1983.56 · Cited by 162
Dynamic Programming and Its Application
Puterman, M. L. (1979) · Academic Press, Inc. eBooks · Cited by 57
Self-citationOn Homogeneous Markov Models with Continuous Time and Finite or Countable State Space
Yushkevic, A. A., & Fainberg, E. A. (1979) · Theory of Probability and Its Applications · doi:10.1137/1124014 · Cited by 34
Finite Dynamic Programming: An Approach to Finite Markov Decision Processes
White, D. J. (1978) · Andalas University Repository (Andalas University) · Cited by 24
Flagged first, then by citations
Cited by
Sorted by citations
Items 201–300 of 1,012
Reinforcement Learning in Finite MDPs: PAC Analysis
Strehl, A. L., Li, L. C., & Littman, M. L. (2009) · doi:10.5555/1577069.1755867 · Cited by 203
PROBMELA: a modeling language for communicating probabilistic processes
Christel Baier, A., Ciesinski, F., & Grosser, M. (2005) · ACM/IEEE International Conference on Formal Methods and Models for Co-Design (MEMOCODE) · doi:10.1109/memcod.2004.1459815 · Cited by 193
Gatekeepers and Referrals in Services
Shumsky, R. A., & Pinker, E. J. (2003) · Management Science · doi:10.1287/mnsc.49.7.839.16387 · Cited by 192
Decision-Theoretic, High-Level Agent Programming in the Situation Calculus
Boutilier, C. E., Reiter, R., Mikhail Soutchanski, B., & Thrun, S. (2000) · Cited by 193
Experimental evaluation of model-free reinforcement learning algorithms for continuous HVAC control
Biemann, M., Scheller, F., Liu, X., & Huang, L. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.117164 · Cited by 196
Reinforcement Learning in Continuous State and Action Spaces
van Hasselt, H. P. (2012) · Adaptation, learning, and optimization · doi:10.1007/978-3-642-27645-3_7 · Cited by 197
Formal models and algorithms for decentralized decision making under uncertainty
Seuken, S., & Shlomo Zilberstein, B. (2008) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-007-9026-5 · Cited by 190
Stochastic model predictive control — how does it work?
Heirung, T. A. N., Paulson, J. A., O’Leary, J., & Mesbah, A. (2017) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2017.10.026 · Cited by 190
Benjaafar, S., & Elhafsi, M. (2006) · Management Science · doi:10.1287/mnsc.1060.0588 · Cited by 189
Information Sharing to Improve Retail Product Freshness of Perishables
Ferguson, M. E., & Ketzenberg, M. E. (2006) · Production and Operations Management · doi:10.1111/j.1937-5956.2006.tb00003.x · Cited by 189
Conditional Computation in Neural Networks for faster modelsPreprint
Bengio, E., Bacon, P., Pineau, J., & Precup, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06297 · Cited by 345
Reinforcement learning, conditioning, and the brain: Successes and challenges
Maia, T. V. (2009) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.9.4.343 · Cited by 191
Apprenticeship learning using linear programming
Syed, U., Bowling, M., & Schapire, R. E. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390286 · Cited by 188
3D UAV Trajectory and Data Collection Optimisation Via Deep Reinforcement Learning
Nguyen, K. K., Duong, T. Q., Do‐Duy, T., Claußen, H., & Hanzo, L. (2022) · IEEE Transactions on Communications · doi:10.1109/tcomm.2022.3148364 · Cited by 187
Learning Finite-State Controllers for Partially Observable EnvironmentsPreprint
Meuleau, N., Peshkin, L., Kim, K.-E., & Kaelbling, L. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6721 · Cited by 198
Apprenticeship Learning using Inverse Reinforcement Learning and Gradient MethodsPreprint
Neu, G., & Szepesvári, C. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.5264 · Cited by 186
Optimal Energy Commitments with Storage and Intermittent Supply
Kim, J. H., & Powell, W. B. (2011) · Operations Research · doi:10.1287/opre.1110.0971 · Cited by 185
Markov decision processes with applications in wireless sensor networks: A survey
ALSHEIKH, A. M., Hoang, D. T., Niyato, D. T., & Tan, H.-P. (2015) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 189
Active learning: theory and applications
Koller, D., & Tong, S. (2001) · Cited by 182
Near-optimal Regret Bounds for Reinforcement Learning
Auer, P. L., Jaksch, T., & Ortner, R. (2008) · Cited by 184
A Review on energy management schemes in energy harvesting wireless sensor networks
Babayo, A. A., Anisi, M. H., & Ali, I. (2017) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2017.03.124 · Cited by 184
Prediction of human emergency behavior and their mobility following large-scale disaster
Song, X., Zhang, Q., Sekimoto, Y., & Shibasaki, R. (2014) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2623330.2623628 · Cited by 183
Traffic flow optimization: A reinforcement learning approach
Walraven, E., Spaan, M. T. J., & Bakker, B. (2016) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2016.01.001 · Cited by 183
Discovering Hierarchy in Reinforcement Learning with HEXQ
Hengst, B. (2002) · Cited by 184
Optimal Control of Wake Up Mechanisms of Femtocells in Heterogeneous Networks
Saker, L., Elayoubi, S. E., Combes, R., & Chahed, T. (2012) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2012.120415 · Cited by 182
A Stochastic Inventory Model with Trade Credit
Gupta, D., & Wang, L. (2008) · Manufacturing & Service Operations Management · doi:10.1287/msom.1070.0191 · Cited by 181
Multiagent Learning: Basics, Challenges, and Prospects
Tuyls, K., & Weiß, G. (2012) · AI Magazine · doi:10.1609/aimag.v33i3.2426 · Cited by 191
Boger, J. N., Hoey, J., Poupart, P., Boutilier, C. E., Fernie, G. R., & Mihailidis, A. (2006) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2006.864480 · Cited by 180
Error bounds for approximate policy iteration
Munos, R. (2003) · Cited by 179
A Unified Analysis of Value-Function-Based Reinforcement-Learning Algorithms
Szepesvári, C., & Littman, M. L. (1999) · Neural Computation · doi:10.1162/089976699300016070 · Cited by 183
Accelerating Reinforcement Learning through Implicit Imitation
Price, B., & Boutilier, C. E. (2003) · Journal of Artificial Intelligence Research · doi:10.1613/jair.898 · Cited by 180
Ergodic Control of Diffusion Processes
Arapostathis, A., Borkar, V. S., & Ghosh, M. K. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9781139003605 · Cited by 177
Planning treatment of ischemic heart disease with partially observable Markov decision processes
Hauskrecht, M., & Fraser, H. (2000) · Artificial Intelligence in Medicine · doi:10.1016/s0933-3657(99)00042-1 · Cited by 176
Optimal Commodity Trading with a Capacitated Storage Asset
Secomandi, N. (2009) · Management Science · doi:10.1287/mnsc.1090.1049 · Cited by 173
Clinic scheduling models with overbooking for patients with heterogeneous no-show probabilities
Zeng, B., Türkcan, A., Lin, J., & Lawley, M. A. (2009) · Annals of Operations Research · doi:10.1007/s10479-009-0569-5 · Cited by 173
Robust adaptive dynamic programming for linear and nonlinear systems: An overview
Jiang, Z., & Jiang, Y. (2013) · European Journal of Control · doi:10.1016/j.ejcon.2013.05.017 · Cited by 171
Elwany, A. H., Gebraeel, N., & Maillart, L. M. (2011) · Operations Research · doi:10.1287/opre.1110.0912 · Cited by 173
Reoptimization Approaches for the Vehicle-Routing Problem with Stochastic Demands
Secomandi, N., & Margot, F. (2008) · Operations Research · doi:10.1287/opre.1080.0520 · Cited by 173
Age of Information: Whittle Index for Scheduling Stochastic Arrivals
Hsu, Y.-P. (2018) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2018.8437712 · Cited by 173
Solving very large weakly coupled Markov decision processes
Meuleau, N., Hauskrecht, M., Kim, K.-E., Peshkin, L., Kaelbling, L. P., Dean, T., & Boutilier, C. E. (1998) · Cited by 174
Optimal Control of Markov Decision Processes With Linear Temporal Logic Constraints
Ding, X. C., Smith, S. L., Belta, C. A., & Rus, D. L. (2014) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2014.2298143 · Cited by 172
Modelling Methods for Pharmacoeconomics and Health Technology Assessment: An Overview and Guide
Stahl, J. E. (2008) · PharmacoEconomics · doi:10.2165/00019053-200826020-00004 · Cited by 171
Mocanu, E., Nguyen, P. H., Kling, W. L., & Gibescu, M. (2016) · Energy and Buildings · doi:10.1016/j.enbuild.2016.01.030 · Cited by 171
Allocation of empty containers between multi-ports
Li, J.-A., Leung, S. C. H., Wu, Y., & Liu, K. (2006) · European Journal of Operational Research · doi:10.1016/j.ejor.2006.09.003 · Cited by 170
Logarithmic Online Regret Bounds for Undiscounted Reinforcement Learning
Auer, P. L., & Ortner, R. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0011 · Cited by 170
Modeling and Understanding Human Routine Behavior
Banović, N., Buzali, T., Chevalier, F., Mankoff, J. C., & Dey, A. K. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858557 · Cited by 168
Hermanns, H., Wachter, B., & Zhang, L. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-70545-1_16 · Cited by 169
Rashid, T., Farquhar, G., Peng, B., & Whiteson, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10800 · Cited by 204
Dynamics of infectious diseases
Rock, K., Brand, S., Moir, J., & Keeling, M. (2014) · Reports on progress in physics. Physical Society · doi:10.1088/0034-4885/77/2/026602 · Cited by 139
Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from ObservationsPreprint
Brown, D. S., Goo, W., Nagarajan, P., & Niekum, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.06387 · Cited by 171
Ipeirotis, P. G., & Gabrilovich, E. (2014) · International World Wide Web Conference (WWW) · doi:10.1145/2566486.2567988 · Cited by 165
The Probabilistic Model Checking Landscape
Katoen, J.-P. (2016) · ACM/IEEE Symposium on Logic in Computer Science (LICS) · doi:10.1145/2933575.2934574 · Cited by 165
Data-driven optimization of railway maintenance for track geometry
Sharma, S. S., Cui, Y. H., He, Q., Mohammadi, R., & Li, Z. (2018) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2018.02.019 · Cited by 173
Yinyu Ye, Y. (2011) · Mathematics of Operations Research · doi:10.1287/moor.1110.0516 · Cited by 167
Joint condition-based maintenance and inventory optimization for systems with multiple components
Keizer, M. C. A. O., Teunter, R. H., & Veldman, J. (2016) · European Journal of Operational Research · doi:10.1016/j.ejor.2016.07.047 · Cited by 164
Lai, G., Margot, F., & Secomandi, N. (2010) · Operations Research · doi:10.1287/opre.1090.0768 · Cited by 163
Multi-armed bandit problems with dependent arms
Pandey, S., Chakrabarti, D., & Agarwal, D. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273587 · Cited by 163
A Survey of Multi-Task Deep Reinforcement Learning
Varghese, N. V., & Mahmoud, Q. H. (2020) · Electronics · doi:10.3390/electronics9091363 · Cited by 173
Quantitative stochastic parity games
Chatterjee, K., Jurdziński, M., & Henzinger, T. A. (2004) · doi:10.5555/982792.982808 · Cited by 162
Comparative branching-time semantics for Markov chains
Christel Baier, A., Katoen, J.-P., Hermanns, H., & Wolf, V. (2005) · Information and Computation · doi:10.1016/j.ic.2005.03.001 · Cited by 164
Robust Deep Reinforcement Learning against Adversarial Perturbations on State ObservationsPreprint
Zhang, H., Chen, H., Xiao, C., Li, B., Liu, M., Boning, D. S., & Hsieh, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.08938 · Cited by 181
Call admission control in mobile cellular networks: a comprehensive survey
Ghaderi, M., & Boutaba, R. (2005) · Wireless Communications and Mobile Computing · doi:10.1002/wcm.246 · Cited by 159
PMaude: Rewrite-based Specification Language for Probabilistic Object Systems
Agha, G., Meseguer, J., & Sen, K. (2006) · Electronic Notes in Theoretical Computer Science · doi:10.1016/j.entcs.2005.10.040 · Cited by 158
Continuous control of a polymerization system with deep reinforcement learning
Ma, Y., Zhu, W., Benton, M. G., & Romagnoli, J. A. (2019) · Journal of Process Control · doi:10.1016/j.jprocont.2018.11.004 · Cited by 159
Mao, S., Leng, S., Maharjan, S., & Zhang, Y. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2959300 · Cited by 157
Deep Reinforcement Learning based Resource Allocation in Low Latency Edge Computing Networks
Yang, T., Hu, Y., Gursoy, M. C., Schmeink, A., & Mathar, R. (2018) · 2018 15th International Symposium on Wireless Communication Systems (ISWCS) · doi:10.1109/iswcs.2018.8491089 · Cited by 156
Wireless Device-to-Device Communications and Networks
Song, L., Niyato, D. T., Han, Z., & Hossain, E. (2015) · Cambridge University Press eBooks · doi:10.1017/cbo9781107478732 · Cited by 156
Intent-aware long-term prediction of pedestrian motion
Karasev, V. I., Ayvacı, A., Heisele, B., & Soatto, S. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487409 · Cited by 157
Reachability Analysis of Probabilistic Systems by Successive Refinements
D’Argenio, P. R., Jeannet, B., Jensen, H. E., & Larsen, K. G. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44804-7_3 · Cited by 155
Asheralieva, A., & Niyato, D. T. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2923702 · Cited by 155
Optimal inspection and mission abort policies for systems subject to degradation
Zhao, X. Q., Sun, J., Qiu, Q., & Chen, K. (2020) · European Journal of Operational Research · doi:10.1016/j.ejor.2020.11.015 · Cited by 155
Modeling Medical Treatment Using Markov Decision Processes
Schaefer, A. J., Bailey, M. D., Shechter, S. M., & Roberts, M. S. (2005) · International Series in Operations Research & Management Science · doi:10.1007/1-4020-8066-2_23 · Cited by 155
A Bayesian nonparametric approach to modeling motion patterns
Joseph, J., Doshi‐Velez, F., Huang, A. S., & Roy, N. (2011) · Autonomous Robots · doi:10.1007/s10514-011-9248-x · Cited by 153
The Dynamic Dispatch Waves Problem for same-day delivery
Klapp, M. A., Erera, A. L., & Toriello, A. (2018) · European Journal of Operational Research · doi:10.1016/j.ejor.2018.05.032 · Cited by 153
Old and New Methods for Lost-Sales Inventory Systems
Zipkin, P. H. (2008) · Operations Research · doi:10.1287/opre.1070.0471 · Cited by 152
Minimax PAC bounds on the sample complexity of reinforcement learning with a generative model
Azar, M. G., Munos, R., & Kappen, H. J. (2013) · Machine Learning · doi:10.1007/s10994-013-5368-1 · Cited by 149
MDPtoolbox: a multi‐platform toolbox to solve stochastic dynamic programming problems
Chadès, I., Chapron, G., Cros, M., Garçia, F., & Sabbadin, R. (2014) · Ecography · doi:10.1111/ecog.00888 · Cited by 148
Deep Reinforcement Learning Multi-UAV Trajectory Control for Target Tracking
Moon, J., Papaioannou, S., Laoudias, C., Kolios, P., & Kim, S. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3073973 · Cited by 149
A Survey of POMDP Applications
Cassandra, A. R. (2003) · Cited by 148
Complex decisions made simple: a primer on stochastic dynamic programming
Marescot, L., Chapron, G., Chadès, I., Fackler, P. L., Duchamp, C., Marboutin, É., & Giménez, O. (2013) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12082 · Cited by 147
Solving POMDPs by Searching the Space of Finite PoliciesPreprint
Meuleau, N., Kim, K.-E., Kaelbling, L. P., & Cassandra, A. R. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6720 · Cited by 146
Determining the Acceptance of Cadaveric Livers Using an Implicit Model of the Waiting List
Alagöz, O., Maillart, L. M., Schaefer, A. J., & Roberts, M. S. (2007) · Operations Research · doi:10.1287/opre.1060.0329 · Cited by 145
Offloading Time Optimization via Markov Decision Process in Mobile-Edge Computing
Yang, G., Hou, L., He, X., He, D., Chan, S., & Guizani, M. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3033285 · Cited by 145
Blythe, J. (1999) · doi:10.1609/aimag.v20i2.1455 · Cited by 145
Cognitive Dynamic Systems: Perception-action Cycle, Radar and Radio
Haykin, S. (2012) · Cited by 144
Reinforcement Learning in Autonomic Computing: A Manifesto and Case Studies
Tesauro, G. (2007) · IEEE Internet Computing · doi:10.1109/mic.2007.21 · Cited by 143
Matching EV Charging Load With Uncertain Wind: A Simulation-Based Policy Improvement Approach
Huang, Q., Jia, Q., Qiu, Z., Guan, X., & Deconinck, G. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2014.2385711 · Cited by 143
Niyato, D. T., Hossain, E., & Camorlinga, S. G. (2009) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2009.090506 · Cited by 143
Data-Driven Stochastic Models and Policies for Energy Harvesting Sensor Communications
Ku, M., Chen, Y., & Liu, K. J. R. (2015) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2015.2391651 · Cited by 143
Opportunistic Channel Access and RF Energy Harvesting in Cognitive Radio Networks
Hoang, D. T., Niyato, D. T., Wang, P., & Kim, D. I. (2014) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2014.141108 · Cited by 142
An SMDP-Based Service Model for Interdomain Resource Allocation in Mobile Cloud Networks
Liang, H., Cai, L., Huang, D., Shen, X., & Peng, D. (2012) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2012.2194748 · Cited by 134
Andriotis, C. P., & Papakonstantinou, K. G. (2021) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2021.107551 · Cited by 156
Punctuality of railway operations and timetable stability analysis
Goverde, R. M. P. (2005) · Research Repository (Delft University of Technology) · Cited by 141
An MDP-Based Approach to Online Mechanism Design
Parkes, D. C., & Singh, S. P. (2003) · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · Cited by 140
Joint Adaptation of Frequency Hopping and Transmission Rate for Anti-Jamming Wireless Systems
Hanawal, M. K., Abdel‐Rahman, M. J., & Krunz, M. (2015) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2015.2492556 · Cited by 140
Patient Choice in Kidney Allocation: The Role of the Queueing Discipline
Su, X., & Zenios, S. A. (2004) · Manufacturing & Service Operations Management · doi:10.1287/msom.1040.0056 · Cited by 139
Autonomously Generating Hints by Inferring Problem Solving Policies
Piech, C., Sahami, M., Huang, J., & Guibas, L. J. (2015) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/2724660.2724668 · Cited by 139
Proactive Caching for Mobile Video Streaming in Millimeter Wave 5G Networks
Qiao, J., He, Y., & Shen, X. (2016) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2016.2598748 · Cited by 139
Stochastic Constraint ProgrammingPreprint
Walsh, T. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0903.1152 · Cited by 139
Optimal timing for managed relocation of species faced with climate change
McDonald‐Madden, E., Runge, M. C., Possingham, H. P., & Martin, T. G. (2011) · Nature Climate Change · doi:10.1038/nclimate1170 · Cited by 142
Metadata
- Authors
- Hazeghi, K., & Puterman, M. L.
- ORCID
- Puterman, Martin L.
- Venue
- Journal of the American Statistical Association
- Publisher
- JSTOR
- Published
- 1995-03
- Volume
- 90
- Issue
- 429
- Pages
- 392
- ISSN
- 0162-1459 (ISSN-L)
- OpenAlex
- W2119567691
- Cited by
- 8,037
- References
- 12
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 86 · 2023: 145 · 2022: 233 · 2021: 616 · 2020: 674 · 2019: 595 · 2018: 480 · 2017: 459 · 2016: 450 · 2015: 507 · 2014: 451 · 2013: 420 · 2012: 392
- Type
- book review
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- Markov decision process, stochastic dynamic programming, discrete-time models, infinite horizon model, finite horizon model, Bellman equation, optimality criteria, average reward criterion, constrained models, sequential decision making under uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
