The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Markov decision processes: Discrete stochastic dynamic programming
Abstract
From the Publisher: The past decade has seen considerable theoretical and applied research on Markov decision processes, as well as the growing use of these models in ecology, economics, communications engineering, and other fields where outcomes are uncertain and sequential decision-making processes are needed. A timely response to this increased activity, Martin L. Puterman's new work provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models. It discusses all major research directions in the field, highlights many significant applications of Markov decision processes models, and explores numerous important topics that have previously been neglected or given cursory coverage in the literature. Markov Decision Processes focuses primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous-time discrete state models. The book is organized around optimality criteria, using a common framework centered on the optimality (Bellman) equation for presenting results. The results are presented in a theorem-proof format and elaborated on through both discussion and examples, including results that are not available in any other book. A two-state Markov decision process model, presented in Chapter 3, is analyzed repeatedly throughout the book and demonstrates many results an…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Rudin, W. (1973) · Cited by 13,740
Introduction to Stochastic Dynamic Programming.
Berry, D. A., & Ross, S. M. (1986) · Journal of the American Statistical Association · doi:10.2307/2289262 · Cited by 1,130
White, C. C., & White, D. J. (1989) · European Journal of Operational Research · doi:10.1016/0377-2217(89)90348-2 · Cited by 422
Dynamic programming: Models and applications
Denardo, E. V. (1982) · CERN Document Server (European Organization for Nuclear Research) · Cited by 411
Finding Optimal (s, S) Policies Is About As Simple As Evaluating a Single Policy
Zheng, Y., & Federgruen, A. (1991) · Operations Research · doi:10.1287/opre.39.4.654 · Cited by 317
Optimization over Time. Dynamic Programming and Stochastic Control. Volume 1
Thomas, L. C., & Whittle, P. (1983) · Journal of the Operational Research Society · doi:10.1057/jors.1983.56 · Cited by 162
Dynamic Programming and Its Application
Puterman, M. L. (1979) · Academic Press, Inc. eBooks · Cited by 57
Self-citationOn Homogeneous Markov Models with Continuous Time and Finite or Countable State Space
Yushkevic, A. A., & Fainberg, E. A. (1979) · Theory of Probability and Its Applications · doi:10.1137/1124014 · Cited by 34
Finite Dynamic Programming: An Approach to Finite Markov Decision Processes
White, D. J. (1978) · Andalas University Repository (Andalas University) · Cited by 24
Flagged first, then by citations
Cited by
Sorted by citations
Items 801–900 of 1,012
Stochastic Games for Verification of Probabilistic Timed Automata
Kwiatkowska, M., Norman, G., & Parker, D. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04368-0_17 · Cited by 64
Reduction and Refinement Strategies for Probabilistic Analysis
D’Argenio, P. R., Jeannet, B., Jensen, H. E., & Larsen, K. G. (2002) · Lecture notes in computer science · doi:10.1007/3-540-45605-8_5 · Cited by 65
Training Beam Sequence Design for Millimeter-Wave MIMO Systems: A POMDP Framework
Seo, J., Sung, Y., Lee, G., & Kim, D. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2496241 · Cited by 64
An Adaptive Memory Programming Framework for the Robust Capacitated Vehicle Routing Problem
Gounaris, C. E., Repoussis, P. P., Tarantilis, C. D., Wiesemann, W., & Floudas, C. A. (2014) · Transportation Science · doi:10.1287/trsc.2014.0559 · Cited by 64
Demand seasonality in retail inventory management
Ehrenthal, J. C. F., Honhon, D., & Woensel, T. V. (2014) · European Journal of Operational Research · doi:10.1016/j.ejor.2014.03.030 · Cited by 64
Markov decision process applied to the control of hospital elective admissions
Nunes, L. G. N., Carvalho, S., & Rodrigues, R. D. C. M. (2009) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2009.07.003 · Cited by 64
Reinforcement Learning in Robust Markov Decision Processes
Lim, S. H., Xu, H., & Mannor, S. (2016) · Mathematics of Operations Research · doi:10.1287/moor.2016.0779 · Cited by 64
Geometric backtracking for combined task and motion planning in robotic systems
Bidot, J., Karlsson, L., Lagriffoul, F., & Saffiotti, A. (2015) · Artificial Intelligence · doi:10.1016/j.artint.2015.03.005 · Cited by 64
Kernel-based reinforcement learning in average-cost problems
Ormoneit, D., & Glynn, P. W. (2002) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2002.803530 · Cited by 64
Modeling and Planning with Macro-Actions in Decentralized POMDPs
Amato, C., Konidaris, G., Kaelbling, L. P., & How, J. P. (2019) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11418 · Cited by 69
An application of yield management for Internet Service Providers
Nair, S. K., & Bapna, R. (2001) · Naval Research Logistics (NRL) · doi:10.1002/nav.1023 · Cited by 64
A STOCHASTIC DYNAMIC PROGRAMMING MODEL FOR THE MANAGEMENT OF THE SAIGA ANTELOPE
Milner‐Gulland, E. J. (1997) · Ecological Applications · doi:10.1890/1051-0761(1997)007[0130:asdpmf]2.0.co;2 · Cited by 64
Behavior From the Void: Unsupervised Active Pre-TrainingPreprint
Liu, H., & Abbeel, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.04551 · Cited by 86
Niyato, D. T., Dong, Q., Wang, P., & Hossain, E. (2013) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2012.2224677 · Cited by 63
Value Alignment or Misalignment – What Will Keep Systems Accountable?
Arnold, T. M., Kasenberg, D., & Scheutz, M. J. (2017) · National Conference on Artificial Intelligence · Cited by 64
Boosted Bellman Residual Minimization Handling Expert Demonstrations
Piot, B., Geist, M., & Pietquin, O. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-44851-9_35 · Cited by 63
To wait or not to wait: Optimal ordering under lead time uncertainty and forecast updating
Wang, Y., & Tomlin, B. T. (2009) · Naval Research Logistics (NRL) · doi:10.1002/nav.20381 · Cited by 63
Handoff Performance Improvements in MIMO-Enabled Communication-Based Train Control Systems
Zhu, L., Yu, F. R., Ning, B., & Tang, T. (2012) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2012.2188288 · Cited by 63
Optimal patient and personnel scheduling policies for care-at-home service facilities
Koeleman, P. M., Bhulai, S., & van Meersbergen, M. (2011) · European Journal of Operational Research · doi:10.1016/j.ejor.2011.10.046 · Cited by 63
Elhafsi, M. (2007) · European Journal of Operational Research · doi:10.1016/j.ejor.2007.12.007 · Cited by 63
Monte Carlo Planning Method Estimates Planning Horizons during Interactive Social Exchange
Hula, A., Montague, P. R., & Dayan, P. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004254 · Cited by 63
Joint call admission control algorithms: Requirements, approaches, and design considerations
Falowo, O. E., & Chan, H. A. (2007) · Computer Communications · doi:10.1016/j.comcom.2007.10.044 · Cited by 63
Hartmanns, A., & Kaminski, B. L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-53291-8_26 · Cited by 63
Primate Orbitofrontal Cortex Codes Information Relevant for Managing Explore–Exploit Tradeoffs
Costa, V. D., & Averbeck, B. B. (2020) · Journal of Neuroscience · doi:10.1523/jneurosci.2355-19.2020 · Cited by 65
A unifying framework for computational reinforcement learning theory
Littman, M. L. (2009) · Rutgers University Community Repository (Rutgers University) · doi:10.7282/t3s46s46 · Cited by 63
Regret-based Reward Elicitation for Markov Decision ProcessesPreprint
Regan, K., & Boutilier, C. E. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2619 · Cited by 63
Zheng, M., Ye, H., Wang, D., & Pan, E. (2021) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2021.107512 · Cited by 62
The RLR-Tree: A Reinforcement Learning Based R-Tree for Spatial Data
Gu, T., Feng, K., Cong, G., Long, C., Wang, Z., & Wang, S. (2023) · Proceedings of the ACM on Management of Data · doi:10.1145/3588917 · Cited by 62
Zhao, B., & Zhao, X. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3094465 · Cited by 62
Kurt, M., & Kharoufeh, J. P. (2010) · Operations Research Letters · doi:10.1016/j.orl.2010.03.001 · Cited by 62
Multi-objectivization of reinforcement learning problems by reward shaping
Brys, T., Harutyunyan, A., Vrancx, P., Taylor, M. E., Kudenko⋆, D., & Nowé, A. (2014) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889732 · Cited by 62
Road maintenance optimization through a discrete-time semi-Markov decision process
Zhang, X., & Gao, H. (2012) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2012.03.011 · Cited by 62
Projected equation methods for approximate solution of large linear systems
Bertsekas, D. P., & Yu, H. (2008) · Journal of Computational and Applied Mathematics · doi:10.1016/j.cam.2008.07.037 · Cited by 68
Robot task plan representation by Petri nets: modelling, identification, analysis and execution
Costelha, H., & Lima, P. U. (2012) · Autonomous Robots · doi:10.1007/s10514-012-9288-x · Cited by 62
Cirillo, M., Karlsson, L., & Saffiotti, A. (2010) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1869397.1869404 · Cited by 62
The Effect of Budgetary Restrictions on Breast Cancer Diagnostic Decisions
Ayvaci, M., Alagöz, O., & Burnside, E. S. (2012) · Manufacturing & Service Operations Management · doi:10.1287/msom.1110.0371 · Cited by 59
An Improved Dynamic Programming Decomposition Approach for Network Revenue Management
Zhang, D. (2010) · Manufacturing & Service Operations Management · doi:10.1287/msom.1100.0302 · Cited by 59
Kim, S. Y., Lewis, M. E., & White, C. C. (2005) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2005.853695 · Cited by 62
Learning from Demonstration for Shaping through Inverse Reinforcement Learning
Suay, H. B., Brys, T., Taylor, M. E., & Chernova, S. (2016) · VUBIR (Vrije Universiteit Brussel) · doi:10.5555/2936924.2936988 · Cited by 62
LTLf/LDLf Non-Markovian Rewards
Brafman, R. I., Giacomo, G. D., & Patrizi, F. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11572 · Cited by 62
Gómez, C., & Baker, J. W. (2018) · Structural Safety · doi:10.1016/j.strusafe.2018.10.002 · Cited by 62
Yin, B., Zhang, S., Cheng, Y., Cai, L. X., Jiang, Z., Zhou, S., & Niu, Z. (2019) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2019.8737508 · Cited by 62
(2003) · Cited by 62
High Frequency Traders: Taking Advantage of Speed
Aı̈t-Sahalia, Y., & Sağlam, M. (2013) · National Bureau of Economic Research · doi:10.3386/w19531 · Cited by 62
Learning Near Optimal Policies with Low Inherent Bellman ErrorPreprint
Zanette, A., Lazaric, A., Kochenderfer, M. J., & Brunskill, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.00153 · Cited by 62
Optimal Pricing Strategy for Operators in Cognitive Femtocell Networks
Jiang, C., Chen, Y., Liu, K. J. R., & Ren, Y. (2014) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2014.2327970 · Cited by 61
Revenue Maximization with Optimal Capacity Control in Infrastructure as a Service Cloud Markets
Toosi, A. N., Vanmechelen, K., Ramamohanarao, K., & Buyya, R. (2014) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2014.2382119 · Cited by 61
Transient Analysis of Serial Production Lines With Perishable Products: Bernoulli Reliability Model
Ju, F., Li, J., & Horst, J. A. (2016) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2016.2572119 · Cited by 61
Liang, H., Zhang, X., Zhang, J., Li, Q., Zhou, S., & Zhao, L. Y. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2937842 · Cited by 61
A Survey of Practical Formal Methods for Security
Kulik, T., Dongol, B., Larsen, P. G., Macedo, H. D., Schneider, S. A., Tran-Jørgensen, P. W. V., & Woodcock, J. (2022) · Formal Aspects of Computing · doi:10.1145/3522582 · Cited by 61
A Convex Analytic Approach to Risk-Aware Markov Decision Processes
Haskell, W. B., & Jain, R. (2015) · SIAM Journal on Control and Optimization · doi:10.1137/140969221 · Cited by 61
Optimizing a dynamic order-picking process
Bukchin, Y., Khmelnitsky, E., & Yakuel, P. (2012) · European Journal of Operational Research · doi:10.1016/j.ejor.2011.12.041 · Cited by 61
Chan, S. L., Ip, W. H., & Cho, V. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2009.06.030 · Cited by 61
A Markov decision process approach to multi-category patient scheduling in a diagnostic facility
Göçgün, Y., Bresnahan, B. W., Ghate, A., & Gunn, M. L. D. (2011) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2011.06.001 · Cited by 61
Collaborative 20 Questions for Target Localization
Tsiligkaridis, T., Sadler, B. M., & Hero, A. O. (2014) · IEEE Transactions on Information Theory · doi:10.1109/tit.2014.2304455 · Cited by 61
The PHEV Charging Scheduling and Power Supply Optimization for Charging Stations
Dong, Q., Niyato, D. T., Wang, P., & Han, Z. (2015) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2015.2399411 · Cited by 58
A tutorial on event-based optimization—a new optimization framework
Xia, L., Jia, Q., & Cao, X. (2013) · Discrete Event Dynamic Systems · doi:10.1007/s10626-013-0170-6 · Cited by 61
Variance-constrained actor-critic algorithms for discounted and average reward MDPs
Prashanth, L. A., & Ghavamzadeh, M. (2016) · Machine Learning · doi:10.1007/s10994-016-5569-5 · Cited by 73
Learning to Predict Sequences of Human Visual Fixations
Jiang, M. X., Boix, X., Roig, G., Xu, J., Gool, L. V., & Zhao, Q. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2496306 · Cited by 61
Model-Free Preference-Based Reinforcement Learning
Wirth, C., Fürnkranz, J., & Neumann, G. M. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10269 · Cited by 69
Reinforcement Learning with Non-Markovian Rewards
Gaon, M., & Brafman, R. I. (2020) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v34i04.5814 · Cited by 61
Statistical Model Checking: Past, Present, and Future
Larsen, K. G., & Legay, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-47166-2_1 · Cited by 61
A Heuristic Search Approach for a Nonstationary Stochastic Shortest Path Problem with Terminal Cost
Bander, J. L., & White, C. C. (2002) · Transportation Science · doi:10.1287/trsc.36.2.218.562 · Cited by 61
Queueing Network Controls via Deep Reinforcement Learning
Dai, J. G., & Gluzman, M. O. (2022) · Stochastic Systems · doi:10.1287/stsy.2021.0081 · Cited by 61
A numerical study of Markov decision process algorithms for multi-component replacement problems
Andersen, J. F., Andersen, A. R., Külahçı, M., & Nielsen, B. F. (2021) · European Journal of Operational Research · doi:10.1016/j.ejor.2021.07.007 · Cited by 60
Automated Verification and Strategy Synthesis for Probabilistic Systems
Kwiatkowska, M., & Parker, D. (2013) · Lecture notes in computer science · doi:10.1007/978-3-319-02444-8_2 · Cited by 60
Dependable Horizontal Scaling Based on Probabilistic Model Checking
Naskos, A., Stachtiari, E., Gounaris, A., Katsaros, P., Tsoumakos, D., Konstantinou, I., & Sioutas, S. (2015) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid.2015.91 · Cited by 60
Optimal empty vehicle repositioning and fleet-sizing for two-depot service systems
Song, D., & Earl, C. (2007) · European Journal of Operational Research · doi:10.1016/j.ejor.2006.12.034 · Cited by 60
Theoretical tools for understanding and aiding dynamic decision making
Busemeyer, J. R., & Pleskac, T. J. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2008.12.007 · Cited by 62
Client intelligence for adaptive streaming solutions
Jarnikov, D., & Özçelebi, T. (2011) · Signal Processing Image Communication · doi:10.1016/j.image.2011.03.003 · Cited by 60
Approximate dynamic programming approach for process control
Lee, J. H., & Wong, W. (2010) · Journal of Process Control · doi:10.1016/j.jprocont.2010.06.007 · Cited by 60
Bounding Performance Loss in Approximate MDP Homomorphisms
Taylor, J., Precup, D., & Panagaden, P. (2008) · Cited by 60
Model-based function approximation in reinforcement learning
Jong, N. K., & Stone, P. (2007) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/1329125.1329242 · Cited by 60
Multimedia wireless transmission with network coding
Nguyen, D. C., Nguyen, T. P., & Yang, X. (2007) · International Packet Video Workshop (PV) · doi:10.1109/packet.2007.4397057 · Cited by 60
Reinforcement learning models of the dopamine system and their behavioral implications
Daw, N. D., & Touretzky, D. S. (2003) · Cited by 62
Bhatnagar, S. (2010) · Systems & Control Letters · doi:10.1016/j.sysconle.2010.08.013 · Cited by 63
A model of risk and mental state shifts during social interaction
Hula, A., Vilares, I., Lohrenz, T. M., Dayan, P., & Montague, P. R. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005935 · Cited by 64
Conditioning in Probabilistic Programming
Olmedo, F., Gretz, F., Jansen, N., Kaminski, B. L., Katoen, J.-P., & McIver, A. K. (2018) · ACM Transactions on Programming Languages and Systems · doi:10.1145/3156018 · Cited by 60
Sanjab, A., Saad, W., & Başar, T. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3010289 · Cited by 60
Distributed Learning for Vehicle Routing Decision in Software Defined Internet of Vehicles
Lin, K., Li, C., Li, Y., Savaglio, C., & Fortino, G. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3023958 · Cited by 60
Computational Methods for Reachability Analysis of Stochastic Hybrid Systems
Koutsoukos, X., & Riley, D. (2006) · Lecture notes in computer science · doi:10.1007/11730637_29 · Cited by 60
Topological value iteration algorithm for Markov decision processes
Dai, P., & Goldsmith, J. (2007) · International Joint Conference on Artificial Intelligence · Cited by 60
Epistemic Communities under Active Inference
Albarracin, M., Demekas, D., Ramstead, M. J. D., & Heins, C. (2022) · Entropy · doi:10.3390/e24040476 · Cited by 60
LEARNING-BASED MODEL PREDICTIVE CONTROL FOR MARKOV DECISION PROCESSES
Negenborn, R. R., De Schutter, B., Wiering, M. A., & Hellendoorn, H. (2005) · IFAC Proceedings Volumes · doi:10.3182/20050703-6-cz-1902.00280 · Cited by 59
Dynamic Chinese Restaurant Game: Theory and Application to Cognitive Radio Networks
Jiang, C., Chen, Y., Yang, Y., Wang, C.-Y., & Liu, K. J. R. (2014) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2014.030314.130632 · Cited by 59
Request acceptance in same-day delivery
Klapp, M. A., Erera, A. L., & Toriello, A. (2020) · Transportation Research Part E Logistics and Transportation Review · doi:10.1016/j.tre.2020.102083 · Cited by 59
UAV Autonomous Tracking and Landing Based on Deep Reinforcement Learning Strategy
Xie, J., Peng, X., Wang, H., Niu, W., & Zheng, X. (2020) · Sensors · doi:10.3390/s20195630 · Cited by 59
An Optimal Battery Charging Algorithm in Electric Vehicle-Assisted Battery Swapping Environments
Ko, H., Pack, S., & Leung, V. C. M. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3038274 · Cited by 59
Partial order reduction for probabilistic systems
Christel Baier, A., Größer, M., & Ciesinski, F. (2004) · doi:10.1109/qest.2004.30 · Cited by 59
A modeling framework for optimizing the flow-level scheduling with time-varying channels
Ayesta, U., Erausquin, M., & Jacko, P. (2010) · Performance Evaluation · doi:10.1016/j.peva.2010.08.015 · Cited by 59
Feature Reinforcement Learning: Part I. Unstructured MDPs
Hütter, M. (2009) · Journal of Artificial General Intelligence · doi:10.2478/v10229-011-0002-8 · Cited by 59
On Energy Harvesting Gain and Diversity Analysis in Cooperative Communications
Ku, M., Li, W., Chen, Y., & Liu, K. J. R. (2015) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2015.2481215 · Cited by 59
Efficient solutions to factored MDPs with imprecise transition probabilities
Delgado, K. V., Sanner, S., & de Barros, L. N. (2011) · Artificial Intelligence · doi:10.1016/j.artint.2011.01.001 · Cited by 59
Multistage Stochastic Programming
Defourny, B., Ernst, D., & Wehenkel, L. A. (2011) · IGI Global eBooks · doi:10.4018/978-1-60960-165-2.ch006 · Cited by 61
Omidshafiei, S., Agha–mohammadi, A., Amato, C., Liu, S., How, J. P., & Vian, J. L. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917692864 · Cited by 59
Wegmann, R., Döge, V., Becker, J., & Sauer, D. U. (2017) · Journal of Energy Storage · doi:10.1016/j.est.2017.09.008 · Cited by 59
Expiration dates and order quantities for perishables
Ketzenberg, M. E., Gaukler, G. M., & Salin, V. (2017) · European Journal of Operational Research · doi:10.1016/j.ejor.2017.10.005 · Cited by 59
Timing Successive Product Introductions with Demand Diffusion and Stochastic Technology Improvement
Krankel, R. M., Duenyas, I., & Kapuściński, R. (2006) · Manufacturing & Service Operations Management · doi:10.1287/msom.1060.0102 · Cited by 59
Incremental Contingency Planning
Dearden, R., Meuleau, N., Ramakrishnan, S., Smith, D. E., & Washington, R. (2003) · International Conference on Automated Planning and Scheduling · Cited by 59
MoDeST — A Modelling and Description Language for Stochastic Timed Systems
D’Argenio, P. R., Hermanns, H., Katoen, J.-P., & Klaren, R. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44804-7_6 · Cited by 59
Metadata
- Authors
- Hazeghi, K., & Puterman, M. L.
- ORCID
- Puterman, Martin L.
- Venue
- Journal of the American Statistical Association
- Publisher
- JSTOR
- Published
- 1995-03
- Volume
- 90
- Issue
- 429
- Pages
- 392
- ISSN
- 0162-1459 (ISSN-L)
- OpenAlex
- W2119567691
- Cited by
- 8,037
- References
- 12
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 86 · 2023: 145 · 2022: 233 · 2021: 616 · 2020: 674 · 2019: 595 · 2018: 480 · 2017: 459 · 2016: 450 · 2015: 507 · 2014: 451 · 2013: 420 · 2012: 392
- Type
- book review
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- Markov decision process, stochastic dynamic programming, discrete-time models, infinite horizon model, finite horizon model, Bellman equation, optimality criteria, average reward criterion, constrained models, sequential decision making under uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
