The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Markov decision processes: Discrete stochastic dynamic programming
Abstract
From the Publisher: The past decade has seen considerable theoretical and applied research on Markov decision processes, as well as the growing use of these models in ecology, economics, communications engineering, and other fields where outcomes are uncertain and sequential decision-making processes are needed. A timely response to this increased activity, Martin L. Puterman's new work provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models. It discusses all major research directions in the field, highlights many significant applications of Markov decision processes models, and explores numerous important topics that have previously been neglected or given cursory coverage in the literature. Markov Decision Processes focuses primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous-time discrete state models. The book is organized around optimality criteria, using a common framework centered on the optimality (Bellman) equation for presenting results. The results are presented in a theorem-proof format and elaborated on through both discussion and examples, including results that are not available in any other book. A two-state Markov decision process model, presented in Chapter 3, is analyzed repeatedly throughout the book and demonstrates many results an…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Rudin, W. (1973) · Cited by 13,740
Introduction to Stochastic Dynamic Programming.
Berry, D. A., & Ross, S. M. (1986) · Journal of the American Statistical Association · doi:10.2307/2289262 · Cited by 1,130
White, C. C., & White, D. J. (1989) · European Journal of Operational Research · doi:10.1016/0377-2217(89)90348-2 · Cited by 422
Dynamic programming: Models and applications
Denardo, E. V. (1982) · CERN Document Server (European Organization for Nuclear Research) · Cited by 411
Finding Optimal (s, S) Policies Is About As Simple As Evaluating a Single Policy
Zheng, Y., & Federgruen, A. (1991) · Operations Research · doi:10.1287/opre.39.4.654 · Cited by 317
Optimization over Time. Dynamic Programming and Stochastic Control. Volume 1
Thomas, L. C., & Whittle, P. (1983) · Journal of the Operational Research Society · doi:10.1057/jors.1983.56 · Cited by 162
Dynamic Programming and Its Application
Puterman, M. L. (1979) · Academic Press, Inc. eBooks · Cited by 57
Self-citationOn Homogeneous Markov Models with Continuous Time and Finite or Countable State Space
Yushkevic, A. A., & Fainberg, E. A. (1979) · Theory of Probability and Its Applications · doi:10.1137/1124014 · Cited by 34
Finite Dynamic Programming: An Approach to Finite Markov Decision Processes
White, D. J. (1978) · Andalas University Repository (Andalas University) · Cited by 24
Flagged first, then by citations
Cited by
Sorted by citations
Items 901–1,000 of 1,012
Integrating communicative action, conversations and decision theory to coordinate agents
Barbuceanu, M., & Fox, M. S. (1997) · doi:10.1145/267658.267667 · Cited by 59
Environment-Independent Task Specifications via GLTLPreprint
Littman, M. L., Topcu, U., Fu, J., Isbell, C. L., Wen, M., & MacGlashan, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.04341 · Cited by 59
Off-Policy Deep Reinforcement Learning by Bootstrapping the Covariate Shift
Gelada, C., & Bellemare, M. G. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013647 · Cited by 64
A reinforcement learning-based algorithm for the aircraft maintenance routing problem
Ruan, J., Wang, Z., Chan, F. T. S., Patnaik, S., & Tiwari, M. K. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114399 · Cited by 60
Scalable Verification of Markov Decision Processes
Legay, A., Sedwards, S., & Traonouez, L. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-15201-1_23 · Cited by 59
Smart sampling for lightweight verification of Markov decision processes
D’Argenio, P. R., Legay, A., Sedwards, S., & Traonouez, L. (2015) · International Journal on Software Tools for Technology Transfer · doi:10.1007/s10009-015-0383-0 · Cited by 58
On the Existence of Optimal Policies for a Class of Static and Sequential Dynamic Teams
Gupta, A., Yüksel, S., Başar, T., & Langbort, C. (2015) · SIAM Journal on Control and Optimization · doi:10.1137/14096534x · Cited by 58
MDP optimal control under temporal logic constraints
Ding, X. C., Smith, S. L., Belta, C. A., & Rus, D. L. (2011) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2011.6161122 · Cited by 58
Risk-based maintenance and rehabilitation decisions for transportation infrastructure networks
Seyedshohadaie, R., Damnjanovic, I. D., & Butenko, S. I. (2010) · Transportation Research Part A Policy and Practice · doi:10.1016/j.tra.2010.01.005 · Cited by 58
Policy gradient in Lipschitz Markov Decision Processes
Pirotta, M., Restelli, M., & Bascetta, L. (2015) · Machine Learning · doi:10.1007/s10994-015-5484-1 · Cited by 61
Multiagent Learning of Coordination in Loosely Coupled Multiagent Systems
Yu, C., Zhang, M., Ren, F., & Tan, G. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2387277 · Cited by 58
Price-Directed Replenishment of Subsets: Methodology and Its Application to Inventory Routing
Adelman, D. (2003) · Manufacturing & Service Operations Management · doi:10.1287/msom.5.4.348.24884 · Cited by 58
Optimal control of reassembly with variable quality returns in a product remanufacturing system
Jin, X., Ni, J., & Koren, Y. (2011) · CIRP Annals · doi:10.1016/j.cirp.2011.03.133 · Cited by 58
Finite time bounds for sampling based fitted value iteration
Szepesvári, C., & Munos, R. (2005) · International Conference on Machine Learning · doi:10.1145/1102351.1102462 · Cited by 58
Dynamic control of a single-server system with abandonments
Down, D. G., Koole, G. M., & Lewis, M. E. (2010) · Queueing Systems · doi:10.1007/s11134-010-9201-2 · Cited by 58
PEPS2007 - Stochastic Automata Networks Software Tool
Brenner, L., Fernandes, P., Plateau, B., & Sbeity, I. (2007) · International Conference on the Quantitative Evaluation of Systems (QEST) · doi:10.1109/qest.2007.33 · Cited by 58
Online Cloud Transcoding and Distribution for Crowdsourced Live Game Video Streaming
Zheng, Y., Wu, D., Ke, Y., Yang, C., Chen, M., & Zhang, G. (2016) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2016.2556584 · Cited by 58
Operations research for resource planning and -use in radiotherapy: a literature review
Vieira, B., Hans, E. W., van Vliet-Vroegindeweij, C., van de Kamer, J. B., & van Harten, W. H. (2016) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0390-4 · Cited by 58
Wang, R., Yadav, A., Makled, E. A., Dobre, O. A., Zhao, R., & Varshney, P. K. (2019) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2019.2948992 · Cited by 58
Sharma, M. K., Zappone, A., Assaad, M., Debbah, M. A., & Vassilaras, S. (2019) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2019.2949589 · Cited by 58
Optimal Policies for a Dual-Sourcing Inventory Problem with Endogenous Stochastic Lead Times
Song, J.-S., Li, X., Zhang, H., & Zipkin, P. H. (2016) · Operations Research · doi:10.1287/opre.2016.1557 · Cited by 58
Evolving Static Representations for Task Transfer
Verbancsics, P., & Stanley, K. O. (2010) · Cited by 57
A survey of reinforcement learning in relational domains
van Otterlo, M. (2005) · CTIT technical reports series · Cited by 58
Distributed Planning in Hierarchical Factored MDPsPreprint
Guestrin, C., & Gordon, G. J. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.0571 · Cited by 58
AI Challenges in Human-Robot Cognitive TeamingPreprint
Chakraborti, T., Subbarao Kambhampati, B., Scheutz, M. J., & Zhang, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.04775 · Cited by 58
A mean field approach for optimization in discrete time
Gast, N., & Gaujal, B. (2010) · Discrete Event Dynamic Systems · doi:10.1007/s10626-010-0094-3 · Cited by 57
Ko, H., Lee, J., & Pack, S. (2018) · IEEE Access · doi:10.1109/access.2018.2818111 · Cited by 56
Managing Vertical Handovers in Millimeter Wave Heterogeneous Networks
Zang, S., Bao, W., Yeoh, P. L., Vucetic, B. S., & Li, Y. (2018) · IEEE Transactions on Communications · doi:10.1109/tcomm.2018.2877326 · Cited by 57
Serhani, A., Naja, N., & Jamali, A. (2019) · Cluster Computing · doi:10.1007/s10586-019-02937-x · Cited by 57
Salazar, A., Berzoy, A., Song, W., & Velni, J. M. (2020) · IEEE Transactions on Industry Applications · doi:10.1109/tia.2020.2980731 · Cited by 57
Adaptive optimal control of continuous-time nonlinear affine systems via hybrid iteration
Qasem, O., Gao, W., & Vamvoudakis, K. G. (2023) · Automatica · doi:10.1016/j.automatica.2023.111261 · Cited by 57
Human Modelling in Assisted Transportation
Cacciabue, P. C., Riccioli, C., Lüdtke, A., & Hjälmdahl, M. (2011) · Springer eBooks · doi:10.1007/978-88-470-1821-1 · Cited by 60
Relaxations of Approximate Linear Programs for the Real Option Management of Commodity Storage
Nadarajah, S., Margot, F., & Secomandi, N. (2015) · Management Science · doi:10.1287/mnsc.2014.2136 · Cited by 57
A Dynamic Decision Model Applied to Hurricane Landfall
Regnier, E., & Harr, P. A. (2006) · Weather and Forecasting · doi:10.1175/waf958.1 · Cited by 57
Inventory optimization in a one product recoverable manufacturing system
Ahiska, S. S., & King, R. E. (2009) · International Journal of Production Economics · doi:10.1016/j.ijpe.2009.09.002 · Cited by 57
Stochastic Safest and Shortest Path Problems
Teichteil-Königsbuch, F. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v26i1.8367 · Cited by 57
A framework for meta-level control in multi-agent systems
Raja, A., & Lesser, V. (2007) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-006-9008-z · Cited by 58
Accounting for State Uncertainty in Collision Avoidance
Chryssanthacopoulos, J. P., & Kochenderfer, M. J. (2011) · Journal of Guidance Control and Dynamics · doi:10.2514/1.53172 · Cited by 57
Johri, R., & Filipi, Z. S. (2009) · SAE International Journal of Engines · doi:10.4271/2009-24-0065 · Cited by 57
Lei, L., Kuang, Y., Cheng, N., Shen, X., Zhong, Z. D., & Lin, C. (2015) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2015.2444795 · Cited by 57
Xu, C., Liu, S., Zhang, C., Huang, Y., Lu, Z., & Yang, L. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3055511 · Cited by 57
Optimizing HTTP-Based Adaptive Streaming in Vehicular Environment Using Markov Decision Process
Bokani, A., Hassan, M., Kanhere, S. S., & Zhu, X. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2494458 · Cited by 56
Player Tracking Data in Sports
Kovalchik, S. A. (2022) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-033021-110117 · Cited by 56
Generalized Optimization Framework for Graph-based Semi-supervised Learning
Avrachenkov, K. E., Mishenin, A., Gonçalvès, P., & Sokol, M. (2012) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972825.83 · Cited by 56
Supply Chain Finance enabled early pay: unlocking trapped value in B2B logistics
Basu, P., & Nair, S. K. (2012) · International Journal of Logistics Systems and Management · doi:10.1504/ijlsm.2012.047605 · Cited by 56
System-Level Model and Stochastic Optimal Control for a PEM Fuel Cell Hybrid Vehicle
Lin, C.-C., Kim, M., Peng, H., & Grizzle, J. W. (2006) · Journal of Dynamic Systems Measurement and Control · doi:10.1115/1.2362785 · Cited by 56
An Analysis of the Control-Algorithm Re-solving Issue in Inventory and Revenue Management
Secomandi, N. (2008) · Manufacturing & Service Operations Management · doi:10.1287/msom.1070.0184 · Cited by 50
Partial Order Reduction for Probabilistic Branching Time
Christel Baier, A., D’Argenio, P. R., & Größer, M. (2006) · Electronic Notes in Theoretical Computer Science · doi:10.1016/j.entcs.2005.10.034 · Cited by 57
Optimal condition-based cleaning of solar power collectors
Ba, H. T., Cholette, M. E., Wang, R., Borghesani, P., Ma, L., & Steinberg, T. (2017) · Solar Energy · doi:10.1016/j.solener.2017.08.076 · Cited by 56
Choi, J., & Reveliotis, S. A. (2003) · IEEE Transactions on Robotics and Automation · doi:10.1109/tra.2003.810577 · Cited by 56
Model checking nondeterministic and randomly timed systems
Neuhäußer, M. R. (2009) · doi:10.3990/1.9789036529754 · Cited by 56
Is multiagent deep reinforcement learning the answer or the question? A brief surveyPreprint
Hernández-Leal, P., Kartal, B., & Taylor, M. E. (2018) · arXiv (Cornell University) · Cited by 56
Fast Task Inference with Variational Intrinsic Successor Features
Hansen, S., Dabney, W., Barreto, A., Van de Wiele, T., Warde-Farley, D., & Mnih, V. (2019) · arXiv · doi:10.48550/arxiv.1906.05030
DSAC: Distributional Soft Actor-Critic for Risk-Sensitive Reinforcement Learning
Ma, X., Chen, J., Xia, L., Yang, J. J., Zhao, Q., & Zhou, Z. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.17526 · Cited by 118
Lee, J. J., & Sutton, R. S. (2021) · Automatica · doi:10.1016/j.automatica.2020.109421 · Cited by 55
Approximate dynamic programming for two-player zero-sum Markov games
Pérolat, J., Scherrer, B., Piot, B., & Pietquin, O. (2015) · HAL · Cited by 55
Christel Baier, A., Daum, M., Dubslaff, C., Klein, J., & Klüppelholz, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-06200-6_24 · Cited by 55
Admission control for a multi-server queue with abandonment
Koçağa, Y. L., & Ward, A. R. (2010) · Queueing Systems · doi:10.1007/s11134-010-9176-z · Cited by 55
Stochastic Games with Unbounded Payoffs: Applications to Robust Control in Economics
Jaśkiewicz, A., & Nowak, A. S. (2011) · Dynamic Games and Applications · doi:10.1007/s13235-011-0013-8 · Cited by 55
The impact of process deterioration on production and maintenance policies
Kazaz, B., & SLOAN, T. W. (2012) · European Journal of Operational Research · doi:10.1016/j.ejor.2012.11.052 · Cited by 55
Capacity Management for Contract Manufacturing
Gupta, D., & Wang, L. (2007) · Operations Research · doi:10.1287/opre.1060.0359 · Cited by 55
Power Allocation for Outage Minimization in State Estimation Over Fading Channels
Leong, A. S., Dey, S., Nair, G. N., & Sharma, P. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2135350 · Cited by 56
Prediction-Based Throughput Optimization for Dynamic Spectrum Access
Yin, S., Chen, D., Zhang, Q., & Li, S. (2010) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2010.2101090 · Cited by 55
DORA: Dynamic Optimal Random Access for Vehicle-to-Roadside Communications
Cheung, M. H., Hou, F., Wong, V. W. S., & Huang, J. (2012) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2012.120513 · Cited by 55
Reductions of Approximate Linear Programs for Network Revenue Management
Vossen, T. W. M., & Zhang, D. (2015) · Operations Research · doi:10.1287/opre.2015.1442 · Cited by 55
Crowdsourcing High Quality Labels with a Tight Budget
Li, Q., Ma, F., Gao, J., Su, L., & Quinn, C. J. (2016) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/2835776.2835797 · Cited by 54
The optimal assignment of spontaneous volunteers
Mayorga, M. E., Lodree, E. J., & Wolczynski, J. (2017) · Journal of the Operational Research Society · doi:10.1057/s41274-017-0219-2 · Cited by 55
Optimal scheduling of multiple sensors over shared channels with packet transmission constraint
Wu, S., Ren, X., Dey, S., & Shi, L. (2018) · Automatica · doi:10.1016/j.automatica.2018.06.019 · Cited by 55
When to preempt? Age of information minimization under link capacity constraint
Wang, B., Feng, S., & Yang, J. (2019) · Journal of Communications and Networks · doi:10.1109/jcn.2019.000036 · Cited by 55
Metrics for Finite Markov Decision ProcessesPreprint
Ferns, N., Panangaden, P., & Precup, D. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1207.4114 · Cited by 55
ROBEL: Robotics Benchmarks for Learning with Low-Cost RobotsPreprint
Ahn, M., Zhu, H., Hartikainen, K., Ponte, H., Gupta, A., Levine, S., & Kumar, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11639 · Cited by 55
Multi-Task Reinforcement Learning with Context-based RepresentationsPreprint
Sodhani, S., Zhang, A., & Pineau, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06177 · Cited by 58
Reinforcement learning-based cooperative sensing in cognitive radio ad hoc networks
Lo, B. F., & Akyildiz, I. F. (2010) · IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) · doi:10.1109/pimrc.2010.5671686 · Cited by 54
Vercraene, S., Gayon, J.-P., & Flapper, S. D. P. (2013) · International Journal of Production Economics · doi:10.1016/j.ijpe.2013.11.001 · Cited by 54
Bayesian Nonparametric Reward Learning From Demonstration
Michini, B., Walsh, T. J., Agha–mohammadi, A., & How, J. P. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2405593 · Cited by 54
Johri, R., & Filipi, Z. S. (2014) · Proceedings of the Institution of Mechanical Engineers Part D Journal of Automobile Engineering · doi:10.1177/0954407014522444 · Cited by 54
An energy-efficient transmission strategy for wireless sensor networks
Ca Van, P. H. A. N., Park, Y.-S., Choi, H. H., Cho, J., & Kim, J. G. (2010) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2010.5505976 · Cited by 54
Managing Inventory in Global Supply Chains Facing Port-of-Entry Disruption Risks
Lewis, B. M., Erera, A. L., Nowak, M., & White, C. C. (2012) · Transportation Science · doi:10.1287/trsc.1120.0406 · Cited by 54
Maximizing the Probability of Arriving on Time: A Practical Q-Learning Method
Cao, Z., Guo, H., Zhang, J., Oliehoek, F. A., & Fastenrath, U. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11170 · Cited by 54
Establishing dynamic expiration dates for perishables: An application of rfid and sensor technology
Gaukler, G. M., Ketzenberg, M. E., & Salin, V. (2017) · International Journal of Production Economics · doi:10.1016/j.ijpe.2017.07.019 · Cited by 54
Ferreira, G. O., Arruda, E. F., & Marujo, L. G. (2018) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2018.05.010 · Cited by 54
Visual Forecasting by Imitating Dynamics in Natural Sequences
Zeng, K.-H., Shen, W. B., Huang, D.-A., Sun, M., & Niebles, J. C. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.326 · Cited by 55
Optimal Policies for Status Update Generation in an IoT Device With Heterogeneous Traffic
Stamatakis, G. J., Παππάς, Ν., & Traganitis, A. P. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2976690 · Cited by 54
Multi-model Markov decision processes
Steimle, L. N., Kaufman, D. L., & Denton, B. T. (2021) · IISE Transactions · doi:10.1080/24725854.2021.1895454 · Cited by 54
SLOAN, T. W. (2004) · Journal of the Operational Research Society · doi:10.1057/palgrave.jors.2601725 · Cited by 54
Niño‐Mora, J. (2006) · Mathematics of Operations Research · doi:10.1287/moor.1050.0165 · Cited by 54
Improving Smartphone Users' Affect and Wellbeing with Personalized Positive Psychology Interventions
Jeong, S., & Breazeal, C. (2016) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/2974804.2974831 · Cited by 54
Crown, W. H., Büyükkaramikli, N., Sır, M. Y., Thokala, P., Morton, A., Marshall, D. A., Tosh, J. C., IJzerman, M. J., Padula, W. V., & Pasupathy, K. S. (2018) · Value in Health · doi:10.1016/j.jval.2018.05.003 · Cited by 53
Lan, G. (2022) · Mathematical Programming · doi:10.1007/s10107-022-01816-5 · Cited by 53
Handover-Enabled Dynamic Computation Offloading for Vehicular Edge Computing Networks
Maleki, H., Başaran, M., & Durak-Ata, L. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3247889 · Cited by 53
Approximate dynamic programming for stochastic reachability
Kariotoglou, N., Summers, S., Summers, T. H., Kamgarpour, M., & Lygeros, J. (2013) · European Control Conference (ECC) · doi:10.23919/ecc.2013.6669603 · Cited by 53
Optimal power allocation for a renewable energy source
Sinha, A. (2012) · National Conference on Communications · doi:10.1109/ncc.2012.6176889 · Cited by 53
Game theoretic anti-jamming dynamic frequency hopping and rate adaptation in wireless systems
Hanawal, M. K., Abdel‐Rahman, M. J., & Krunz, M. (2014) · International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt) · doi:10.1109/wiopt.2014.6850306 · Cited by 53
Q-Learning Based Energy Management Policies for a Single Sensor Node with Finite Buffer
Prabuchandran, K. J., Meena, S. K., & Bhatnagar, S. (2012) · IEEE Wireless Communications Letters · doi:10.1109/wcl.2012.112012.120754 · Cited by 53
Optimization of the Mobile Router and Traffic Sources in Vehicular Delay-Tolerant Network
Niyato, D. T., & Wang, P. (2009) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2009.2025379 · Cited by 53
Optimal Policies for Distributed Data Aggregation in Wireless Sensor Networks
Ye, Z., Abouzeid, A. A., & Ai, J. (2007) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.2007.196 · Cited by 53
Mohsin, M., Sardar, M. U., Hasan, O., & Anwar, Z. (2017) · IEEE Access · doi:10.1109/access.2017.2696031 · Cited by 53
Distributionally Robust Counterpart in Markov Decision Processes
Yu, P., & Xu, H. (2015) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2015.2495174 · Cited by 53
Tutorial on risk neutral, distributionally robust and risk averse multistage stochastic programming
Shapiro, A. (2020) · European Journal of Operational Research · doi:10.1016/j.ejor.2020.03.065 · Cited by 53
Cognitive prostheses for goal achievement
Lieder, F., Chen, O. X., Krueger, P. M., & Griffiths, T. L. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0672-9 · Cited by 32
Metadata
- Authors
- Hazeghi, K., & Puterman, M. L.
- ORCID
- Puterman, Martin L.
- Venue
- Journal of the American Statistical Association
- Publisher
- JSTOR
- Published
- 1995-03
- Volume
- 90
- Issue
- 429
- Pages
- 392
- ISSN
- 0162-1459 (ISSN-L)
- OpenAlex
- W2119567691
- Cited by
- 8,037
- References
- 12
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 86 · 2023: 145 · 2022: 233 · 2021: 616 · 2020: 674 · 2019: 595 · 2018: 480 · 2017: 459 · 2016: 450 · 2015: 507 · 2014: 451 · 2013: 420 · 2012: 392
- Type
- book review
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- Markov decision process, stochastic dynamic programming, discrete-time models, infinite horizon model, finite horizon model, Bellman equation, optimality criteria, average reward criterion, constrained models, sequential decision making under uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
