The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Markov decision processes: Discrete stochastic dynamic programming
Abstract
From the Publisher: The past decade has seen considerable theoretical and applied research on Markov decision processes, as well as the growing use of these models in ecology, economics, communications engineering, and other fields where outcomes are uncertain and sequential decision-making processes are needed. A timely response to this increased activity, Martin L. Puterman's new work provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models. It discusses all major research directions in the field, highlights many significant applications of Markov decision processes models, and explores numerous important topics that have previously been neglected or given cursory coverage in the literature. Markov Decision Processes focuses primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous-time discrete state models. The book is organized around optimality criteria, using a common framework centered on the optimality (Bellman) equation for presenting results. The results are presented in a theorem-proof format and elaborated on through both discussion and examples, including results that are not available in any other book. A two-state Markov decision process model, presented in Chapter 3, is analyzed repeatedly throughout the book and demonstrates many results an…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Rudin, W. (1973) · Cited by 13,740
Introduction to Stochastic Dynamic Programming.
Berry, D. A., & Ross, S. M. (1986) · Journal of the American Statistical Association · doi:10.2307/2289262 · Cited by 1,130
White, C. C., & White, D. J. (1989) · European Journal of Operational Research · doi:10.1016/0377-2217(89)90348-2 · Cited by 422
Dynamic programming: Models and applications
Denardo, E. V. (1982) · CERN Document Server (European Organization for Nuclear Research) · Cited by 411
Finding Optimal (s, S) Policies Is About As Simple As Evaluating a Single Policy
Zheng, Y., & Federgruen, A. (1991) · Operations Research · doi:10.1287/opre.39.4.654 · Cited by 317
Optimization over Time. Dynamic Programming and Stochastic Control. Volume 1
Thomas, L. C., & Whittle, P. (1983) · Journal of the Operational Research Society · doi:10.1057/jors.1983.56 · Cited by 162
Dynamic Programming and Its Application
Puterman, M. L. (1979) · Academic Press, Inc. eBooks · Cited by 57
Self-citationOn Homogeneous Markov Models with Continuous Time and Finite or Countable State Space
Yushkevic, A. A., & Fainberg, E. A. (1979) · Theory of Probability and Its Applications · doi:10.1137/1124014 · Cited by 34
Finite Dynamic Programming: An Approach to Finite Markov Decision Processes
White, D. J. (1978) · Andalas University Repository (Andalas University) · Cited by 24
Flagged first, then by citations
Cited by
Sorted by citations
Items 401–500 of 1,012
Optimal Sampling and Scheduling for Timely Status Updates in Multi-Source Networks
Bedewy, A. M. L., Sun, Y. J., Kompella, S., & Shroff, N. B. (2021) · IEEE Transactions on Information Theory · doi:10.1109/tit.2021.3060387 · Cited by 115
User-Centric View of Jamming Games in Cognitive Radio Networks
Xiao, L., Liu, J., Li, Q., Mandayam, N. B., & Poor, H. V. (2015) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2015.2467593 · Cited by 113
Wasserstein Distributionally Robust Stochastic Control: A Data-Driven Approach
Yang, I. (2020) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2020.3030884 · Cited by 110
Dynamic Energy Dispatch Based on Deep Reinforcement Learning in IoT-Driven Smart Isolated Microgrids
Lei, L., Tan, Y., Dahlenburg, G., Xiang, W., & Zheng, K. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3042007 · Cited by 119
Waiting Strategies for Anticipating Service Requests from Known Customer Locations
Thomas, B. W. (2007) · Transportation Science · doi:10.1287/trsc.1060.0183 · Cited by 114
The One-Dimensional Dynamic Dispatch Waves Problem
Klapp, M. A., Erera, A. L., & Toriello, A. (2016) · Transportation Science · doi:10.1287/trsc.2016.0682 · Cited by 113
JANI: Quantitative Model and Tool Interaction
Budde, C. E., Dehnert, C., Hahn, E. M., Hartmanns, A., Junges, S., & Turrini, A. (2017) · Lecture notes in computer science · doi:10.1007/978-3-662-54580-5_9 · Cited by 113
Noyé, S., Mulero, R., Carnieletto, L., Carli, M. D., & Castelruiz, A. (2021) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2021.111685 · Cited by 118
Parietal Cortex and Insula Relate to Evidence Seeking Relevant to Reward-Related Decisions
Nicholas Furl, A., & Averbeck, B. B. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.4236-11.2011 · Cited by 113
PROST: Probabilistic Planning Based on UCT
Keller, T., & Eyerich, P. (2012) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v22i1.13518 · Cited by 113
The Effect of New Links on Google PageRank
Litvak, N., & Avrachenkov, K. E. (2004) · HAL · Cited by 113
Silver, T., Allen, K., Tenenbaum, J., & Kaelbling, L. (2018) · arXiv · doi:10.48550/arxiv.1812.06298 · Cited by 113
A survey of inverse reinforcement learning techniques
Shao, Z., & Joo, E. M. (2012) · International Journal of Intelligent Computing and Cybernetics · doi:10.1108/17563781211255862 · Cited by 115
Relay Scheduling for Cooperative Communications in Sensor Networks with Energy Harvesting
Li, H., Jaggi, N., & Sikdar, B. (2011) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2011.070711.100778 · Cited by 112
Adaptive Dynamic Bipartite Graph Matching: A Reinforcement Learning Approach
Wang, Y., Tong, Y., Long, C., Xu, P., Xu, K., & Lv, W. (2019) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2019.00133 · Cited by 118
Resource Allocation for Edge Computing in IoT Networks via Reinforcement Learning
Liu, X., Qin, Z., & Gao, Y. (2019) · International Conference on Communications · doi:10.1109/icc.2019.8761385 · Cited by 112
The Asymptotic Convergence-Rate of Q-learning
Szepesvári, C. (1997) · Cited by 111
Use of Proximal Policy Optimization for the Joint Replenishment Problem
Vanvuchelen, N., Gijsbrechts, J., & Boute, R. N. (2020) · Computers in Industry · doi:10.1016/j.compind.2020.103239 · Cited by 119
Randomized Shortest-Path Problems: Two Related Models
Saerens, M., Achbany, Y., Fouss, F., & Yen, L. (2009) · Neural Computation · doi:10.1162/neco.2009.11-07-643 · Cited by 111
Resource allocation in wireless networks with RF energy harvesting and transfer
Lu, X., Wang, P., Niyato, D. T., & Han, Z. (2015) · IEEE Network · doi:10.1109/mnet.2015.7340427 · Cited by 110
Stochastic Content-Centric Multicast Scheduling for Cache-Enabled Heterogeneous Cellular Networks
Zhou, B., Cui, Y., & Tao, M. (2016) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2016.2582689 · Cited by 110
Model-Checking Markov Chains in the Presence of Uncertainties
Sen, K., Viswanathan, M., & Agha, G. (2006) · Lecture notes in computer science · doi:10.1007/11691372_26 · Cited by 110
Inventory models with minimal service level constraints
Chen, F. Y., & Krass, D. (2001) · European Journal of Operational Research · doi:10.1016/s0377-2217(00)00243-5 · Cited by 110
Optimal Policies for a Capacitated Two-Echelon Inventory System
Parker, R. P., & Kapuściński, R. (2004) · Operations Research · doi:10.1287/opre.1040.0131 · Cited by 110
Mean-field models in swarm robotics: a survey
Elamvazhuthi, K., & Berman, S. (2019) · Bioinspiration & Biomimetics · doi:10.1088/1748-3190/ab49a4 · Cited by 110
Optimal Channel Access Management with QoS Support for Cognitive Vehicular Networks
Niyato, D. T., Hossain, E., & Wang, P. (2010) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2010.191 · Cited by 109
Optimal Feedback Controlled Assembly of Perfect Crystals
Tang, X., Rupp, B. B., Yang, Y., Edwards, T. D., Grover, M. A., & Bevan, M. A. (2016) · ACS Nano · doi:10.1021/acsnano.6b02400 · Cited by 109
Crown, W. H., Büyükkaramikli, N., Thokala, P., Morton, A., Sır, M. Y., Marshall, D. A., Tosh, J. C., Padula, W. V., IJzerman, M. J., Wong, P. K., et al. (2017) · Value in Health · doi:10.1016/j.jval.2017.01.013 · Cited by 109
Data-driven predictive maintenance scheduling policies for railways
Gerum, P. C. L., Altay, A., & Baykal‐Gürsoy, M. (2019) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2019.07.020 · Cited by 109
Optimal Dynamic Spectrum Access via Periodic Channel Sensing
Zhao, Q., Geirhofer, S., Tong, L., & Sadler, B. M. (2007) · IEEE Wireless Communications and Networking Conference · doi:10.1109/wcnc.2007.12 · Cited by 109
DeepMDP: Learning Continuous Latent Space Models for Representation LearningPreprint
Gelada, C., Kumar, S., Buckman, J., Nachum, O., & Bellemare, M. G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02736 · Cited by 118
Integration of control theory and scheduling methods for supply chain management
Subramanian, K., Rawlings, J. B., Maravelias, C. T., Flores‐Cerrillo, J., & Megan, L. (2012) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2012.06.012 · Cited by 108
Yuan, J., Yang, L., & Chen, Q. (2018) · International Journal of Hydrogen Energy · doi:10.1016/j.ijhydene.2018.03.033 · Cited by 108
Neural Approaches to Conversational AI
Gao, J., Galley, M., & Li, L. (2019) · now publishers, Inc. eBooks · doi:10.1561/9781680835533 · Cited by 125
Robot Scheduling for Mobile‐Rack Warehouses: Human–Robot Coordinated Order Picking Systems
Wang, Z., Sheu, J., Teo, C., & Xue, G. (2021) · Production and Operations Management · doi:10.1111/poms.13406 · Cited by 108
Whittle's index policy for a multi-class queueing system with convex holding costs
O'Keeffe, M., Ansell, P. S., Glazebrook, K. D., & Niño‐Mora, J. (2003) · Mathematical Methods of Operations Research · doi:10.1007/s001860200257 · Cited by 108
Providing radiology health care services to stochastic demand of different customer classes
Kolisch, R., & Sickinger, S. (2007) · OR Spectrum · doi:10.1007/s00291-007-0116-1 · Cited by 108
A Call-Routing Problem with Service-Level Constraints
Gans, N., & Zhou, Y. (2003) · Operations Research · doi:10.1287/opre.51.2.255.12787 · Cited by 108
Policy Gradients with Variance Related Risk CriteriaPreprint
Castro, D. D., Tamar, A., & Mannor, S. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6404 · Cited by 108
Action robust reinforcement learning and applications in continuous control
Tessler, C., Efroni, Y., & Mannor, S. (2019) · arXiv · doi:10.48550/arxiv.1901.09184 · Cited by 108
The use of an intelligent prompting system for people with dementia
Mihailidis, A., Boger, J. N., Canido, M., & Hoey, J. (2007) · interactions · doi:10.1145/1273961.1273982 · Cited by 107
Jointly optimized bit-rate/delay control policy for wireless packet networks with fading channels
Razavilar, J., Liu, K. J. R., & Marcus, S. I. (2002) · IEEE Transactions on Communications · doi:10.1109/26.990910 · Cited by 107
Fast Reinforcement Learning for Energy-Efficient Wireless Communication
Mastronarde, N., & van der Schaar, M. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2165211 · Cited by 107
Pareto Curves for Probabilistic Model Checking
Forejt, V., Kwiatkowska, M., & Parker, D. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33386-6_25 · Cited by 106
Optimization and Control of Cyber-Physical Vehicle Systems
Bradley, J. M., & Atkins, E. M. (2015) · Sensors · doi:10.3390/s150923020 · Cited by 107
Rapid specification and automated generation of prompting systems to assist people with dementia
Hoey, J., Plötz, T., Jackson, D., Monk, A. F., Pham, C., & Olivier, P. L. (2010) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2010.11.007 · Cited by 106
Operational planning and optimal sizing of microgrid considering multi-scale wind uncertainty
Shin, J., Lee, J. H., & Realff, M. J. (2017) · Applied Energy · doi:10.1016/j.apenergy.2017.03.081 · Cited by 106
Model Checking Probabilistic Systems
Christel Baier, A., de Alfaro, L., Forejt, V., & Kwiatkowska, M. (2018) · Springer eBooks · doi:10.1007/978-3-319-10575-8_28 · Cited by 107
A survey of decision making and optimization under uncertainty
Keith, A. J., & Ahner, D. K. (2019) · Annals of Operations Research · doi:10.1007/s10479-019-03431-8 · Cited by 114
Likmeta, A., Metelli, A. M., Tirinzoni, A., Giol, R., Restelli, M., & Romano, D. (2020) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2020.103568 · Cited by 106
Approximating Matches Made in Heaven
Chen, N., Immorlica, N., Karlin, A. R., Mahdian, M., & Atri Rudra, B. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02927-1_23 · Cited by 106
Intertemporal Pricing and Consumer Stockpiling
Su, X. (2010) · Operations Research · doi:10.1287/opre.1090.0797 · Cited by 102
Integrating Sample-Based Planning and Model-Based Reinforcement Learning
Walsh, T., Goschin, S., & Littman, M. L. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7689 · Cited by 105
Theory of Choice in Bandit, Information Sampling and Foraging Tasks
Averbeck, B. B. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004164 · Cited by 110
Neural Basis of Strategic Decision Making
Lee, D., & Seo, H. (2015) · Trends in Neurosciences · doi:10.1016/j.tins.2015.11.002 · Cited by 114
Qin, J., Li, M., Wang, J., Shi, L., Kang, Y., & Zheng, W. X. (2020) · Automatica · doi:10.1016/j.automatica.2020.109090 · Cited by 105
Transmit Power Control Policies for Energy Harvesting Sensors With Retransmissions
Aprem, A., Murthy, C. R., & Mehta, N. B. (2013) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2013.2258656 · Cited by 104
Optimal sensor scheduling for multiple linear dynamical systems
Han, D., Wu, J., Zhang, H., & Shi, L. (2016) · Automatica · doi:10.1016/j.automatica.2016.09.015 · Cited by 104
Hashemi, M., Koksal, C. E., & Shroff, N. B. (2017) · IEEE Transactions on Communications · doi:10.1109/tcomm.2017.2771308 · Cited by 104
Dynamic Lookahead Policies for Stochastic-Dynamic Inventory Routing in Bike Sharing Systems
Brinkmann, J., Ulmer, M. W., & Mattfeld, D. C. (2018) · Computers & Operations Research · doi:10.1016/j.cor.2018.06.004 · Cited by 104
Sequeira, P., & Gervasio, M. T. (2020) · Artificial Intelligence · doi:10.1016/j.artint.2020.103367 · Cited by 116
Large-Scale Wireless-Powered Networks With Backscatter Communications—A Comprehensive Survey
Rezaei, F., Tellambura, C., & Herath, S. P. (2020) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2020.3012466 · Cited by 104
Adaptation of Frequency Hopping Interval for Radar Anti-Jamming Based on Reinforcement Learning
Ailiya, Yi, W., & Varshney, P. K. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3197425 · Cited by 109
A review of research on reinforcement learning algorithms for multi-agents
Hu, K., Li, M., Song, Z., Xu, K., Xia, Q., Sun, N., Zhou, P., & Xia, M. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128068 · Cited by 117
Missile defense and interceptor allocation by neuro-dynamic programming
Bertsekas, D. P., Homer, M. L., Logan, D. A., Patek, S. D., & SANDELL, N. R. (2000) · IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans · doi:10.1109/3468.823480 · Cited by 106
Theoretical results on reinforcement learning with temporally abstract options
Precup, D., Sutton, R. S., & Singh, S. P. (1998) · Lecture notes in computer science · doi:10.1007/bfb0026709 · Cited by 103
Thomas, B. W., & White, C. C. (2004) · Transportation Science · doi:10.1287/trsc.1030.0071 · Cited by 103
Faster Heuristic Search Algorithms for Planning with Uncertainty and Full Feedback
Bonet, B., & Geffner, H. (2003) · Cited by 103
Adaptive Opportunistic Routing for Wireless Ad Hoc Networks
Bhorkar, A. A., Naghshvar, M., Javidi, T., & Rao, B. D. (2012) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2011.2159844 · Cited by 102
Expressing Arbitrary Reward Functions as Potential-Based Advice
Harutyunyan, A., Devlin, S., Vrancx, P., & Nowé, A. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9628 · Cited by 102
Online Markov Decision Processes Under Bandit Feedback
Neu, G., György, A., Szepesvári, C., & Antos, A. (2014) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2013.2292137 · Cited by 101
Managing Perishables with Time and Temperature History
Ketzenberg, M. E., Bloemhof‐Ruwaard, J. M., & Gaukler, G. M. (2014) · Production and Operations Management · doi:10.1111/poms.12209 · Cited by 101
Simultaneously Learning and Advising in Multiagent Reinforcement Learning
da Silva, F. L., Glatt, R., & Costa, A. H. R. (2017) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/3091125.3091280 · Cited by 101
Condition-based maintenance for systems with economic dependence and load sharing
Keizer, M. C. A. O., Teunter, R. H., Veldman, J., & Babaï, M. Z. (2017) · International Journal of Production Economics · doi:10.1016/j.ijpe.2017.10.030 · Cited by 101
Global Optimality Guarantees for Policy Gradient Methods
Bhandari, J., & Russo, D. (2024) · Operations Research · doi:10.1287/opre.2021.0014 · Cited by 142
Morales, E. F., Murrieta-Cid, R., Becerra, I., & Esquivel-Basaldua, M. A. (2021) · Intelligent Service Robotics · doi:10.1007/s11370-021-00398-z · Cited by 101
Using Probabilistic Models for Data Management in Acquisitional Environments
Deshpande, A., Guestrin, C., & Madden, S. R. (2005) · Conference on Innovative Data Systems Research · Cited by 101
Deep Reinforcement Learning and the Deadly TriadPreprint
van Hasselt, H. P., Doron, Y., Strub, F., Hessel, M., Sonnerat, N., & Modayil, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.02648 · Cited by 264
Optimal Policies for the Management of an Electric Vehicle Battery Swap Station
Widrick, R. S., Nurre, S. G., & Robbins, M. J. (2016) · Transportation Science · doi:10.1287/trsc.2016.0676 · Cited by 100
Deep RL-based Trajectory Planning for AoI Minimization in UAV-assisted IoT
Zhou, C., He, H., Yang, P., Lyu, F., Wu, W. R., Cheng, N., & Shen, X. (2019) · International Conference on Wireless Communications and Signal Processing (WCSP) · doi:10.1109/wcsp.2019.8928091 · Cited by 100
Global supply chain management: A reinforcement learning approach
Pontrandolfo, P., Gosavi, A., Okogbaa, O. G., & Das, T. K. (2002) · International Journal of Production Research · doi:10.1080/00207540110118640 · Cited by 100
Bisimulation Metrics for Continuous Markov Decision Processes
Ferns, N., Panangaden, P., & Precup, D. (2011) · SIAM Journal on Computing · doi:10.1137/10080484x · Cited by 100
Dorazio, R. M., & Johnson, F. A. (2003) · Ecological Applications · doi:10.1890/1051-0761(2003)013[0556:biadta]2.0.co;2 · Cited by 100
Dynamic Revenue Management in Airline Alliances
Wright, C. P., Groenevelt, H., & Shumsky, R. A. (2008) · Transportation Science · doi:10.1287/trsc.1090.0300 · Cited by 100
Max-norm Projections for Factored MDPs
Guestrin, C., Koller, D., & Parr, R. (2001) · Cited by 100
Synthesis of Human-in-the-Loop Control Protocols for Autonomous Systems
Feng, L., Wiltsche, C., Humphrey, L. R., & Topcu, U. (2016) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2016.2530623 · Cited by 100
Optimistic Knowledge Gradient Policy for Optimal Budget Allocation in Crowdsourcing
Chen, X., Lin, Q., & Zhou, D. (2013) · Cited by 103
Planning for robotic exploration based on forward simulation
Lauri, M., & Ritala, R. K. (2016) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2016.06.008 · Cited by 98
Software Engineering Meets Control Theory
Filieri, A., Maggio, M., Angelopoulos, K., D’Ippolito, N., Gerostathopoulos, I., Hempel, A. B., Hoffmann, H., Jamshidi, P., Kalyvianaki, E., Klein, C., et al. (2015) · International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS) · doi:10.1109/seams.2015.12 · Cited by 98
Real-Time Ambulance Dispatching and Relocation
Nasrollahzadeh, A. A., Khademi, A., & Mayorga, M. E. (2018) · Manufacturing & Service Operations Management · doi:10.1287/msom.2017.0649 · Cited by 98
bCharge: Data-Driven Real-Time Charging Scheduling for Large-Scale Electric Bus Fleets
Wang, G., Xie, X., Zhang, F., Liu, Y., & Zhang, D. (2018) · IEEE Real-Time Systems Symposium (RTSS) · doi:10.1109/rtss.2018.00015 · Cited by 98
Baccara, M., Lee, S. M., & Yariv, L. (2020) · Theoretical Economics · doi:10.3982/te3740 · Cited by 98
Morato, P. G., Papakonstantinou, K. G., Andriotis, C. P., Nielsen, J. S., & Rigo, P. (2021) · Structural Safety · doi:10.1016/j.strusafe.2021.102140 · Cited by 98
On an Optimization Problem in Sensor Selection*
Debouk, R., Lafortune, S., & Teneketzis, D. (2002) · Discrete Event Dynamic Systems · doi:10.1023/a:1019770124060 · Cited by 98
Adaptive resource management and the value of information
Williams, B. K., Eaton, M. J., & Breininger, D. R. (2011) · Ecological Modelling · doi:10.1016/j.ecolmodel.2011.07.003 · Cited by 98
Model-based average reward reinforcement learning
Tadepalli, P. V., & Ok, D. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00002-2 · Cited by 100
An adaptive distance-based location update algorithm for next-generation PCS networks
Wong, V. W. S., & Leung, V. C. M. (2001) · IEEE Journal on Selected Areas in Communications · doi:10.1109/49.957309 · Cited by 98
Modeling sustainable product lifecycle decision support systems
Hu, G., & Bidanda, B. (2009) · International Journal of Production Economics · doi:10.1016/j.ijpe.2009.06.011 · Cited by 98
Karlsruhe Brainstormers - A Reinforcement Learning approach to robotic soccer
Riedmiller, M., Merke, A., Meier, D., Hoffmann, A., Sinner, A., Thate, O., & Ehrmann, R. (2001) · Lecture notes in computer science · doi:10.1007/3-540-45324-5_40 · Cited by 99
Generative Adversarial Imitation from ObservationPreprint
Torabi, F., Warnell, G., & Stone, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.06158 · Cited by 125
Metadata
- Authors
- Hazeghi, K., & Puterman, M. L.
- ORCID
- Puterman, Martin L.
- Venue
- Journal of the American Statistical Association
- Publisher
- JSTOR
- Published
- 1995-03
- Volume
- 90
- Issue
- 429
- Pages
- 392
- ISSN
- 0162-1459 (ISSN-L)
- OpenAlex
- W2119567691
- Cited by
- 8,037
- References
- 12
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 86 · 2023: 145 · 2022: 233 · 2021: 616 · 2020: 674 · 2019: 595 · 2018: 480 · 2017: 459 · 2016: 450 · 2015: 507 · 2014: 451 · 2013: 420 · 2012: 392
- Type
- book review
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- Markov decision process, stochastic dynamic programming, discrete-time models, infinite horizon model, finite horizon model, Bellman equation, optimality criteria, average reward criterion, constrained models, sequential decision making under uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
