The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Markov decision processes: Discrete stochastic dynamic programming
Abstract
From the Publisher: The past decade has seen considerable theoretical and applied research on Markov decision processes, as well as the growing use of these models in ecology, economics, communications engineering, and other fields where outcomes are uncertain and sequential decision-making processes are needed. A timely response to this increased activity, Martin L. Puterman's new work provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models. It discusses all major research directions in the field, highlights many significant applications of Markov decision processes models, and explores numerous important topics that have previously been neglected or given cursory coverage in the literature. Markov Decision Processes focuses primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous-time discrete state models. The book is organized around optimality criteria, using a common framework centered on the optimality (Bellman) equation for presenting results. The results are presented in a theorem-proof format and elaborated on through both discussion and examples, including results that are not available in any other book. A two-state Markov decision process model, presented in Chapter 3, is analyzed repeatedly throughout the book and demonstrates many results an…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Rudin, W. (1973) · Cited by 13,740
Introduction to Stochastic Dynamic Programming.
Berry, D. A., & Ross, S. M. (1986) · Journal of the American Statistical Association · doi:10.2307/2289262 · Cited by 1,130
White, C. C., & White, D. J. (1989) · European Journal of Operational Research · doi:10.1016/0377-2217(89)90348-2 · Cited by 422
Dynamic programming: Models and applications
Denardo, E. V. (1982) · CERN Document Server (European Organization for Nuclear Research) · Cited by 411
Finding Optimal (s, S) Policies Is About As Simple As Evaluating a Single Policy
Zheng, Y., & Federgruen, A. (1991) · Operations Research · doi:10.1287/opre.39.4.654 · Cited by 317
Optimization over Time. Dynamic Programming and Stochastic Control. Volume 1
Thomas, L. C., & Whittle, P. (1983) · Journal of the Operational Research Society · doi:10.1057/jors.1983.56 · Cited by 162
Dynamic Programming and Its Application
Puterman, M. L. (1979) · Academic Press, Inc. eBooks · Cited by 57
Self-citationOn Homogeneous Markov Models with Continuous Time and Finite or Countable State Space
Yushkevic, A. A., & Fainberg, E. A. (1979) · Theory of Probability and Its Applications · doi:10.1137/1124014 · Cited by 34
Finite Dynamic Programming: An Approach to Finite Markov Decision Processes
White, D. J. (1978) · Andalas University Repository (Andalas University) · Cited by 24
Cited by
Sorted by citations
Items 301–400 of 1,012
Optimal Price and Quantity Determination for Retail Electric Power Contracts
Gabriel, S. A., Conejo, A. J., Plazas, M. A., & Balakrishnan, S. (2006) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2005.860920 · Cited by 138
Zhang, N., & Si, W. (2020) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2020.107094 · Cited by 146
A stochastic programming approach for clinical trial planning in new drug development
Colvin, M., & Maravelias, C. T. (2007) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2007.11.010 · Cited by 138
Bayesian modelling of Jumping-to-Conclusions bias in delusional patients
Moutoussis, M., Bentall, R. P., El-Deredy, W., & Dayan, P. (2011) · Cognitive Neuropsychiatry · doi:10.1080/13546805.2010.548678 · Cited by 137
On Minimizing Broadcast Completion Delay for Instantly Decodable Network Coding
Sorour, S., & Valaee, S. (2010) · International Conference on Communications · doi:10.1109/icc.2010.5502758 · Cited by 137
Liu, B., Wu, S., Xie, M., & Kuo, W. (2017) · European Journal of Operational Research · doi:10.1016/j.ejor.2017.05.006 · Cited by 137
A real-time bus dispatching policy to minimize passenger wait on a high frequency route
Berrebi, S., Watkins, K. E., & Laval, J. A. (2015) · Transportation Research Part B Methodological · doi:10.1016/j.trb.2015.05.012 · Cited by 137
Algorithm Selection using Reinforcement Learning
Lagoudakis, M. G., & Littman, M. L. (2000) · Cited by 137
Approaches for performing uncertainty analysis in large-scale energy/economic policy models
Kann, A., & Weyant, J. P. (2000) · Environmental Modeling & Assessment · doi:10.1023/a:1019041023520 · Cited by 137
Value of information in remanufacturing complex products
Ferrer, G., & Ketzenberg, M. E. (2004) · IIE Transactions · doi:10.1080/07408170490274223 · Cited by 137
Energy Efficient Transmission Strategies for Body Sensor Networks with Energy Harvesting
Seyedi, A., & Sikdar, B. (2010) · IEEE Transactions on Communications · doi:10.1109/tcomm.2010.07.080544 · Cited by 136
Transfer via inter-task mappings in policy search reinforcement learning
Taylor, M. E., Whiteson, S., & Stone, P. (2007) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/1329125.1329170 · Cited by 136
Apprenticeship Learning About Multiple Intentions
Babes, M., Marivate, V. N., Subramanian, K. A., & Littman, M. L. (2011) · International Conference on Machine Learning · Cited by 136
Zheng, Q., Zheng, K., Zhang, H., & Leung, V. C. M. (2016) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2016.2538461 · Cited by 136
Lin, C.-C., Deng, D.-J., & Yao, C.-C. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2690961 · Cited by 136
Zerka, F., Barakat, S., Walsh, S., Bogowicz, M., Leijenaar, R. T. H., Jochems, A., Miraglio, B., Townend, D. M. R., & Lambin, P. (2020) · JCO Clinical Cancer Informatics · doi:10.1200/cci.19.00047 · Cited by 145
A multi-layer and multi-ensemble stock trader using deep learning and deep reinforcement learning
Carta, S., Corriga, A., Ferreira, A., Podda, A. S., & Recupero, D. R. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01839-5 · Cited by 136
Dynamic priority allocation via restless bandit marginal productivity indices
Niño‐Mora, J. (2007) · Top · doi:10.1007/s11750-007-0025-0 · Cited by 136
On Probabilistic Computation Tree Logic
Ciesinski, F., & Größer, M. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24611-4_5 · Cited by 138
Abstraction and approximate decision-theoretic planning
Dearden, R., & Boutilier, C. E. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(96)00023-9 · Cited by 140
Planning under Continuous Time and Resource Uncertainty: A Challenge for AIPreprint
Bresina, J. L., Dearden, R., Meuleau, N., Ramkrishnan, S., Smith, D. E., & Washington, R. M. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.0559 · Cited by 136
A theoretical analysis of Model-Based Interval Estimation
Strehl, A. L., & Littman, M. L. (2005) · International Conference on Machine Learning · doi:10.1145/1102351.1102459 · Cited by 135
Subcortical Substrates of Explore-Exploit Decisions in Primates
Costa, V. D., Mitz, A. R., & Averbeck, B. B. (2019) · Neuron · doi:10.1016/j.neuron.2019.05.017 · Cited by 135
On modeling stochastic dynamic vehicle routing problems
Ulmer, M. W., Goodson, J. C., Mattfeld, D. C., & Thomas, B. W. (2020) · EURO Journal on Transportation and Logistics · doi:10.1016/j.ejtl.2020.100008 · Cited by 135
A general approach for population games with application to vaccination
Reluga, T. C., & Galvani, A. P. (2011) · Mathematical Biosciences · doi:10.1016/j.mbs.2011.01.003 · Cited by 134
Optimal Inspection and Replacement Policies for Multi-Unit Systems Subject to Degradation
Sun, Q., Ye, Z., & Chen, N. (2017) · IEEE Transactions on Reliability · doi:10.1109/tr.2017.2778283 · Cited by 134
DexMV: Imitation Learning for Dexterous Manipulation from Human Videos
Qin, Y., Wu, Y.-H., Liu, S., Jiang, H., Yang, R., Fu, Y. L., & Wang, X. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19842-7_33 · Cited by 135
Baek, J.-Y., & Kaddoum, G. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3009540 · Cited by 133
The First Probabilistic Track of the International Planning Competition
Younes, H. L. S., Littman, M. L., Weissman, D., & Asmuth, J. C. (2005) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1880 · Cited by 133
Transferring Instances for Model-Based Reinforcement Learning
Taylor, M. E., Jong, N. K., & Stone, P. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87481-2_32 · Cited by 139
Markov decision processes with delays and asynchronous cost collection
Katsikopoulos, K. V., & Engelbrecht, S. E. (2003) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2003.809799 · Cited by 129
Deep Q-Learning With Q-Matrix Transfer Learning for Novel Fire Evacuation Environment
Sharma, J., Andersen, P., Granmo, O.-C., & Goodwin, M. (2020) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2020.2967936 · Cited by 136
Yoo, H., Kim, B., Kim, J. W., & Lee, J. H. (2020) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2020.107133 · Cited by 137
Machine Learning Meets Communication Networks: Current Trends and Future Challenges
Ahmad, I., Shahabuddin, S., Malik, H. H., Harjula, E., Leppänen, T., Lovén, L., Anttonen, A., Sodhro, A. H., Alam, M. M., Juntti, M. J., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3041765 · Cited by 140
The orienteering problem with stochastic travel and service times
Campbell, A. M., Michel Gendreau, A., & Thomas, B. W. (2011) · Annals of Operations Research · doi:10.1007/s10479-011-0895-2 · Cited by 132
REGAL: A Regularization based Algorithm for Reinforcement Learning in Weakly Communicating MDPsPreprint
Bartlett, P. L., & Tewari, A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2661 · Cited by 132
Integrating reinforcement learning with human demonstrations of varying ability
Taylor, M. E., Suay, H. B., & Chernova, S. (2011) · doi:10.5555/2031678.2031705 · Cited by 131
Optimal admission control policy for mobile cloud computing hotspot with cloudlet
Hoang, D. T., Niyato, D. T., & Wang, P. (2012) · IEEE Wireless Communications and Networking Conference · doi:10.1109/wcnc.2012.6214347 · Cited by 128
An elective surgery scheduling problem considering patient priority
Min, D., & Yih, Y. (2009) · Computers & Operations Research · doi:10.1016/j.cor.2009.09.016 · Cited by 130
He, M., Murugesan, S., & Zhang, J. (2011) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.2011.5935204 · Cited by 130
Budgeting Time for Dynamic Vehicle Routing with Stochastic Customer Requests
Ulmer, M. W., Mattfeld, D. C., & Köster, F. (2017) · Transportation Science · doi:10.1287/trsc.2016.0719 · Cited by 130
Hierarchical Reinforcement Learning for Air-to-Air Combat
Pope, A. P., Ide, J. S., Mićović, D., Díaz, H., Rosenbluth, D., Ritholtz, L., Twedt, J. C., Walker, T. T., Alcedo, K., & Javorsek, D. (2021) · International Conference on Unmanned Aircraft Systems (ICUAS) · doi:10.1109/icuas51884.2021.9476700 · Cited by 135
Constraint Solving in Uncertain and Dynamic Environments: A Survey
Verfaillie, G., & Jussien, N. (2005) · Constraints · doi:10.1007/s10601-005-2239-9 · Cited by 129
Adaptive resource allocation for efficient patient scheduling
Vermeulen, I. B., Bohté, S. M., Elkhuizen, S., Lameris, H., Bakker, P. J. M., & Poutré, H. L. (2008) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2008.07.019 · Cited by 129
Technical challenges in the application of adaptive management
Williams, B. K., & Brown, E. D. (2016) · Biological Conservation · doi:10.1016/j.biocon.2016.01.012 · Cited by 129
Optimal Attack Energy Allocation against Remote State Estimation
Peng, L., Shi, L., Cao, X., & Sun, C. (2017) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2017.2775344 · Cited by 129
Intra-Option Learning about Temporally Abstract Actions
Sutton, R. S., Precup, D., & Singh, S. P. (1998) · Cited by 129
Design and planning under uncertainty: issues on problem formulation and solution
Cheng, L., Subrahmanian, E., & Westerberg, A. (2003) · Computers & Chemical Engineering · doi:10.1016/s0098-1354(02)00264-8 · Cited by 129
Cheng, L., & Durán, M. A. (2003) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2003.09.025 · Cited by 130
Lei, L., Xu, H., Xiong, X., Zheng, K., & Xiang, W. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2900550 · Cited by 129
Gayon, J.-P., Benjaafar, S., & de Véricourt, F. (2008) · Manufacturing & Service Operations Management · doi:10.1287/msom.1070.0201 · Cited by 128
Pricing in vehicle sharing systems: optimization in queuing networks with product forms
Waserhole, A., & Jost, V. (2014) · EURO Journal on Transportation and Logistics · doi:10.1007/s13676-014-0054-4 · Cited by 127
Qiu, Q., Cui, L., & Wu, B. (2020) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2020.107069 · Cited by 127
Outpatient appointment scheduling given individual day-dependent no-show predictions
Samorani, M., & LaGanga, L. R. (2015) · European Journal of Operational Research · doi:10.1016/j.ejor.2014.06.034 · Cited by 126
Collision Avoidance for Unmanned Aircraft using Markov Decision Processes*
Temizer, S., Kochenderfer, M. J., Kaelbling, L. P., Lozano‐Pérez, T., & Kuchar, J. K. (2010) · AIAA Guidance, Navigation, and Control Conference and Exhibit · doi:10.2514/6.2010-8040 · Cited by 126
Time Consistent Dynamic Risk Measures
Boda, K., & Filar, J. A. (2006) · Mathematical Methods of Operations Research · doi:10.1007/s00186-005-0045-1 · Cited by 126
Networked Multi-Agent Reinforcement Learning in Continuous Spaces
Zhang, K., Yang, Z., & Başar, T. (2018) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2018.8619581 · Cited by 128
Silva, M. A. L., de Souza, S. R., Souza, M. J. F., & Bazzan, A. L. C. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.04.056 · Cited by 132
Hoeffding's inequality for uniformly ergodic Markov chains
Glynn, P. W., & Ormoneit, D. (2002) · Statistics & Probability Letters · doi:10.1016/s0167-7152(01)00158-4 · Cited by 126
Optimum maintenance policy with Markov processes
Chan, G. K., & Asgarpoor, S. (2005) · Electric Power Systems Research · doi:10.1016/j.epsr.2005.09.010 · Cited by 126
Asghari, A., Sohrabi, M. K., & Yaghmaee, F. (2020) · The Journal of Supercomputing · doi:10.1007/s11227-020-03364-1 · Cited by 125
Managing Slow‐Moving Perishables in the Grocery Industry
Ketzenberg, M. E., & Ferguson, M. E. (2008) · Production and Operations Management · doi:10.3401/poms.1080.0052 · Cited by 125
Hadfield-Menell, D., Milli, S., Abbeel, P., Russell, S., & Dragan, A. (2017) · arXiv · doi:10.48550/arxiv.1711.02827 · Cited by 125
Yang, H., Li, W., & Wang, B. (2021) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2021.107713 · Cited by 124
Distributed and adaptive traffic signal control within a realistic traffic simulation
McKenney, D., & White, T. (2012) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2012.04.008 · Cited by 124
A review on computational intelligence methods for controlling traffic signal timing
Araghi, S., Khosravi, A., & Creighton, D. (2014) · Expert Systems with Applications · doi:10.1016/j.eswa.2014.09.003 · Cited by 131
Behavior transfer for value-function-based reinforcement learning
Taylor, M. E., & Stone, P. (2005) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/1082473.1082482 · Cited by 123
Charging Electric Vehicles in the Smart City: A Survey of Economy-Driven Approaches
Shuai, W., Maillé, P., & Pelov, A. P. (2016) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2016.2519499 · Cited by 123
Managing Learning and Turnover in Employee Staffing
Gans, N., & Zhou, Y. (2002) · Operations Research · doi:10.1287/opre.50.6.991.343 · Cited by 123
Frank, K. A., & Fahrbach, K. R. (1999) · Organization Science · doi:10.1287/orsc.10.3.253 · Cited by 125
Fair Dynamic Routing in Large-Scale Heterogeneous-Server Systems
Armony, M., & Ward, A. R. (2010) · Operations Research · doi:10.1287/opre.1090.0777 · Cited by 122
Optimal Breast Biopsy Decision-Making Based on Mammographic Features and Demographic Factors
Chhatwal, J., Alagöz, O., & Burnside, E. S. (2010) · Operations Research · doi:10.1287/opre.1100.0877 · Cited by 122
Knows what it knows: a framework for self-aware learning
Li, L. C., Littman, M. L., Walsh, T. J., & Strehl, A. L. (2010) · Machine Learning · doi:10.1007/s10994-010-5225-4 · Cited by 127
SlateQ: A Tractable Decomposition for Reinforcement Learning with Recommendation Sets
Ie, E., Jain, V., Wang, J., Narvekar, S., Agarwal, R., Wu, R., Cheng, H.-T., Chandra, T., & Boutilier, C. E. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/360 · Cited by 122
Model-based approximate querying in sensor networks
Deshpande, A., Guestrin, C., Madden, S. R., Hellerstein, J. M., & Hong, W. (2005) · The VLDB Journal · doi:10.1007/s00778-005-0159-3 · Cited by 122
Learning action-oriented models through active inference
Tschantz, A., Seth, A. K., & Buckley, C. L. (2020) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1007805 · Cited by 121
Zhu, L., Yu, F. R., Ning, B., & Tang, T. (2012) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2012.120506 · Cited by 121
A Tutorial on Linear Function Approximators for Dynamic Programming and Reinforcement Learning
Geramifard, A., Walsh, T. J., Tellex, S., Chowdhary, G. V., Roy, N., & How, J. P. (2013) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000042 · Cited by 124
Demand Response Management for Profit Maximizing Energy Loads in Real-Time Electricity Market
Wang, S., Bi, S., & Zhang, Y. A. (2018) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2018.2827401 · Cited by 121
Christel Baier, A., Hermanns, H., Katoen, J.-P., & Haverkort, B. R. H. M. (2005) · Theoretical Computer Science · doi:10.1016/j.tcs.2005.07.022 · Cited by 121
Smart Maintenance Decision Support Systems (SMDSS) based on corporate big data analytics
Bumblauskas, D., Gemmill, D., Igou, A., & Anzengruber, J. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.08.025 · Cited by 120
Completion Delay Minimization for Instantly Decodable Network Codes
Sorour, S., & Valaee, S. (2014) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2014.2338053 · Cited by 119
Li, H., & Womer, N. K. (2015) · European Journal of Operational Research · doi:10.1016/j.ejor.2015.04.015 · Cited by 119
Aviv, Y., Wei, M. M., & Zhang, F. (2019) · Management Science · doi:10.1287/mnsc.2018.3114 · Cited by 118
Zhang, X., Zhang, J., Liu, Z., Cui, Q., Tao, X., & Wang, S. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2965159 · Cited by 119
APRICODD: Approximate Policy Construction Using Decision Diagrams
St‐Aubin, R., Hoey, J., & Boutilier, C. E. (2000) · Cited by 119
Preference-based reinforcement learning: a formal framework and a policy iteration algorithm
Fürnkranz, J., Hüllermeier, E., Cheng, W., & Park, S.-H. (2012) · Machine Learning · doi:10.1007/s10994-012-5313-8 · Cited by 129
Liu, B., Pandey, M. D., Wang, X., & Zhao, X. (2021) · European Journal of Operational Research · doi:10.1016/j.ejor.2021.03.010 · Cited by 118
Effect of human guidance and state space size on Interactive Reinforcement Learning
Suay, H. B., & Chernova, S. (2011) · IEEE RO-MAN · doi:10.1109/roman.2011.6005223 · Cited by 118
A tutorial on partially observable Markov decision processes
Littman, M. L. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2009.01.005 · Cited by 117
DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution CorrectionsPreprint
Nachum, O., Chow, Y., Dai, B., & Li, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04733 · Cited by 130
Estimating Dynamic Treatment Regimes in Mobile Health Using V-Learning
Luckett, D. J., Laber, E. B., Kahkoska, A. R., Maahs, D. M., Mayer‐Davis, E. J., & Kosorok, M. R. (2018) · Journal of the American Statistical Association · doi:10.1080/01621459.2018.1537919 · Cited by 124
Predictive Resilience Analysis of Complex Systems Using Dynamic Bayesian Networks
Yodo, N., Wang, P., & Zhou, Z. (2017) · IEEE Transactions on Reliability · doi:10.1109/tr.2017.2722471 · Cited by 116
Omega-Regular Objectives in Model-Free Reinforcement Learning
Hahn, E. M., Perez, M., Schewe, S., Somenzi, F., Trivedi, A., & Wojtczak, D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-17462-0_27 · Cited by 122
Online Haggling at a Name-Your-Own-Price Retailer: Theory and Application
Terwiesch, C., Savin, S. V., & Hann, I.-H. (2005) · Management Science · doi:10.1287/mnsc.1040.0337 · Cited by 116
Algorithms, Measures and Upper Bounds for Satisfiability and Related Problems
Wahlström, M. (2007) · Cited by 122
Dynamic Revenue Maximization with Heterogeneous Objects: A Mechanism Design Approach
Gershkov, A., & Moldovanu, B. (2009) · American Economic Journal Microeconomics · doi:10.1257/mic.1.2.168 · Cited by 111
Protocol and Practice in the Adaptive Management of Waterfowl Harvests
Johnson, F. A., & Williams, K. (1999) · Conservation Ecology · doi:10.5751/es-00091-030108 · Cited by 116
QoS-Aware and Energy-Efficient Resource Management in OFDMA Femtocells
Le, L. B., Niyato, D. T., Hossain, E., Kim, D. I., & Hoang, D. T. (2012) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2012.120412.120141 · Cited by 115
Decentralized Q-Learning for Stochastic Teams and Games
Arslan, G., & Yüksel, S. (2016) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2016.2598476 · Cited by 115
Metadata
- Authors
- Hazeghi, K., & Puterman, M. L.
- ORCID
- Puterman, Martin L.
- Venue
- Journal of the American Statistical Association
- Publisher
- JSTOR
- Published
- 1995-03
- Volume
- 90
- Issue
- 429
- Pages
- 392
- ISSN
- 0162-1459 (ISSN-L)
- OpenAlex
- W2119567691
- Cited by
- 8,037
- References
- 12
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 86 · 2023: 145 · 2022: 233 · 2021: 616 · 2020: 674 · 2019: 595 · 2018: 480 · 2017: 459 · 2016: 450 · 2015: 507 · 2014: 451 · 2013: 420 · 2012: 392
- Type
- book review
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- Markov decision process, stochastic dynamic programming, discrete-time models, infinite horizon model, finite horizon model, Bellman equation, optimality criteria, average reward criterion, constrained models, sequential decision making under uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
