The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Markov decision processes: Discrete stochastic dynamic programming
Abstract
From the Publisher: The past decade has seen considerable theoretical and applied research on Markov decision processes, as well as the growing use of these models in ecology, economics, communications engineering, and other fields where outcomes are uncertain and sequential decision-making processes are needed. A timely response to this increased activity, Martin L. Puterman's new work provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models. It discusses all major research directions in the field, highlights many significant applications of Markov decision processes models, and explores numerous important topics that have previously been neglected or given cursory coverage in the literature. Markov Decision Processes focuses primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous-time discrete state models. The book is organized around optimality criteria, using a common framework centered on the optimality (Bellman) equation for presenting results. The results are presented in a theorem-proof format and elaborated on through both discussion and examples, including results that are not available in any other book. A two-state Markov decision process model, presented in Chapter 3, is analyzed repeatedly throughout the book and demonstrates many results an…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Rudin, W. (1973) · Cited by 13,740
Introduction to Stochastic Dynamic Programming.
Berry, D. A., & Ross, S. M. (1986) · Journal of the American Statistical Association · doi:10.2307/2289262 · Cited by 1,130
White, C. C., & White, D. J. (1989) · European Journal of Operational Research · doi:10.1016/0377-2217(89)90348-2 · Cited by 422
Dynamic programming: Models and applications
Denardo, E. V. (1982) · CERN Document Server (European Organization for Nuclear Research) · Cited by 411
Finding Optimal (s, S) Policies Is About As Simple As Evaluating a Single Policy
Zheng, Y., & Federgruen, A. (1991) · Operations Research · doi:10.1287/opre.39.4.654 · Cited by 317
Optimization over Time. Dynamic Programming and Stochastic Control. Volume 1
Thomas, L. C., & Whittle, P. (1983) · Journal of the Operational Research Society · doi:10.1057/jors.1983.56 · Cited by 162
Dynamic Programming and Its Application
Puterman, M. L. (1979) · Academic Press, Inc. eBooks · Cited by 57
Self-citationOn Homogeneous Markov Models with Continuous Time and Finite or Countable State Space
Yushkevic, A. A., & Fainberg, E. A. (1979) · Theory of Probability and Its Applications · doi:10.1137/1124014 · Cited by 34
Finite Dynamic Programming: An Approach to Finite Markov Decision Processes
White, D. J. (1978) · Andalas University Repository (Andalas University) · Cited by 24
Flagged first, then by citations
Cited by
Sorted by citations
Items 501–600 of 1,012
Neural Policy Gradient Methods: Global Optimality and Rates of ConvergencePreprint
Wang, L., Cai, Q., Yang, Z., & Wang, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01150 · Cited by 172
Das, T. K., Savachkin, A. A., & Zhu, Y. (2008) · IIE Transactions · doi:10.1080/07408170802165856 · Cited by 97
Filtering Random Graph Processes Over Random Time-Varying Graphs
Isufi, E., Loukas, A., Simonetto, A., & Leus, G. J. T. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2017.2706186 · Cited by 97
Niño‐Mora, J. (2002) · Mathematical Programming · doi:10.1007/s10107-002-0362-6 · Cited by 97
Modeling and probabilistic reasoning of population evacuation during large-scale disaster
Song, X., Zhang, Q., Sekimoto, Y., Horanont, T., Ueyama, S., & Shibasaki, R. (2013) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2487575.2488189 · Cited by 97
Mean Field for Markov Decision Processes: From Discrete to Continuous Optimization
Gast, N., Gaujal, B., & Boudec, J. L. (2012) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2012.2186176 · Cited by 97
Exact Solutions to Time-Dependent MDPs
Boyan, J. A., & Littman, M. L. (2000) · Cited by 97
Efficient Exploration via State Marginal MatchingPreprint
Lee, L., Eysenbach, B., Parisotto, E., Xing, E. P., Levine, S., & Salakhutdinov, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05274 · Cited by 97
Iglesias, A., Martínez, P., Aler, R., & Fernández, F. (2009) · Applied Intelligence · doi:10.1007/s10489-008-0115-1 · Cited by 96
On Bayesian index policies for sequential resource allocation
Kaufmann, E. (2018) · The Annals of Statistics · doi:10.1214/17-aos1569 · Cited by 96
Li, R., Ma, Q., Gong, J., Zhou, Z., & Chen, X. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3064055 · Cited by 96
Multi-channel opportunistic access: A case of restless bandits with multiple plays
Ahmad, S. H. A., & Liu, M. (2009) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2009.5394517 · Cited by 96
Checkpointing strategies for parallel jobs
Bougeret, M., Casanova, H., Rabie, M., Robert, Y. L., & Vivien, F. (2011) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/2063384.2063428 · Cited by 95
Asymptotic Optimality of Tailored Base-Surge Policies in Dual-Sourcing Inventory Systems
Xin, L., & Goldberg, D. A. (2017) · Management Science · doi:10.1287/mnsc.2016.2607 · Cited by 95
Sustainable Task Offloading in UAV Networks via Multi-Agent Reinforcement Learning
Sacco, A., Esposito, F., Marchetto, G., & Montuschi, P. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3074304 · Cited by 94
Monte Carlo-based assessment of system availability. A case study for cogeneration plants
Márquez, A. C., Sánchez-Herguedas, A., & Iung, B. (2004) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2004.07.018 · Cited by 95
Toward an Individual Customer Profitability Model
Libai, B., Narayandas, D., & Humby, C. (2002) · Journal of Service Research · doi:10.1177/1094670502005001007 · Cited by 93
Approximate dynamic programming for capacity allocation in the service industry
Schütz, H.-J., & Kolisch, R. (2011) · European Journal of Operational Research · doi:10.1016/j.ejor.2011.09.007 · Cited by 101
Zhu, D., Yang, B., Liu, Y., Wang, Z., Ma, K., & Guan, X. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.118636 · Cited by 94
Planning and scheduling of semi-urgent surgeries
Zonderland, M. E., Boucherie, R. J., Litvak, N., & Vleggeert-Lankamp, C. L. A. (2010) · Health Care Management Science · doi:10.1007/s10729-010-9127-6 · Cited by 94
Planning with qualitative temporal preferences
Bienvenu, M., Fritz, C., & McIlraith, S. A. (2006) · Cited by 94
Decentralized MDPs with sparse interactions
Melo, F. S., & Veloso, M. (2011) · Artificial Intelligence · doi:10.1016/j.artint.2011.05.001 · Cited by 94
Should Start-up Companies Be Cautious? Inventory Policies Which Maximise Survival Probabilities
Archibald, T. W., Thomas, L. C., Betts, J. M., & Johnston, R. B. (2002) · Management Science · doi:10.1287/mnsc.48.9.1161.176 · Cited by 94
Curtailing Intermittent Generation in Electrical Systems
Wu, O. Q., & Kapuściński, R. (2013) · Manufacturing & Service Operations Management · doi:10.1287/msom.2013.0446 · Cited by 94
Xuan, P., & Lesser, V. (2002) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/545056.545078 · Cited by 94
Proving Positive Almost-Sure Termination
Olivier Bournez, A., & Garnier, F. (2005) · Lecture notes in computer science · doi:10.1007/978-3-540-32033-3_24 · Cited by 93
Gwon, Y., Dastangoo, S., Fossa, C. E., & Kung, H. T. (2013) · IEEE Conference on Communications and Network Security · doi:10.1109/cns.2013.6682689 · Cited by 93
Asymptotically optimal priority policies for indexable and nonindexable restless bandits
Verloop, I. M. (2016) · The Annals of Applied Probability · doi:10.1214/15-aap1137 · Cited by 93
A Policy Iteration Algorithm for Computing Fixed Points in Static Analysis of Programs
Costan, A., Gaubert, S., Goubault, É., Martel, M., & Putot, S. (2005) · Lecture notes in computer science · doi:10.1007/11513988_46 · Cited by 93
Albert, L. A., & Mayorga, M. E. (2012) · IIE Transactions · doi:10.1080/0740817x.2012.665200 · Cited by 93
Energy Efficient Multiple Target Tracking in Wireless Sensor Networks
Yeow, W.-L., Tham, C., & Wong, W. (2007) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2007.891480 · Cited by 93
A Laplacian Framework for Option Discovery in Reinforcement LearningPreprint
Machado, M. C., Bellemare, M. G., & Bowling, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00956 · Cited by 93
Synthesis for PCTL in Parametric Markov Decision Processes
Hahn, E. M., Han, T., & Zhang, L. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-20398-5_12 · Cited by 92
Hamilton–Jacobi–Bellman Equations and Approximate Dynamic Programming on Time Scales
Seiffertt, J., Sanyal, S., & Wunsch, D. C. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.923532 · Cited by 92
Making reinforcement learning work on real robots
Kaelbling, L. P., & Smart, W. D. (2002) · Cited by 92
Planning for decentralized control of multiple robots under uncertainty
Amato, C., Konidaris, G., Cruz, G., Maynor, C. A., How, J. P., & Kaelbling, L. P. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139350 · Cited by 91
Optimizing the Start Time of Statin Therapy for Patients with Diabetes
Denton, B. T., Kurt, M., Shah, N. D., Bryant, S. C., & Smith, S. A. (2009) · Medical Decision Making · doi:10.1177/0272989x08329462 · Cited by 88
A theoretic and practical framework for scheduling in a stochastic environment
Bidot, J., Vidal, T., Laborie, P., & Beck, J. C. (2008) · Journal of Scheduling · doi:10.1007/s10951-008-0080-x · Cited by 91
Basic Ideas for Event-Based Optimization of Markov Systems
Cao, X. (2005) · Discrete Event Dynamic Systems · doi:10.1007/s10626-004-6211-4 · Cited by 92
Parameter Synthesis for Markov Models: Faster Than Ever
Quatmann, T., Dehnert, C., Jansen, N., Junges, S., & Katoen, J.-P. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46520-3_4 · Cited by 91
An optimal energy allocation algorithm for energy harvesting wireless sensor networks
Mao, S., Cheung, M. H., & Wong, V. W. S. (2012) · International Conference on Communications · doi:10.1109/icc.2012.6364174 · Cited by 90
Joint Energy Allocation for Sensing and Transmission in Rechargeable Wireless Sensor Networks
Mao, S., Cheung, M. H., & Wong, V. W. S. (2014) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2013.2295603 · Cited by 90
Sample-efficient batch reinforcement learning for dialogue management optimization
Pietquin, O., Geist, M., Chandramohan, S., & Frezza-Buet, H. (2011) · ACM Transactions on Speech and Language Processing · doi:10.1145/1966407.1966412 · Cited by 90
Broek, M. A. J. U. H., Teunter, R. H., de Jonge, B., & Veldman, J. (2021) · European Journal of Operational Research · doi:10.1016/j.ejor.2021.03.044 · Cited by 90
Shou, Z., Di, X. S., Ye, J., Hongtu Zhu, B., Zhang, H., & Hampshire, R. C. (2019) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2019.12.005 · Cited by 90
Shone, R., Glazebrook, K. D., & Zografos, K. G. (2020) · European Journal of Operational Research · doi:10.1016/j.ejor.2020.10.039 · Cited by 90
Trial-Based Heuristic Tree Search for Finite Horizon MDPs
Keller, T., & Helmert, M. (2013) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v23i1.13557 · Cited by 90
Comparing evolutionary and temporal difference methods in a reinforcement learning domain
Taylor, M. E., Whiteson, S., & Stone, P. (2006) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1143997.1144202 · Cited by 90
Hierarchical Adaptive Dynamic Power Management
Ren, Z., Krogh, B. H., & Mărculescu, R. (2005) · IEEE Transactions on Computers · doi:10.1109/tc.2005.66 · Cited by 90
Dynamic Pricing and Learning with Finite Inventories
den Boer, A. V., & Zwart, B. (2015) · Operations Research · doi:10.1287/opre.2015.1397 · Cited by 87
Power Control of an Energy Harvesting Sensor for Remote State Estimation
Li, Y., Zhang, F., Quevedo, D. E., Lau, V. K. N., Dey, S., & Shi, L. (2016) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2016.2553128 · Cited by 90
Probabilistic inference for determining options in reinforcement learning
Daniel, C., van Hoof, H., Peters, J., & Neumann, G. M. (2016) · Machine Learning · doi:10.1007/s10994-016-5580-x · Cited by 95
CANDYMAN: Classifying Android malware families by modelling dynamic traces with Markov chains
Martín, A. G., Rodríguez-Fernández, V., & Camacho, D. (2018) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2018.06.006 · Cited by 89
Wireless Power Transfer and Data Collection in Wireless Sensor Networks
Li, K., Ni, W., Duan, L., Abolhasan, M., & Niu, J. (2017) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2772895 · Cited by 89
Artificial intelligence-based inventory management: a Monte Carlo tree search approach
Preil, D., & Krapp, M. (2021) · Annals of Operations Research · doi:10.1007/s10479-021-03935-2 · Cited by 104
Testing facility location and dynamic capacity planning for pandemics with demand uncertainty
Liu, K., Liu, C., Xiang, X., & Tian, Z. (2021) · European Journal of Operational Research · doi:10.1016/j.ejor.2021.11.028 · Cited by 89
Kwiatkowska, M. (2007) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1287624.1287688 · Cited by 88
Imitation Learning from Imperfect DemonstrationPreprint
Wu, Y.-H., Charoenphakdee, N., Bao, H., Tangkaratt, V., & Sugiyama, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09387 · Cited by 89
Hou, L., Lei, L., Zheng, K., & Wang, X. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2883762 · Cited by 89
Provably Efficient Reinforcement Learning with Linear Function Approximation
Jin, C., Yang, Z., Wang, Z., & Jordan, M. I. (2023) · Mathematics of Operations Research · doi:10.1287/moor.2022.1309 · Cited by 312
Optimal Reliability in Energy Harvesting Industrial Wireless Sensor Networks
Lei, L., Kuang, Y., Shen, X., Yang, K., Qiao, J., & Zhong, Z. D. (2016) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2016.2558146 · Cited by 88
Jeong, H., Shen, Y. C., Jeong, J. P., & Oh, T. T. (2021) · Vehicular Communications · doi:10.1016/j.vehcom.2021.100349 · Cited by 89
Deep Reinforcement Learning Approach for Trading Automation in the Stock Market
Kabbani, T., & Duman, E. (2022) · IEEE Access · doi:10.1109/access.2022.3203697 · Cited by 88
Exponential Lower Bounds for Policy Iteration
Fearnley, J. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-14162-1_46 · Cited by 88
Haijema, R. (2011) · International Journal of Production Economics · doi:10.1016/j.ijpe.2011.05.021 · Cited by 88
Bramley, N. R., Lagnado, D. A., & Speekenbrink, M. (2014) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/xlm0000061 · Cited by 89
Online Markov Decision Processes under Bandit Feedback
Neu, G., Antos, A., György, A., & Szepesvári, C. (2010) · Cited by 88
Relational Reinforcement Learning: An Overview
Tadepalli, P. V., Givan, R. L., & Driessens, K. (2004) · Lirias · Cited by 88
Optimally Solving Dec-POMDPs as Continuous-State MDPs
Dibangoye, J., Amato, C., Buffet, O., & Charpillet, F. (2016) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4623 · Cited by 88
Integrated voice/data call admission control for wireless DS-CDMA systems
Singh, S. S., Krishnamurthy, V., & Poor, H. V. (2002) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2002.1003071 · Cited by 88
Nowak, A. S., & Szajowski, K. (1999) · Birkhäuser Boston eBooks · doi:10.1007/978-1-4612-1592-9_7 · Cited by 87
Optimistic Posterior Sampling for Reinforcement Learning: Worst-Case Regret Bounds
Agrawal, S., & Jia, R. (2022) · Mathematics of Operations Research · doi:10.1287/moor.2022.1266 · Cited by 120
Dynamic pricing of multiple home delivery options
Asdemir, K., Jacob, V. S., & Krishnan, R. (2008) · European Journal of Operational Research · doi:10.1016/j.ejor.2008.03.005 · Cited by 87
A Dispatching Model for Server-to-Customer Systems That Balances Efficiency and Equity
Albert, L. A., & Mayorga, M. E. (2012) · Manufacturing & Service Operations Management · doi:10.1287/msom.1120.0411 · Cited by 87
An object-oriented representation for efficient reinforcement learning
Diuk, C., Cohen, A., & Littman, M. L. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390187 · Cited by 226
Deployment of Blockchain Technology in Software Defined Networks: A Survey
Alharbi, T. (2020) · IEEE Access · doi:10.1109/access.2020.2964751 · Cited by 86
Romaus, C., Gathmann, K., & Böcker, J. (2010) · IEEE Vehicle Power and Propulsion Conference (VPPC) · doi:10.1109/vppc.2010.5728979 · Cited by 86
Variable Resolution Discretization for High-Accuracy Solutions of Optimal Control Problems
Munos, R., & Moore, A. (2018) · doi:10.1184/r1/6561533 · Cited by 86
$\mathsf{Hap-SliceR}$: A Radio Resource Slicing Framework for 5G Networks With Haptic Communications
Aijaz, A. (2017) · IEEE Systems Journal · doi:10.1109/jsyst.2017.2647970 · Cited by 87
Toyoda, M., & Wu, Y. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2954849 · Cited by 85
A Reinforcement Learning Approach to Age of Information in Multi-User Networks With HARQ
Ceran, E. T., Gündüz, D., & György, A. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3065057 · Cited by 85
Distributional Reinforcement Learning
Bellemare, M. G., Dabney, W., & Rowland, M. (2023) · The MIT Press eBooks · doi:10.7551/mitpress/14207.001.0001 · Cited by 102
Exploration of Multi-State Environments: Local Measures and Back-Propagation of Uncertainty
Meuleau, N., & Bourgine, P. (1999) · Machine Learning · doi:10.1023/a:1007541107674 · Cited by 91
Optimization of Prostate Biopsy Referral Decisions
Zhang, J., Denton, B. T., Balasubramanian, H., Shah, N. D., & Inman, B. A. (2012) · Manufacturing & Service Operations Management · doi:10.1287/msom.1120.0388 · Cited by 82
Modelling and control strategy development for fuel cell electric vehicles
Schell, A., Peng, H., Tran, D., Stamos, E., Lin, C.-C., & Kim, M. (2005) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2005.02.001 · Cited by 85
Bridging the Gap Between Imitation Learning and Inverse Reinforcement Learning
Piot, B., Geist, M., & Pietquin, O. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2543000 · Cited by 85
Arcidiacono, P. S., Bayer, P. J., Blevins, J. R., & Ellickson, P. B. (2016) · The Review of Economic Studies · doi:10.1093/restud/rdw012 · Cited by 85
Optimality of Myopic Sensing in Multi-Channel Opportunistic Access
Javidi, T., Krishnamachari, B., Zhao, Q., & Liu, M. (2008) · International Conference on Communications · doi:10.1109/icc.2008.404 · Cited by 84
Intention-Aware Routing of Electric Vehicles
Weerdt, M. M. D., Stein, S., Gerding, E., Robu, V., & Jennings, N. R. (2015) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2015.2506900 · Cited by 84
Joint condition-based maintenance and condition-based production optimization
Broek, M. A. J. U. H., Teunter, R. H., de Jonge, B., & Veldman, J. (2021) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2021.107743 · Cited by 84
Models for maintenance optimization: a study for repairable systems and finite time periods
Márquez, A. C., & Sánchez-Herguedas, A. (2002) · Reliability Engineering & System Safety · doi:10.1016/s0951-8320(01)00131-4 · Cited by 84
Spatially-explicit modeling and intensity analysis of China's land use change 2000–2050
Feng, Y., Lei, Z., Tong, X., Gao, C., Chen, S., Wang, J., & Wang, S. (2020) · Journal of Environmental Management · doi:10.1016/j.jenvman.2020.110407 · Cited by 83
Some indexable families of restless bandit problems
Glazebrook, K. D., Ruiz-Hernández, D., & Kirkbride, C. (2006) · Advances in Applied Probability · doi:10.1017/s000186780000121x · Cited by 83
Bisimulation and cocongruence for probabilistic systems
Vincent Danos, A., Desharnais, J., Laviolette, F., & Panangaden, P. (2006) · Information and Computation · doi:10.1016/j.ic.2005.02.004 · Cited by 83
Multitime scale Markov decision processes
Chang, H. S., Fard, P. J., Marcus, S. I., & Shayman, M. A. (2003) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2003.812782 · Cited by 83
Prioritization Methods for Accelerating MDP Solvers
Wingate, D., & Seppi, K. D. (2005) · ScholarsArchive (Brigham Young University) · doi:10.5555/1046920.1088701 · Cited by 84
Safe Exploration Techniques for Reinforcement Learning – An Overview
Pecka, M., & Svoboda, T. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-13823-7_31 · Cited by 83
Huang, Q., Jia, Q., & Guan, X. (2016) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2016.2585180 · Cited by 82
Scheduling the Operation of a Connected Vehicular Network Using Deep Reinforcement Learning
Atallah, R. F., Assi, C. M., & Khabbaz, M. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2832219 · Cited by 82
Real-Time Remote Estimation With Hybrid ARQ in Wireless Networked Control
Huang, K., Liu, W., Shirvanimoghaddam, M., Li, Y., & Vucetic, B. S. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2974225 · Cited by 82
Metadata
- Authors
- Hazeghi, K., & Puterman, M. L.
- ORCID
- Puterman, Martin L.
- Venue
- Journal of the American Statistical Association
- Publisher
- JSTOR
- Published
- 1995-03
- Volume
- 90
- Issue
- 429
- Pages
- 392
- ISSN
- 0162-1459 (ISSN-L)
- OpenAlex
- W2119567691
- Cited by
- 8,037
- References
- 12
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 86 · 2023: 145 · 2022: 233 · 2021: 616 · 2020: 674 · 2019: 595 · 2018: 480 · 2017: 459 · 2016: 450 · 2015: 507 · 2014: 451 · 2013: 420 · 2012: 392
- Type
- book review
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- Markov decision process, stochastic dynamic programming, discrete-time models, infinite horizon model, finite horizon model, Bellman equation, optimality criteria, average reward criterion, constrained models, sequential decision making under uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
