Constrained Markov Decision Processes: Stochastic Modeling
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
This book provides a unified approach for the study of constrained Markov decision processes with a finite state space and unbounded costs. Unlike the single controller case considered in many other books, the author considers a single controller with several objectives, such as minimizing delays and loss, probabilities, and maximization of throughputs. It is desirable to design a controller that minimizes one cost objective, subject to inequality constraints on other cost objectives. This framework describes dynamic decision problems arising frequently in many engineering fields. A thorough overview of these applications is presented in the introduction. The book is then divided into three sections that build upon each other.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Horn, R. A., & Johnson, C. R. (1985) · Cited by 22,203
Convergence of Probability Measures
Billingsley, P. (1999) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316962 · Cited by 13,910
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (1994) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316887 · Cited by 6,024
Markov Chains and Stochastic Stability
Meyn, S. P., & Tweedie, R. L. (1993) · Springer eBooks · doi:10.1007/978-1-4471-3267-7 · Cited by 5,041
An Introduction to the Theory of Point Processes
Daley, D. J., & Vere-Jones, D. (2008) · Probability and Its Applications · doi:10.1007/978-0-387-49835-5 · Cited by 3,351
Shapley, L. S. (1953) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.39.10.1095 · Cited by 2,382
Sion, M. (1958) · Pacific Journal of Mathematics · doi:10.2140/pjm.1958.8.171 · Cited by 1,843
Finite State Markovian Decision Processes
Derman, C. (1970) · Cited by 956
11. Extensive Games and the Problem of Information
Kuhn, H. W. (1953) · Contributions to the Theory of Games (AM-28), Volume II · doi:10.1515/9781400881970-012 · Cited by 913
Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey
Arapostathis, A., Borkar, V. S., Fernández-Gaucherand, E., Ghosh, M. K., & Marcus, S. I. (1993) · SIAM Journal on Control and Optimization · doi:10.1137/0331018 · Cited by 592
Linear Programming and Sequential Decisions
Manne, A. S. (1960) · Management Science · doi:10.1287/mnsc.6.3.259 · Cited by 437
A Statewide Pavement Management System
Golabi, K., Kulkarni, R. B., & Way, G. B. (1982) · INFORMS Journal on Applied Analytics · doi:10.1287/inte.12.6.5 · Cited by 378
On the continuity of the minimum set of a continuous function
Dantzig, G. B., Folkman, J., & Shapiro, N. (1967) · Journal of Mathematical Analysis and Applications · doi:10.1016/0022-247x(67)90139-4 · Cited by 244
28. Mixed and Behavior Strategies in Infinite Extensive Games
Aumann, R. J. (1964) · Princeton University Press eBooks · doi:10.1515/9781400882014-029 · Cited by 217
A Probabilistic Production and Inventory Problem
d'Epenoux, F. (1963) · Management Science · doi:10.1287/mnsc.10.1.98 · Cited by 201
Multichain Markov Renewal Programs
Denardo, E. V., & Fox, B. L. (1968) · SIAM Journal on Applied Mathematics · doi:10.1137/0116038 · Cited by 158
Linear Programming and Markov Decision Chains
Hordijk, A., & Kallenberg, L. C. M. (1979) · Management Science · doi:10.1287/mnsc.25.4.352 · Cited by 143
Constrained Semi-Markov decision processes with average rewards
Feinberg, E. A. (1994) · Mathematical Methods of Operations Research · doi:10.1007/bf01435458 · Cited by 101
A Markovian Model for Hospital Admission Scheduling
KOLESAR, P. J. (1970) · Management Science · doi:10.1287/mnsc.16.6.b384 · Cited by 95
Time-average optimal constrained semi-Markov decision processes
Beutler, F. J., & Ross, K. W. (1986) · Advances in Applied Probability · doi:10.2307/1427303 · Cited by 89
Constrained Markov Decision Models with Weighted Discounted Rewards
Feinberg, E. A., & Shwartz, A. (1995) · Mathematics of Operations Research · doi:10.1287/moor.20.2.302 · Cited by 76
Mean-Variance Tradeoffs in an Undiscounted MDP
Sobel, M. J. (1994) · Operations Research · doi:10.1287/opre.42.1.175 · Cited by 63
Some Remarks on Finite Horizon Markovian Decision Models
Derman, C., & Klein, M. (1965) · Operations Research · doi:10.1287/opre.13.2.272 · Cited by 58
Adaptive control of constrained Markov chains
Altman, E., & Schwartz, A. (1991) · IEEE Transactions on Automatic Control · doi:10.1109/9.75103 · Cited by 37
Self-citationAsymptotic properties of constrained Markov Decision Processes
Altman, E. (1993) · Mathematical Methods of Operations Research · doi:10.1007/bf01414154 · Cited by 25
Self-citationSolving stochastic dynamic programming problems by linear programming — An annotated bibliography
Heilmann, W. R. (1978) · Mathematical Methods of Operations Research · doi:10.1007/bf01917643 · Cited by 14
Cited by
Stochastic Network Optimization with Application to Communication and Queueing Systems
Neely, M. J. (2010) · Synthesis lectures on communication networks · doi:10.2200/s00271ed1v01y201006cnt007 · Cited by 2,218
Update or Wait: How to Keep Your Data Fresh
Sun, Y., Uysal-Biyikoglu, E., Yates, R. D., Koksal, C. E., & Shroff, N. B. (2017) · IEEE Transactions on Information Theory · doi:10.1109/tit.2017.2735804 · Cited by 817
Constrained Policy Optimization
Achiam, J., Held, D., Tamar, A., & Abbeel, P. (2017) · arXiv · doi:10.48550/arxiv.1705.10528 · Cited by 804
Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning
Brunke, L., Greeff, M., Hall, A. W., Yuan, Z., Zhou, S., Panerati, J., & Schoellig, A. P. (2022) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-042920-020211 · Cited by 769
Delay-optimal computation task scheduling for mobile-edge computing systems
Liu, J., Mao, Y., Zhang, J., & Letaief, K. B. (2016) · 2016 IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2016.7541539 · Cited by 759
Metadata
- Authors
- Altman, E.
- Venue
- Routledge eBooks
- Publisher
- Routledge
- Published
- 2021-12-13
- OpenAlex
- W1518931405
- Cited by
- 1,693
- References
- 26
- FWCI
- 101.30
- Type
- book
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.