The DOI matches its Crossref record.
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Abstract
oDepartnrentofStatisticsandoPeratiotts,Resen,ii,i,j::,?!:],,::::::,:,,,,,i:;i:,,:,,,,;;;;'i
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160
A Comprehensive Survey of Multiagent Reinforcement Learning
Busoniu, L., Babuska, R., & De Schutter, B. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2007.913919 · Cited by 2,235
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · Nature · doi:10.1038/s41586-020-03051-4 · Cited by 1,214
The Complexity of Decentralized Control of Markov Decision Processes
Bernstein, D. S., Givan, R., Immerman, N., & Zilberstein, S. (2002) · Mathematics of Operations Research · doi:10.1287/moor.27.4.819.297 · Cited by 1,192
Doya, K. (2008) · Nature Neuroscience · doi:10.1038/nn2077 · Cited by 642
Advances in Energy Harvesting Communications: Past, Present, and Future Challenges
Ku, M.-L., Li, W., Chen, Y., & Ray Liu, K. J. (2016) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2015.2497324 · Cited by 606
A Survey on Machine-Learning Techniques in Cognitive Radios
Bkassiny, M., Li, Y., & Jayaweera, S. K. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.100412.00017 · Cited by 580
State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force–3
Siebert, U., Alagoz, O., Bayoumi, A. M., Jahn, B., Owens, D. K., Cohen, D. J., & Kuntz, K. M. (2012) · Medical Decision Making · doi:10.1177/0272989x12455463 · Cited by 348
A Survey on Service Migration in Mobile Edge Computing
Wang, S., Xu, J., Zhang, N., & Liu, Y. (2018) · IEEE Access · doi:10.1109/access.2018.2828102 · Cited by 348
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Discrete Event Dynamic Systems · doi:10.1023/a:1025696116075 · Cited by 317
Robust Markov Decision Processes
Wiesemann, W., Kuhn, D., & Rustem, B. (2013) · Mathematics of Operations Research · doi:10.1287/moor.1120.0566 · Cited by 312
A Neural Computational Model of Incentive Salience
Zhang, J., Berridge, K. C., Tindell, A. J., Smith, K. S., & Aldridge, J. W. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000437 · Cited by 299
Habits, action sequences and reinforcement learning
Dezfouli, A., & Balleine, B. W. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08050.x · Cited by 297
Reinforcement learning in artificial and biological systems
Neftci, E. O., & Averbeck, B. B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0025-4 · Cited by 268
An Experimental Review of Reinforcement Learning Algorithms for Adaptive Traffic Signal Control
Mannion, P., Duggan, J., & Howley, E. (2016) · Autonomic Road Transport Support Systems · doi:10.1007/978-3-319-25808-9_4 · Cited by 267
Markov decision processes: a tool for sequential decision making under uncertainty
Alagoz, O., Hsu, H., Schaefer, A. J., & Roberts, M. S. (2010) · Medical Decision Making · doi:10.1177/0272989x09353194 · Cited by 266
Relaxations of Weakly Coupled Stochastic Dynamic Programs
Adelman, D., & Mersereau, A. J. (2008) · Operations Research · doi:10.1287/opre.1070.0445 · Cited by 198
A Reinforcement Learning Model to Assess Market Power Under Auction-Based Energy Pricing
Nanduri, V., & Das, T. K. (2007) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2006.888977 · Cited by 168
Modeling Medical Treatment Using Markov Decision Processes
Schaefer, A. J., Bailey, M. D., Shechter, S. M., & Roberts, M. S. (2005) · International Series in Operations Research & Management Science · doi:10.1007/1-4020-8066-2_23 · Cited by 155
Bayesian modelling of Jumping-to-Conclusions bias in delusional patients
Moutoussis, M., Bentall, R. P., El-Deredy, W., & Dayan, P. (2011) · Cognitive Neuropsychiatry · doi:10.1080/13546805.2010.548678 · Cited by 137
Drones’ Edge Intelligence Over Smart Environments in B5G: Blockchain and Federated Learning Synergy
Alsamhi, S. H., Almalki, F. A., Afghah, F., Hawbani, A., Shvetsov, A. V., Lee, B., & Song, H. (2022) · IEEE Transactions on Green Communications and Networking · doi:10.1109/tgcn.2021.3132561 · Cited by 139
Hierarchical Reinforcement Learning: A Survey and Open Research Challenges
Hutsebaut-Buysse, M., Mets, K., & Latré, S. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4010009 · Cited by 129
Markov decision processes with delays and asynchronous cost collection
Katsikopoulos, K. V., & Engelbrecht, S. E. (2003) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2003.809799 · Cited by 129
Outpatient appointment scheduling given individual day-dependent no-show predictions
Samorani, M., & LaGanga, L. R. (2015) · European Journal of Operational Research · doi:10.1016/j.ejor.2014.06.034 · Cited by 126
Learning action-oriented models through active inference
Tschantz, A., Seth, A. K., & Buckley, C. L. (2020) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1007805 · Cited by 121
Adaptive Opportunistic Routing for Wireless Ad Hoc Networks
Bhorkar, A. A., Naghshvar, M., Javidi, T., & Rao, B. D. (2012) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2011.2159844 · Cited by 102
Iglesias, A., Martínez, P., Aler, R., & Fernández, F. (2009) · Applied Intelligence · doi:10.1007/s10489-008-0115-1 · Cited by 96
Epistemic Communities under Active Inference
Albarracin, M., Demekas, D., Ramstead, M. J. D., & Heins, C. (2022) · Entropy · doi:10.3390/e24040476 · Cited by 60
Distributed control design for balancing the grid using flexible loads
Chen, Y., Hashmi, M. U., Mathias, J., Bušić, A., & Meyn, S. (2018) · The IMA Volumes in Mathematics and its Applications · doi:10.1007/978-1-4939-7822-9_16 · Cited by 51
Offsetting omnichannel grocery fulfillment cost through advance ordering of perishables
Siawsolit, C., & Gaukler, G. M. (2021) · International Journal of Production Economics · doi:10.1016/j.ijpe.2021.108192 · Cited by 28
Benchmark Data Contamination of Large Language Models: A Survey
Xu, C., Guan, S., Greene, D., & Kechadi, M.-T. (2024) · arXiv · doi:10.48550/arxiv.2406.04244 · Cited by 22
Interactive explanation for planning-based systems: WIP abstract
Zhao, E., & Sukkerd, R. (2019) · Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems · doi:10.1145/3302509.3313322 · Cited by 7
Metadata
- Authors
- Puterman, M. L.
- Venue
- Wiley Series in Probability and Statistics
- Publisher
- Wiley
- Published
- 1994-04-15
- ISSN
- 1940-6347 (ISSN-L)
- OpenAlex
- W2334782222
- Cited by
- 6,024
- References
- 0
- FWCI
- 9.55
- Type
- book
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.