The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Markov decision processes: Discrete stochastic dynamic programming
Abstract
From the Publisher: The past decade has seen considerable theoretical and applied research on Markov decision processes, as well as the growing use of these models in ecology, economics, communications engineering, and other fields where outcomes are uncertain and sequential decision-making processes are needed. A timely response to this increased activity, Martin L. Puterman's new work provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models. It discusses all major research directions in the field, highlights many significant applications of Markov decision processes models, and explores numerous important topics that have previously been neglected or given cursory coverage in the literature. Markov Decision Processes focuses primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous-time discrete state models. The book is organized around optimality criteria, using a common framework centered on the optimality (Bellman) equation for presenting results. The results are presented in a theorem-proof format and elaborated on through both discussion and examples, including results that are not available in any other book. A two-state Markov decision process model, presented in Chapter 3, is analyzed repeatedly throughout the book and demonstrates many results an…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Rudin, W. (1973) · Cited by 13,740
Introduction to Stochastic Dynamic Programming.
Berry, D. A., & Ross, S. M. (1986) · Journal of the American Statistical Association · doi:10.2307/2289262 · Cited by 1,130
White, C. C., & White, D. J. (1989) · European Journal of Operational Research · doi:10.1016/0377-2217(89)90348-2 · Cited by 422
Dynamic programming: Models and applications
Denardo, E. V. (1982) · CERN Document Server (European Organization for Nuclear Research) · Cited by 411
Finding Optimal (s, S) Policies Is About As Simple As Evaluating a Single Policy
Zheng, Y., & Federgruen, A. (1991) · Operations Research · doi:10.1287/opre.39.4.654 · Cited by 317
Optimization over Time. Dynamic Programming and Stochastic Control. Volume 1
Thomas, L. C., & Whittle, P. (1983) · Journal of the Operational Research Society · doi:10.1057/jors.1983.56 · Cited by 162
Dynamic Programming and Its Application
Puterman, M. L. (1979) · Academic Press, Inc. eBooks · Cited by 57
Self-citationOn Homogeneous Markov Models with Continuous Time and Finite or Countable State Space
Yushkevic, A. A., & Fainberg, E. A. (1979) · Theory of Probability and Its Applications · doi:10.1137/1124014 · Cited by 34
Finite Dynamic Programming: An Approach to Finite Markov Decision Processes
White, D. J. (1978) · Andalas University Repository (Andalas University) · Cited by 24
Cited by
Sorted by citations
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Machine learning: Algorithms, real-world applications and research directions
Sarker, I. H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00592-x · Cited by 5,583
“The problem to be solved in reinforcement learning (RL) is defined as a Markov Decision Process (MDP) [86], i.”
InfluentialPlanning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149
Bayesian Networks and Decision Graphs
Jensen, F. V., & Nielsen, T. D. (2007) · Information science and statistics · doi:10.1007/978-0-387-68282-2 · Cited by 4,157
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160
Sarker, I. H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00815-1 · Cited by 3,137
Theory and Applications of Robust Optimization
Brown, D. B., Caramanis, C., & Bertsimas, D. (2010) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 2,524
Deep reinforcement learning for autonomous driving: A survey
Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A. A. A., Yogamani, S., & Perez, P. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3054625 · Cited by 2,342
A Comprehensive Survey of Multiagent Reinforcement Learning
Busoniu, L., Babuska, R., & De Schutter, B. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2007.913919 · Cited by 2,235
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D. T., Wang, P., Liang, Y., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,181
Generative adversarial imitation learning
Ho, J., & Ermon, S. (2016) · arXiv · doi:10.48550/arxiv.1606.03476 · Cited by 1,942
Mobile cloud computing: A survey
Fernando, N., Loke, S. W., & Rahayu, W. J. (2012) · Future Generation Computer Systems · doi:10.1016/j.future.2012.05.023 · Cited by 1,663
Handbook of Constraint Programming
Rossi, F., van Beek, P., & Walsh, T. (2006) · Foundations of artificial intelligence · doi:10.1016/s1574-6526(06)x8001-x · Cited by 1,845
Transfer Learning for Reinforcement Learning Domains: A Survey
Taylor, M. E., & Stone, P. (2009) · doi:10.5555/1577069.1755839 · Cited by 1,524
NICHOLS, J., & WILLIAMS, B. (2006) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2006.08.007 · Cited by 1,104
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · RePEc: Research Papers in Economics · Cited by 1,310
Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. C., & Faisal, A. A. (2018) · Nature Medicine · doi:10.1038/s41591-018-0213-5 · Cited by 1,290
Artificial Intelligence: A New Synthesis
(1998) · Elsevier eBooks · doi:10.1016/c2009-0-27773-7 · Cited by 1,211
Foundations of Machine Learning
Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2018) · doi:10.7551/mitpress/10290.001.0001 · Cited by 1,122
(2015) · Journal of Machine Learning Research · Cited by 1,119
Algorithms for Reinforcement Learning
Szepesvári, C. (2010) · Synthesis lectures on artificial intelligence and machine learning · doi:10.1007/978-3-031-01551-9 · Cited by 1,331
Action understanding as inverse planning
Baker, C. L., Saxe, R., & Tenenbaum, J. B. (2009) · Cognition · doi:10.1016/j.cognition.2009.07.005 · Cited by 920
Nash Q-Learning for General-Sum Stochastic Games
Hu, J., Wellman, M. P., & Boutilier, C. (2003) · Cited by 923
The misinterpretation of noncanonical sentences
Ferreira, F. (2003) · Cognitive Psychology · doi:10.1016/s0010-0285(03)00005-7 · Cited by 907
Opponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844
Deep Direct Reinforcement Learning for Financial Signal Representation and Trading
Deng, Y., Bao, F., Kong, Y., Ren, Z., & Dai, Q. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2522401 · Cited by 822
Near-Optimal Reinforcement Learning in Polynomial Time
Kearns, M. J., & Singh, S. P. (2002) · Machine Learning · doi:10.1023/a:1017984413808 · Cited by 861
SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces
Kurniawati, H., Hsu, D., & Lee, W. S. (2008) · Robotics: Science and Systems · doi:10.15607/rss.2008.iv.009 · Cited by 776
Model-checking algorithms for continuous-time markov chains
Christel Baier, A., Haverkort, B. R. H. M., Hermanns, H., & Katoen, J.-P. (2003) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2003.1205180 · Cited by 762
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762
Multi-agent Reinforcement Learning: An Overview
Buşoniu, L., Babuška, R., & Schutter, B. D. (2010) · Studies in computational intelligence · doi:10.1007/978-3-642-14435-6_7 · Cited by 772
The Option-Critic Architecture
Bacon, P., Harb, J., & Precup, D. (2017) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v31i1.10916 · Cited by 779
Near-optimal Regret Bounds for Reinforcement Learning
Jaksch, T., Ortner, R., & Auer, P. L. (2010) · Cited by 716
A Distributional Perspective on Reinforcement LearningPreprint
Bellemare, M. G., Dabney, W., & Munos, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.06887 · Cited by 724
Robust Control of Markov Decision Processes with Uncertain Transition Matrices
Nilim, A., & Ghaoui, L. E. (2005) · Operations Research · doi:10.1287/opre.1050.0216 · Cited by 770
A survey and critique of multiagent deep reinforcement learning
Hernández-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 799
Reinforcement learning algorithms: A brief survey
Shakya, A. K., Pillai, G. N., & Chakrabarty, S. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120495 · Cited by 760
Noisy Networks for ExplorationPreprint
Fortunato, M., Azar, M. G., Piot, B., Menick, J., Osband, I., Graves, A., Mnih, V., Munos, R., Hassabis, D., Pietquin, O., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.10295 · Cited by 880
Scripting: higher level programming for the 21st Century
Ousterhout, J. K. (1998) · Computer · doi:10.1109/2.660187 · Cited by 648
Doya, K. (2008) · Nature Neuroscience · doi:10.1038/nn2077 · Cited by 642
Q-Learning Algorithms: A Comprehensive Classification and Applications
Jang, B., Kim, M., Harerimana, G., & Kim, J. W. (2019) · IEEE Access · doi:10.1109/access.2019.2941229 · Cited by 702
Reinforcement Learning and Markov Decision Processes
van Otterlo, M., & Wiering, M. (2012) · Adaptation, learning, and optimization · doi:10.1007/978-3-642-27645-3_1 · Cited by 672
A survey of inverse reinforcement learning: Challenges, methods and progress
Arora, S., & Doshi, P. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103500 · Cited by 619
Reinforcement Learning with Hierarchies of Machines
Parr, R., & Russell, S. (1997) · Cited by 618
Ratliff, N., Bagnell, J. A., & Zinkevich, M. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143936 · Cited by 636
Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms
Singh, S. P., Jaakkola, T., Littman, M. L., & Szepesvári, C. (2000) · Machine Learning · doi:10.1023/a:1007678930559 · Cited by 637
Advances in Energy Harvesting Communications: Past, Present, and Future Challenges
Ku, M.-L., Li, W., Chen, Y., & Ray Liu, K. J. (2016) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2015.2497324 · Cited by 606
Distributional Reinforcement Learning With Quantile Regression
Dabney, W., Rowland, M., Bellemare, M., & Munos, R. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11791 · Cited by 612
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580
Iyengar, G. (2005) · Mathematics of Operations Research · doi:10.1287/moor.1040.0129 · Cited by 580
Natural actor–critic algorithms
Bhatnagar, S., Sutton, R. S., Ghavamzadeh, M., & Lee, M. (2009) · Automatica · doi:10.1016/j.automatica.2009.07.008 · Cited by 581
Value-Function Approximations for Partially Observable Markov Decision Processes
Hauskrecht, M. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.678 · Cited by 534
Model-based Reinforcement Learning: A Survey
Moerland, T. M., Broekens, J., Plaat, A., & Jonker, C. M. (2023) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000086 · Cited by 871
Peng, H., & Shen, X. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036962 · Cited by 524
A comprehensive survey on model compression and acceleration
Choudhary, T., Mishra, V. K., Goswami, A., & Jagannathan, S. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09816-7 · Cited by 555
Perseus: Randomized point-based value iteration for POMDPs
Spaan, M. T. J., & Vlassis, N. A. (2005) · UvA-DARE (University of Amsterdam) · Cited by 526
Security and Game Theory: Algorithms, Deployed Systems, Lessons Learned
Tambe, M. (2011) · Cited by 509
Frameworks and Results in Distributionally Robust Optimization
Rahimian, H., & Mehrotra, S. (2022) · Open Journal of Mathematical Optimization · doi:10.48550/arxiv.1908.05659 · Cited by 999
Two formal analyses of attack graphs
Jha, S., Sheyner, O., & Wing, J. M. (2003) · IEEE Computer Security Foundations Workshop (CSFW) · doi:10.1109/csfw.2002.1021806 · Cited by 502
Online Planning Algorithms for POMDPs
Ross, S., Pineau, J., Paquet, S., & Chaib-draa, B. (2008) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2567 · Cited by 503
Value-Decomposition Networks For Cooperative Multi-Agent LearningPreprint
Sunehag, P., Lever, G., Gruslys, A., Czarnecki, W. M., Zambaldi, V., Jaderberg, M., Lanctot, M., Sonnerat, N., Leibo, J. Z., Tuyls, K., et al. (2017) · arXiv (Cornell University) · doi:10.4230/lipics.cp.2023.25 · Cited by 489
Agent-Oriented Supply-Chain Management
Fox, M. S., Barbuceanu, M., & Teigen, R. (2001) · Springer eBooks · doi:10.1007/978-1-4615-1599-9_5 · Cited by 482
On the Sample Complexity of Reinforcement Learning
Kakade, S. M. (2003) · OpenGrey (Institut de l'Information Scientifique et Technique) · Cited by 510
Relative Entropy Policy Search
Peters, J., Mülling, K., & Altün, Y. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7727 · Cited by 506
Reinforcement learning for control: Performance, stability, and deep approximators
Buşoniu, L., de Bruin, T., Tolić, D., Kober, J., & Palunko, I. (2018) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2018.09.005 · Cited by 490
Opponency revisited: competition and cooperation between dopamine and serotonin
Boureau, Y.-L., & Dayan, P. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.151 · Cited by 456
Cooperative Heterogeneous Multi-Robot Systems
Rizk, Y., Awad, M., & Tunstel, E. W. (2019) · ACM Computing Surveys · doi:10.1145/3303848 · Cited by 491
A stochastic control strategy for hybrid electric vehicles
Lin, C.-C., Peng, H., & Grizzle, J. W. (2004) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.2004.1384056 · Cited by 450
Silver, D., Singh, S., Precup, D., & Sutton, R. S. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103535 · Cited by 445
Applications of reinforcement learning in energy systems
Perera, A. T. D., & Kamalaruban, P. (2020) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.110618 · Cited by 457
An analysis of model-based Interval Estimation for Markov Decision Processes
Strehl, A. L., & Littman, M. L. (2008) · Journal of Computer and System Sciences · doi:10.1016/j.jcss.2007.08.009 · Cited by 428
A survey of point-based POMDP solvers
Shani, G., Pineau, J., & Kaplow, R. (2012) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-012-9200-2 · Cited by 427
Risk-averse dynamic programming for Markov decision processes
Ruszczyński, A. (2010) · Mathematical Programming · doi:10.1007/s10107-010-0393-3 · Cited by 421
System-level power optimization
Luca Benini, C., & Micheli, G. D. (2000) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/335043.335044 · Cited by 415
Fully Decentralized Multi-Agent Reinforcement Learning with Networked AgentsPreprint
Zhang, K., Yang, Z., Liu, H., Zhang, T., & Başar, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.08757 · Cited by 439
State of the art on automatic road extraction for GIS update: a novel classification
Mena, J. B. (2003) · Pattern Recognition Letters · doi:10.1016/s0167-8655(03)00164-8 · Cited by 411
Autonomous experimentation systems for materials development: A community perspective
Stach, E. A., DeCost, B., Kusne, A. G., Hattrick-Simpers, J. R., Brown, K. A., Reyes, K. G., Schrier, J., Billinge, S. J. L., Buonassisi, T., Foster, I. T., et al. (2021) · Matter · doi:10.1016/j.matt.2021.06.036 · Cited by 456
Value-function reinforcement learning in Markov games
Littman, M. L. (2001) · Cognitive Systems Research · doi:10.1016/s1389-0417(01)00015-8 · Cited by 401
An MDP-Based Vertical Handoff Decision Algorithm for Heterogeneous Wireless Networks
Stevens‐Navarro, E., Lin, Y., & Wong, V. W. S. (2008) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2007.907072 · Cited by 395
Reinforcement learning in robotic applications: a comprehensive survey
Singh, B., Kumar, R., & Singh, V. P. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-09997-9 · Cited by 438
Markov Decision Processes with Applications to Finance
Bäuerle, N., & Rieder, U. (2011) · Universitext · doi:10.1007/978-3-642-18324-9 · Cited by 395
Lin, J., Rao, Y., Lu, J., & Zhou, J. (2017) · neural information processing systems · Cited by 399
Lee, J. H., Shin, J., & Realff, M. J. (2017) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2017.10.008 · Cited by 405
Average Reward Reinforcement Learning: Foundations, Algorithms, and Empirical Results
Mahadevan, S. (1996) · Machine Learning · doi:10.1007/bf00114727 · Cited by 419
Fuzzy inference system learning by reinforcement methods
Jouffe, L. (1998) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/5326.704563 · Cited by 385
Planning, Learning and Coordination in Multiagent Decision Processes
Boutilier, C. E. (1996) · Cited by 394
Serotonin in Affective Control
Dayan, P., & Huys, Q. J. M. (2009) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135607 · Cited by 381
SPUDD: Stochastic Planning using Decision DiagramsPreprint
Hoey, J., St‐Aubin, R., Hu, A. J., & Boutilier, C. E. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6704 · Cited by 387
Opportunistic Spectrum Access via Periodic Channel Sensing
Zhao, Q., Geirhofer, S., Tong, L., & Sadler, B. M. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.907867 · Cited by 372
FOR LEAST-SQUARES POLICY ITERATION
Schwab, D. (2016) · Cited by 372
Greenwald, A., & Hall, K. (2003) · Cited by 384
Online Learning for Offloading and Autoscaling in Energy Harvesting Mobile Edge Computing
Xu, J., Chen, L., & Ren, S. (2017) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2017.2725277 · Cited by 368
Efficient Solution Algorithms for Factored MDPs
Guestrin, C., Koller, D., Parr, R., & Venkataraman, S. (2003) · Cited by 384
Efficient Reductions for Imitation Learning
Ross, S., & Bagnell, D. (2018) · doi:10.1184/r1/6554699 · Cited by 366
Kernel-Based Reinforcement Learning
Ormoneit, D., & Sen, S. (2017) · Springer eBooks · doi:10.1007/978-1-4899-7687-1_100235 · Cited by 376
Optimality of Myopic Sensing in Multichannel Opportunistic Access
Ahmad, S. H. A., Liu, M., Javidi, T., Zhao, Q. Q., & Krishnamachari, B. (2009) · IEEE Transactions on Information Theory · doi:10.1109/tit.2009.2025561 · Cited by 364
Even-Dar, E., & Mansour, Y. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44581-1_39 · Cited by 373
Stochastic dynamic programming with factored representations
Boutilier, C., Dearden, R., & Goldszmidt, M. (2000) · Artificial Intelligence · doi:10.1016/s0004-3702(00)00033-3 · Cited by 361
Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation
Dayan, P., & Berridge, K. C. (2014) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0277-8 · Cited by 357
Exploiting Structure in Policy Construction
Boutilier, C. E., Dearden, R., & Goldszmidt, M. (1995) · Cited by 350
Metadata
- Authors
- Hazeghi, K., & Puterman, M. L.
- ORCID
- Puterman, Martin L.
- Venue
- Journal of the American Statistical Association
- Publisher
- JSTOR
- Published
- 1995-03
- Volume
- 90
- Issue
- 429
- Pages
- 392
- ISSN
- 0162-1459 (ISSN-L)
- OpenAlex
- W2119567691
- Cited by
- 8,037
- References
- 12
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 86 · 2023: 145 · 2022: 233 · 2021: 616 · 2020: 674 · 2019: 595 · 2018: 480 · 2017: 459 · 2016: 450 · 2015: 507 · 2014: 451 · 2013: 420 · 2012: 392
- Type
- book review
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- Markov decision process, stochastic dynamic programming, discrete-time models, infinite horizon model, finite horizon model, Bellman equation, optimality criteria, average reward criterion, constrained models, sequential decision making under uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
