On Solving a Stochastic Shortest-Path Markov Decision Process as Probabilistic Inference
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Previous work on planning as active inference addresses finite horizon problems and solutions valid for online planning. We propose solving the general Stochastic Shortest-Path Markov Decision Process (SSP MDP) as probabilistic inference. Furthermore, we discuss online and offline methods for planning under uncertainty. In an SSP MDP, the horizon is indefinite and unknown a priori. SSP MDPs generalize finite and infinite horizon MDPs and are widely used in the artificial intelligence community. Additionally, we highlight some of the differences between solving an MDP using dynamic programming approaches widely used in the artificial intelligence community and approaches used in the active inference community.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As concluded in [43], for the parts of a task subject to frequent unpredictable changes, computing locally an online plan as we do with active inference is preferable to offline policies.”
From Active Inference and Behavior Trees for Reactive Action Planning and Execution in Robotics (2023) · cites this paper as [43]
Citations
Cites
Adhi Harmoko, S., M.Komp, Jacquard, J. M., Zuse, K., & Eniac (2005) · Cited by 18,035
(2012) · Cited by 9,062
Koller, D., & Friedman, N. (2009) · The MIT Press eBooks · Cited by 7,555
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
(2009) · Cited by 6,153
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018) · Science · doi:10.1126/science.aar6404 · Cited by 3,562
Campbell, M., Hoane, A. J., & Hsu, F.-H. (2002) · Artificial Intelligence · doi:10.1016/s0004-3702(01)00129-1 · Cited by 1,226
Active inference: a process theory
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017) · Neural Computation · doi:10.1162/neco_a_00912 · Cited by 1,156
Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2017) · arXiv · doi:10.48550/arxiv.1712.01815 · Cited by 1,102
A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients
Grondman, I., Busoniu, L., Lopes, G. A. D., & Babuska, R. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2012.2218595 · Cited by 1,042
Research works printed but not linked: 2
- Sutton, R.S., McAllester, D.A., Singh, S.P., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: Advances in Neural Information Processing Systems, pp. 1057–1063 (2000)
- Verma, D., Rao, R.P.: Goal-based imitation as probabilistic inference over graphical models. In: Advances in Neural Information Processing Systems, pp. 1393–1400 (2006)
Cited by
Active Inference and Behavior Trees for Reactive Action Planning and Execution in Robotics
Pezzato, C., Corbato, C. H., Bonhof, S., & Wisse, M. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3226144 · Cited by 44
“As concluded in [43], for the parts of a task subject to frequent unpredictable changes, computing locally an online plan as we do with active inference is preferable to offline policies.”
BackgroundKamp, M., Koprinska, I., Bibal, A., Bouadi, T., Frénay, B., Galárraga, L. A., Jose Oramas, M., Adilova, L., Krishnamurthy, Y., Kang, B., et al. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-93736-2 · Cited by 39
Kamp, M., Koprinska, I., Bibal, A., Bouadi, T., Frénay, B., Galárraga, L. A., Jose Oramas, M., Adilova, L., Krishnamurthy, Y., Kang, B., et al. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-93733-1 · Cited by 18
Unbiased Active Inference for Classical Control
Baioumy, M., Pezzato, C., Ferrari, R., & Hawes, N. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981095 · Cited by 9
Self-citation
Metadata
- Authors
- Baioumy, M., Lacerda, B., Duckworth, P., & Hawes, N.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Baioumy, Mohamed
- Venue
- Communications in Computer and Information Science
- Publisher
- Springer International Publishing
- Published
- 2021
- Pages
- 819-829
- ISSN
- 1865-0929 (ISSN-L)
- 1865-0937
- OpenAlex
- W3199477924
- Cited by
- 4
- References
- 33
- Citations by year
- 2023: 1 · 2022: 1 · 2021: 2
- FWCI
- 0.46
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Communications in computer and information science (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
- Keywords
- active inference, probabilistic inference, planning under uncertainty, online planning, offline planning, dynamic programming, finite-horizon MDPs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: On Solving a Stochastic Shortest-Path Markov Decision Process as Probabilistic Inference. https://app.cochat.ai/papers/10.1007/978-3-030-93736-2_58
- The paper
- Baioumy, M., Lacerda, B., Duckworth, P., & Hawes, N. (2021). On Solving a Stochastic Shortest-Path Markov Decision Process as Probabilistic Inference. Communications in Computer and Information Science. https://doi.org/10.1007/978-3-030-93736-2_58
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.