The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Perseus: Randomized Point-based Value Iteration for POMDPs
Open accessAbstract
Partially observable Markov decision processes (POMDPs) form an attractive and principled framework for agent planning under uncertainty. Point-based approximate techniques for POMDPs compute a policy based on a finite set of points collected in advance from the agent's belief space. We present a randomized point-based value iteration algorithm called Perseus. The algorithm performs approximate value backup stages, ensuring that in each backup stage the value of each point in the belief set is improved; the key observation is that a single backup may improve the value of many belief points. Contrary to other point-based methods, Perseus backs up only a (randomly selected) subset of points in the belief set, sufficient for improving the value of each belief point in the set. We show how the same idea can be extended to dealing with continuous action spaces. Experimental results show the potential of Perseus in large scale POMDP problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Parallel and Distributed Computation: Numerical Methods
Bertsekas, D. P., & Tsitsiklis, J. N. (1989) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 5,293
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149
The Complexity of Markov Decision Processes
Papadimitriou, C. H., & Tsitsiklis, J. N. (1987) · Mathematics of Operations Research · doi:10.1287/moor.12.3.441 · Cited by 1,329
Point-based value iteration: an anytime algorithm for POMDPs
Pineau, J., Gordon, G., & Thrun, S. (2003) · Cited by 918
Optimal control of Markov processes with incomplete state information
Åström, K. J. (1965) · Journal of Mathematical Analysis and Applications · doi:10.1016/0022-247x(65)90154-x · Cited by 854 · cited 2×
“Partially observable Markov decision processes (POMDPs) provide a rich mathematical framework for acting optimally in such partially observable and stochastic environments (Dynkin, 1965; ˚Astr¨om, 1965; Aoki, 1965; Sondik, 1971; Lovejoy, 1991; Kaelbling, Littman, & Cassandra, 1998).”
BackgroundPrioritized sweeping: Reinforcement learning with less data and less time
Moore, A. W., & Atkeson, C. G. (1993) · Machine Learning · doi:10.1007/bf00993104 · Cited by 823
Learning policies for partially observable environments: Scaling up
Littman, M. L., Cassandra, A. R., & Kaelbling, L. P. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50052-9 · Cited by 650
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (2005) · Cited by 588
Value-Function Approximations for Partially Observable Markov Decision Processes
Hauskrecht, M. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.678 · Cited by 527
(2004) · doi:10.5555/1036843.1036906 · Cited by 417
Cited by
POMDP-Based Statistical Spoken Dialog Systems: A Review
Young, S., Gasic, M., Thomson, B., & Williams, J. D. (2013) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2225812 · Cited by 882
Partially observable Markov decision processes for spoken dialog systems
Williams, J. D., & Young, S. (2007) · Computer Speech & Language · doi:10.1016/j.csl.2006.06.008 · Cited by 853
A survey of point-based POMDP solvers
Shani, G., Pineau, J., & Kaplow, R. (2013) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-012-9200-2 · Cited by 422
Partially Observable Markov Decision Processes
Spaan, M. T. J. (2012) · Adaptation, Learning, and Optimization · doi:10.1007/978-3-642-27645-3_12 · Cited by 315
Self-citationA˚ström, K. J., & Kumar, P. R. (2014) · Automatica · doi:10.1016/j.automatica.2013.10.012 · Cited by 304
Hoey, J., Poupart, P., Bertoldi, A. V., Craig, T., Boutilier, C., & Mihailidis, A. (2010) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2009.06.008 · Cited by 229
Identifying and Avoiding Confusion in Dialogue with People with Alzheimer's Disease
Chinaei, H., Currie, L. C., Danks, A., Lin, H., Mehta, T., & Rudzicz, F. (2017) · Computational Linguistics · doi:10.1162/coli_a_00290 · Cited by 30
“We evaluate the learned POMDP policies in 1,000 simulation runs using the default simulation in Perseus (Spaan and Vlassis 2005) in which the true state of the user is estimated from the provided transition function, and the observations are sampled from the provided observation function.”
MethodOptimal policies for Bayesian olfactory search in turbulent flows
Heinonen, R. A., Biferale, L., Celani, A., & Vergassola, M. (2023) · Physical Review E · doi:10.1103/physreve.107.055105 · Cited by 19
Learning State-Variable Relationships in POMCP: A Framework for Mobile Robots
Zuccotto, M., Piccinelli, M., Castellini, A., Marchesini, E., & Farinelli, A. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.819107 · Cited by 2
Metadata
- Authors
- Spaan, M. T. J., & Vlassis, N.
- Venue
- Journal of Artificial Intelligence Research
- Publisher
- AI Access Foundation
- Published
- 2005-08-01
- Volume
- 24
- Pages
- 195-220
- ISSN
- 1076-9757 (ISSN-L)
- OpenAlex
- W2158907787
- Cited by
- 331
- References
- 36
- FWCI
- 9.96
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.