The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Perseus: Randomized Point-based Value Iteration for POMDPs
Open accessAbstract
Partially observable Markov decision processes (POMDPs) form an attractive and principled framework for agent planning under uncertainty. Point-based approximate techniques for POMDPs compute a policy based on a finite set of points collected in advance from the agent's belief space. We present a randomized point-based value iteration algorithm called Perseus. The algorithm performs approximate value backup stages, ensuring that in each backup stage the value of each point in the belief set is improved; the key observation is that a single backup may improve the value of many belief points. Contrary to other point-based methods, Perseus backs up only a (randomly selected) subset of points in the belief set, sufficient for improving the value of each belief point in the set. We show how the same idea can be extended to dealing with continuous action spaces. Experimental results show the potential of Perseus in large scale POMDP problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Parallel and Distributed Computation: Numerical Methods
Bertsekas, D. P., & Tsitsiklis, J. N. (1989) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 5,293
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149
The Complexity of Markov Decision Processes
Papadimitriou, C. H., & Tsitsiklis, J. N. (1987) · Mathematics of Operations Research · doi:10.1287/moor.12.3.441 · Cited by 1,329
Point-based value iteration: an anytime algorithm for POMDPs
Pineau, J., Gordon, G., & Thrun, S. (2003) · Cited by 918
Optimal control of Markov processes with incomplete state information
Åström, K. J. (1965) · Journal of Mathematical Analysis and Applications · doi:10.1016/0022-247x(65)90154-x · Cited by 854
Prioritized sweeping: Reinforcement learning with less data and less time
Moore, A. W., & Atkeson, C. G. (1993) · Machine Learning · doi:10.1007/bf00993104 · Cited by 823
Learning policies for partially observable environments: Scaling up
Littman, M. L., Cassandra, A. R., & Kaelbling, L. P. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50052-9 · Cited by 650
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (2005) · Cited by 588
Value-Function Approximations for Partially Observable Markov Decision Processes
Hauskrecht, M. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.678 · Cited by 527
(2004) · doi:10.5555/1036843.1036906 · Cited by 417
A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms
Monahan, G. E., & MONAHANt, G. E. (2007) · Cited by 368
Incremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processes
Cassandra, A. R., Littman, M. L., & Zhang, N. L. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.1525 · Cited by 361
Computationally Feasible Bounds for Partially Observed Markov Decision Processes
Lovejoy, W. S. (1991) · Operations Research · doi:10.1287/opre.39.1.162 · Cited by 307
Madani, O., Hanks, S., & Condon, A. (1999) · National Conference on Artificial Intelligence · Cited by 305
Finding approximate POMDP solutions through belief compression
Roy, N., Gordon, G., & Thrun, S. (2005) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1496 · Cited by 254
Exploiting structure to efficiently solve large scale partially observable markov decision processes
Poupart, P. (2005) · Cited by 247
Solving transition independent decentralized Markov decision processes
Becker, R., Shlomo Zilberstein, B., Lesser, V., & Goldman, C. V. (2004) · doi:10.5555/1622487.1622500 · Cited by 226
The Optimal Control of Partially Observable Markov Decision Processes.
Sondik, E. J. (1971) · Medical Entomology and Zoology · Cited by 170
Approximate Solutions for Partially Observable Stochastic Games with Common Payoffs
Emery, R., Gordon, G., Schneider, J., & Thrun, S. (2004) · doi:10.1109/aamas.2004.67 · Cited by 167
(2005) · Discovery Research Portal (University of Dundee) · Cited by 158
Solving POMDPs by Searching the Space of Finite PoliciesPreprint
Meuleau, N., Kim, K.-E., Kaelbling, L. P., & Cassandra, A. R. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6720 · Cited by 156
Algorithms for partially observable Markov decision processes
Cheng, H.-T. (2010) · cIRcle (University of British Columbia) · doi:10.14288/1.0098252 · Cited by 148
Speeding Up the Convergence of Value Iteration in Partially Observable Markov Decision Processes
Zhang, N. L., & Zhang, W. (2001) · Journal of Artificial Intelligence Research · doi:10.1613/jair.761 · Cited by 140
Approximate Planning in Large POMDPs via Reusable Trajectories
Kearns, M. J., Mansour, Y., & Ng, A. Y. (1999) · Cited by 136
An improved grid-based approximation algorithm for POMDPs
Zhou, R., & Hansen, E. A. (2001) · Cited by 100
Solving POMDPs with continuous or large discrete observation spaces
Hoey, J., & Poupart, P. (2005) · Cited by 98
An online POMDP algorithm for complex multiagent environments
Paquet, S., Tobin, L., & Chaib-draa, B. (2005) · Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems · doi:10.1145/1082473.1082620 · Cited by 92
A heuristic variable grid solution method for POMDPs
Brafman, R. I. (1997) · Cited by 86
Generalized Markov Decision Processes: Dynamic-programming and Reinforcement-learning Algorithms
Ari, C. S., & Littman, M. L. (1996) · Cited by 70
Representing hierarchical POMDPs as DBNs for multi-scale robot localization
Theocharous, G., Murphy, K., & Kaelbling, L. P. (2004) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2004.1307288 · Cited by 63
Robot Planning in Partially Observable Continuous Domains
Porta, J. M., Spaan, M. T. J., & Vlassis, N. A. (2005) · Robotics: Science and Systems · doi:10.15607/rss.2005.i.029 · Cited by 50
Self-citationMarkovian Decision Processes with Probabilistic Observation of States
Satia, J., & Lave, R. E. (1973) · Management Science · doi:10.1287/mnsc.20.1.1 · Cited by 47
An epsilon-Optimal Grid-Based Algorithm for Partially Observable Markov Decision Processes
Bonet, B. (2002) · International Conference on Machine Learning · Cited by 38
Solving Factored POMDPs with Linear Value Functions
Guestrin, C., Koller, D., & Parr, R. (n.d.) · Cited by 37
A fast heuristic algorithm for decision-theoretic planning
Poon, L. K. M. (2001) · doi:10.14711/thesis-b711441 · Cited by 30
Propagating uncertainty in POMDP value iteration with Gaussian processes
Tuttle, E. R., & Ghahramani, Z. (2004) · Cambridge University Engineering Department Publications Database · Cited by 1
Cited by
POMDP-Based Statistical Spoken Dialog Systems: A Review
Young, S., Gasic, M., Thomson, B., & Williams, J. D. (2013) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2225812 · Cited by 882
Partially observable Markov decision processes for spoken dialog systems
Williams, J. D., & Young, S. (2007) · Computer Speech & Language · doi:10.1016/j.csl.2006.06.008 · Cited by 853
A survey of point-based POMDP solvers
Shani, G., Pineau, J., & Kaplow, R. (2013) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-012-9200-2 · Cited by 422
Partially Observable Markov Decision Processes
Spaan, M. T. J. (2012) · Adaptation, Learning, and Optimization · doi:10.1007/978-3-642-27645-3_12 · Cited by 315
Self-citationA˚ström, K. J., & Kumar, P. R. (2014) · Automatica · doi:10.1016/j.automatica.2013.10.012 · Cited by 304
Metadata
- Authors
- Spaan, M. T. J., & Vlassis, N.
- Venue
- Journal of Artificial Intelligence Research
- Publisher
- AI Access Foundation
- Published
- 2005-08-01
- Volume
- 24
- Pages
- 195-220
- ISSN
- 1076-9757 (ISSN-L)
- OpenAlex
- W2158907787
- Cited by
- 331
- References
- 36
- FWCI
- 9.96
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.