The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Anytime Point-Based Approximations for Large POMDPs
Open accessAbstract
The Partially Observable Markov Decision Process has long been recognized as a rich framework for real-world planning and control problems, especially in robotics. However exact solutions in this framework are typically computationally intractable for all but the smallest problems. A well-known technique for speeding up POMDP solving involves performing value backups at specific belief points, rather than over the entire belief simplex. The efficiency of this approach, however, depends greatly on the selection of points. This paper presents a set of novel techniques for selecting informative belief points which work well in practice. The point selection procedure is combined with point-based value backups to form an effective anytime POMDP algorithm called Point-Based Value Iteration (PBVI). The first aim of this paper is to introduce this algorithm and present a theoretical analysis justifying the choice of belief selection technique. The second aim of this paper is to provide a thorough empirical comparison between PBVI and other state-of-the-art POMDP methods, in particular the Perseus algorithm, in an effort to highlight their similarities and differences. Evaluation is performed using both standard POMDP domains and realistic robotic tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A New Approach to Linear Filtering and Prediction Problems
Kalman, R. E. (1960) · Journal of Basic Engineering · doi:10.1115/1.3662552 · Cited by 31,751
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Stochastic Processes and Filtering Theory
(1970) · Mathematics in Science and Engineering · doi:10.1016/s0076-5392(09)x6022-4 · Cited by 7,302
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving
Fikes, R. E., & Nilsson, N. J. (1971) · Artificial Intelligence · doi:10.1016/0004-3702(71)90010-5 · Cited by 4,882
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149
Non-Uniform Random Variate Generation
Devroye, L. (1986) · Springer eBooks · doi:10.1007/978-1-4613-8643-8 · Cited by 4,041
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,515
A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition
RABINER, L. R. (1990) · Readings in Speech Recognition · doi:10.1016/b978-0-08-051584-7.50027-9 · Cited by 1,869
Robust Monte Carlo localization for mobile robots
Thrun, S., Fox, D., Burgard, W., & Dellaert, F. (2001) · Artificial Intelligence · doi:10.1016/s0004-3702(01)00069-8 · Cited by 1,809
Self-citationFast planning through planning graph analysis
Blum, A. L., & Furst, M. L. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(96)00047-1 · Cited by 1,564
Neuro-Dynamic Programming: NDP
Bertsekas, D. P. (2001) · Encyclopedia of Optimization · doi:10.1007/0-306-48332-7_333 · Cited by 1,466
Planning for conjunctive goals
Chapman, D. (1987) · Artificial Intelligence · doi:10.1016/0004-3702(87)90092-0 · Cited by 1,112
Decision-Theoretic Planning: Structural Assumptions and Computational Leverage
Boutilier, C., Dean, T., & Hanks, S. (1999) · Journal of Artificial Intelligence Research · doi:10.1613/jair.575 · Cited by 1,074
Optimal control of Markov processes with incomplete state information
Åström, K. J. (1965) · Journal of Mathematical Analysis and Applications · doi:10.1016/0022-247x(65)90154-x · Cited by 854
Experiences with an interactive museum tour-guide robot
Burgard, W., Cremers, A. B., Fox, D., Hähnel, D., Lakemeyer, G., Schulz, D., Steiner, W., & Thrun, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00070-3 · Cited by 818
Self-citationUCPOP: a sound, complete, partial order planner for ADL
Penberthy, J. S., & Weld, D. S. (1992) · Cited by 676
Towards robotic assistants in nursing homes: Challenges and results
Pineau, J., Montemerlo, M., Pollack, M., Roy, N., & Thrun, S. (2003) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(02)00381-0 · Cited by 640
Self-citationA survey of algorithmic methods for partially observed Markov decision processes
Lovejoy, W. S. (1991) · Annals of Operations Research · doi:10.1007/bf02055574 · Cited by 532
Value-Function Approximations for Partially Observable Markov Decision Processes
Hauskrecht, M. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.678 · Cited by 527
Perseus: Randomized point-based value iteration for POMDPs
Spaan, M. T. J., & Vlassis, N. A. (2005) · UvA-DARE (University of Amsterdam) · Cited by 526
McAllester, D., & Rosenblitt, D. (1991) · Cited by 525
Handbook of Industrial Robotics
(1999) · Wiley eBooks · doi:10.1002/9780470172506 · Cited by 486
Tractable Inference for Complex Stochastic Processes
Boyen, X., & Koller, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.7362 · Cited by 471
The optimal control of partially observable Markov processes
Sondik, E. J. (1971) · Defense Technical Information Center (DTIC) · Cited by 440
(2004) · doi:10.5555/1036843.1036906 · Cited by 417
A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms
Monahan, G. E., & MONAHANt, G. E. (2007) · Cited by 368
Incremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processes
Cassandra, A. R., Littman, M. L., & Zhang, N. L. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.1525 · Cited by 361
Algorithms for Sequential Decision Making
Littman, M. L. (1996) · Cited by 341
Montemerlo, M., Roy, N., & Thrun, S. (2004) · Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453) · doi:10.1109/iros.2003.1249235 · Cited by 326
Self-citationComputationally Feasible Bounds for Partially Observed Markov Decision Processes
Lovejoy, W. S. (1991) · Operations Research · doi:10.1287/opre.39.1.162 · Cited by 307
Bounded Finite State Controllers
Poupart, P., & Boutilier, C. E. (2003) · Cited by 167
Probabilistic temporal reasoning
Dean, T., & Kanazawa, K. (1988) · National Conference on Artificial Intelligence · Cited by 165
A survey of solution techniques for the partially observed Markov decision process
White, C. C. (1991) · Annals of Operations Research · doi:10.1007/bf02204836 · Cited by 163
Algorithms for partially observable Markov decision processes
Cheng, H.-T. (2010) · cIRcle (University of British Columbia) · doi:10.14288/1.0098252 · Cited by 148
The application of robotics to a mobility aid for the elderly blind
Lacey, G., & Dawson‐Howe, K. (1998) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(98)00011-6 · Cited by 148
Experiences with an Interactive Museum Tour-Guide Robot
Burgard, W., Cremers, A. B., Fox, D., Haehnel, D., & Lakemeyer, G. (1998) · doi:10.21236/ada352295 · Cited by 146
Speeding Up the Convergence of Value Iteration in Partially Observable Markov Decision Processes
Zhang, N. L., & Zhang, W. (2001) · Journal of Artificial Intelligence Research · doi:10.1613/jair.761 · Cited by 140
Exponential Family PCA for Belief Compression in POMDPs
Roy, N., & Gordon, G. J. (2002) · Cited by 116
Self-citationValue-Directed Compression of POMDPs
Poupart, P., & Boutilier, C. E. (2002) · Cited by 103
An improved grid-based approximation algorithm for POMDPs
Zhou, R., & Hansen, E. A. (2001) · Cited by 100
A heuristic variable grid solution method for POMDPs
Brafman, R. I. (1997) · Cited by 86
Incremental methods for computing bounds in partially observable Markov decision processes
Hauskrecht, M. (1997) · Cited by 63
Planning technology for intelligent cognitive orthotics
Pollack, M. E. (2002) · Cited by 55
Stochastic local search for POMDP controllers
Braziunas, D., & Boutilier, C. E. (2004) · TSpace (University of Toronto) · Cited by 52
Locating moving entities in indoor environments with teams of mobile robots
Rosencrantz, M., Gordon, G. J., & Thrun, S. (2003) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/860575.860613 · Cited by 48
Self-citationAn epsilon-Optimal Grid-Based Algorithm for Partially Observable Markov Decision Processes
Bonet, B. (2002) · International Conference on Machine Learning · Cited by 38
A fast heuristic algorithm for decision-theoretic planning
Poon, L. K. M. (2001) · doi:10.14711/thesis-b711441 · Cited by 30
Value-Directed Belief State Approximation for POMDPsPreprint
Poupart, P., & Boutilier, C. E. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3887 · Cited by 27
A fast point-based algorithm for POMDPs
Vlassis, N. A., & Spaan, M. T. J. (2004) · UvA-DARE (University of Amsterdam) · Cited by 26
Paquet, E. (2006) · Cited by 22
Cited by
Reinforcement Learning and Dynamic Programming Using Function Approximators
Busoniu, L., Babuska, R., De Schutter, B., & Ernst, D. (2017) · CRC Press eBooks · doi:10.1201/9781439821091 · Cited by 919
Online Planning Algorithms for POMDPs
Ross, S., Pineau, J., Paquet, S., & Chaib-draa, B. (2008) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2567 · Cited by 496
Self-citationAdaptive Submodularity: Theory and Applications in Active Learning and Stochastic Optimization
Golovin, D., & Krause, A. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1003.3967 · Cited by 461
A Survey of Multi-Objective Sequential Decision-Making
Roijers, D. M., Vamplew, P., Whiteson, S., & Dazeley, R. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3987 · Cited by 461
A survey of point-based POMDP solvers
Shani, G., Pineau, J., & Kaplow, R. (2013) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-012-9200-2 · Cited by 422
Self-citation
Metadata
- Authors
- Pineau, J., Gordon, G., & Thrun, S.
- Venue
- Journal of Artificial Intelligence Research
- Publisher
- AI Access Foundation
- Published
- 2006-11-26
- Volume
- 27
- Pages
- 335-380
- ISSN
- 1076-9757 (ISSN-L)
- OpenAlex
- W2110962519
- Cited by
- 370
- References
- 54
- FWCI
- 13.39
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.