A Bayesian exploration-exploitation approach for optimal online sensing and planning with a visually guided mobile robot
The DOI matches its Crossref record; 59 of the 62 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Ideally, a robot executing an active SLAM algorithm should evaluate every possible action in the robot and map space, but the computational complexity of the evaluation grows exponentially with the search space which proves to be computationally intractable in real applications [24, 169].”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [169]
Citations
Cites
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
(1995) · Cited by 10,558
Efficient Global Optimization of Expensive Black-Box Functions
Jones, D. R., Schonlau, M., & Welch, W. J. (1998) · Journal of Global Optimization · doi:10.1023/a:1008306431147 · Cited by 8,070 · cited 4×
“The objective of Bayesian optimization is to find the minimum of the cost function with as few cost evaluations as possible [Kushner (1964); Jones et al (1998); Locatelli (1997); Mockus et al (1978)].”
BackgroundMethodInfluentialPredictive Control With Constraints
Maciejowski, J. M. (2001) · Cambridge University Engineering Department Publications Database · Cited by 4,368
Simultaneous localization and mapping: part I
Durrant-Whyte, H., & Bailey, T. (2006) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2006.1638022 · Cited by 4,199
Predictive control with constraints
Camacho, E. F. (2003) · Automatica · doi:10.1016/s0005-1098(03)00056-6 · Cited by 3,530
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Optimization—Theory and Applications
Cesari, L. (1983) · Springer eBooks · doi:10.1007/978-1-4613-8165-5 · Cited by 2,559
Simultaneous localization and mapping (SLAM): part II
Bailey, T., & Durrant-Whyte, H. (2006) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2006.1678144 · Cited by 2,554
Marker tracking and HMD calibration for a video-based augmented reality conferencing system
Kato, H., & Billinghurst, M. (2003) · Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99) · doi:10.1109/iwar.1999.803809 · Cited by 2,194 · cited 1×
“The map is built using fiducial landmarks with the ARToolkit Tracker of Kato and Billinghurst (1999).”
MethodA Taxonomy of Global Optimization Methods Based on Response Surfaces
Jones, D. R. (2001) · Journal of Global Optimization · doi:10.1023/a:1012771025575 · Cited by 2,159 · cited 2×
“The objective of Bayesian optimization is to find the minimum of the cost function with as few cost evaluations as possible [Kushner (1964); Jones et al (1998); Locatelli (1997); Mockus et al (1978)].”
BackgroundMethodLipschitzian optimization without the Lipschitz constant
Jones, D. R., Perttunen, C. D., & Stuckman, B. E. (1993) · Journal of Optimization Theory and Applications · doi:10.1007/bf00941892 · Cited by 1,976 · cited 2×
“To optimize the expected improvement function, we used the DIRect algorithm [Jones et al (1993); Finkel (2003); Gablonsky (2001)], though other methods such as sequential quadratic programming could also be adopted.”
MethodBayesian Experimental Design: A Review
Chaloner, K., & Verdinelli, I. (1995) · Statistical Science · doi:10.1214/ss/1177009939 · Cited by 1,839 · cited 5×
“This choice of cost function is a sensible one when the objective is to minimize the uncertainty in the model parameters [Chaloner and Verdinelli (1995); Sim and Roy (2005)].”
BackgroundInfluentialThe Optimal Control of Partially Observable Markov Processes over a Finite Horizon
Smallwood, R. D., & Sondik, E. J. (1973) · Operations Research · doi:10.1287/opre.21.5.1071 · Cited by 1,479 · cited 2×
“The discretized POMDP is too large for stochastic dynamic programming techniques derived from the seminal work of Smallwood and Sondik (1973).”
BackgroundMethodA New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise
Kushner, H. J. (1964) · Journal of Basic Engineering · doi:10.1115/1.3653121 · Cited by 1,082 · cited 2×
“Bayesian optimization has a long history, starting with the seminal work of Kushner (1964) with one-dimensional Wiener processes.”
BackgroundMethodReinforcement learning of motor skills with policy gradients
Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831 · cited 1×
“However, since the dynamic model is non-differentiable, one cannot use gradient-based stochastic approximations (Konda and Tsitsiklis 2003; Singh et al. 2005) or policy gradient methods (Baxter and Bartlett 2001; Peters and Schaal 2008a, 2008b ) to update the parameters of the policy.”
MethodPeters, J., & Schaal, S. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.11.026 · Cited by 738 · cited 1×
“However, since the dynamic model is non-differentiable, one cannot use gradient-based stochastic approximations (Konda and Tsitsiklis 2003; Singh et al. 2005) or policy gradient methods (Baxter and Bartlett 2001; Peters and Schaal 2008a, 2008b ) to update the parameters of the policy.”
MethodKonda, V. R., & Tsitsiklis, J. N. (2003) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012901385691 · Cited by 709 · cited 2×
“However, since the dynamic model is non-differentiable, one cannot use gradient-based stochastic approximations [Konda and Tsitsiklis (2003); Singh et al (2005)] or policy gradient methods [Baxter and Bartlett (2001); Peters and Schaal (2008a,b)] to update the parameters of the policy.”
BackgroundMethodYARP: Yet Another Robot Platform
Metta, G., Fitzpatrick, P., & Natale, L. (2006) · International Journal of Advanced Robotic Systems · doi:10.5772/5761 · Cited by 640 · cited 2×
“However, the system was built using the YARP middleware by Metta et al (2006) and therefore it allows us to use local feature detectors, such as SIFT or SURF.”
MethodConsistency of the EKF-SLAM Algorithm
Bailey, T., Nieto, J., Guivant, J., Stevens, M., & Nebot, E. (2006) · 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2006.281644 · Cited by 566 · cited 2×
“Path planning also plays a key role in the theoretical and practical convergence of SLAM algorithms [Bailey et al (2006); Martinez-Cantin et al (2006)].”
BackgroundInformation Gain-based Exploration Using Rao-Blackwellized Particle Filters
Stachniss, C., Grisetti, G., & Burgard, W. (2005) · Robotics: Science and Systems I · doi:10.15607/rss.2005.i.009 · Cited by 542 · cited 4×
“Online path planning is essential for proper simultaneous localization and mapping (SLAM) [Sim and Roy (2005); Stachniss et al (2005)].”
BackgroundInfluentialInfinite-Horizon Policy-Gradient Estimation
Baxter, J., & Bartlett, P. L. (2001) · Cited by 525
Recursive Bayesian Estimation : Navigation and Tracking Applications
Bergman, N. (1999) · Cited by 523
Policy Gradient Methods for Robotics
Peters, J., & Schaal, S. (2006) · 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2006.282564 · Cited by 498 · cited 2×
“In policy search, the approximated cost function is used to update the policy parameters θ . Typically, this is done by following stochastic gradients (Baxter and Bartlett 2001; Peters and Schaal 2006 ).”
BackgroundMethodGlobal versus local search in constrained optimization of computer models
Schonlau, M., Welch, W. J., & Jones, D. R. (1998) · Institute of Mathematical Statistics Lecture Notes - Monograph Series · doi:10.1214/lnms/1215456182 · Cited by 435 · cited 2×
“Several refinements of this infill function have been proposed in Schonlau et al (1998) and Sasena (2002).”
BackgroundMethodPEGASUS: A Policy Search Method for Large MDPs and POMDPs
Ng, A. Y., & Jordan, M. I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3878 · Cited by 360 · cited 2×
“To overcome these difficulties, we adopt the direct policy search method for solving POMDPs [Williams (1992); Baxter and Bartlett (2001); Ng and Jordan (2000)]; see also the papers of [Howard et al (2009); Stolle and Atkeson (2009); N. Vlassis and Piperidis (2009)] in this special issue.”
MethodSasena, M. J. (2002) · Cited by 350
Infinite-Horizon Policy-Gradient Estimation
Baxter, J., & Bartlett, P. L. (2001) · Journal of Artificial Intelligence Research · doi:10.1613/jair.806 · Cited by 343
Automatic gait optimization with Gaussian process regression
Lizotte, D. J., Wang, T., Bowling, M., & Schuurmans, D. (2007) · International Joint Conference on Artificial Intelligence · Cited by 335
Efficient Informative Sensing using Multiple Robots
Singh, A., Krause, A., Guestrin, C., & Kaiser, W. J. (2009) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2674 · Cited by 255 · cited 1×
“In Singh et al (2009), a sound way of exploiting sub-modularity in these planning and sensing domains is proposed.”
BackgroundReinforcement learning for robot soccer
Riedmiller, M., Gabel, T., Hafner, R., & Lange, S. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9120-4 · Cited by 255 · cited 2×
“Most existing reinforcement learning techniques are unable to cope with our high-dimensional, non-differentiable, continuous POMDPs (see [Riedmiller et al (2009)] in this special issue for an introduction to reinforcement learning).”
BackgroundEfficient Informative Sensing using Multiple Robots
Singh, A., Krause, A., Guestrin, C., & Kaiser, W. J. (2010) · Cited by 245
Global A-Optimal Robot Exploration in SLAM
Sim, R., & Roy, N. (2006) · Proceedings of the 2005 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2005.1570193 · Cited by 238 · cited 4×
“Online path planning is essential for proper simultaneous localization and mapping (SLAM) [Sim and Roy (2005); Stachniss et al (2005)].”
BackgroundInfluentialMultisensor resource deployment using posterior Cramer-Rao bounds
Hernandez, M. L., Kirubarajan, T., & Bar-Shalom, Y. (2004) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2004.1309993 · Cited by 219 · cited 2×
“To optimize the expected improvement function, we used the DIRect algorithm [Jones et al (1993); Finkel (2003); Gablonsky (2001)], though other methods such as sequential quadratic programming could also be adopted.”
BackgroundMethodPractical Bayesian optimization
Lizotte, D. J. (2008) · ERA: Education and Research Archive (University of Alberta) · doi:10.7939/r3-379f-gw15 · Cited by 215
Direct optimization algorithm user guide
Finkel, D. (2003) · NCSU Libraries Repository (North Carolina State University Libraries) · Cited by 181
Efficient Planning of Informative Paths for Multiple Robots
Singh, A., Singh, A., Krause, A., Krause, A., Guestrin, C., Kaiser, W., & Batalin, M. A. (2006) · eScholarship (California Digital Library) · Cited by 168
Observability analysis and active control for airborne SLAM
Bryson, M., & Sukkarieh, S. (2008) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2008.4517003 · Cited by 159 · cited 2×
“…et al (2005) and Singh et al (2007), a small set of actions and myopic planning is adopted in [Stachniss et al (2005), Vidal-Callej et al (2006),Bryson and Sukkarieh (2008)], and discretization of the state and/or actions spaces is required in Hernandez (2004), Kollar and Roy (2008) and Sim…”
BackgroundMethodActive Policy Learning for Robot Planning and Exploration under Uncertainty
Martinez-Cantin, R., de Freitas, N., Doucet, A., & Castellanos, J. (2007) · Robotics: Science and Systems III · doi:10.15607/rss.2007.iii.041 · Cited by 140 · cited 3×
“Although we actually learn the mean function in the manner proposed in Martinez-Cantin et al. (2007a) , for presentation clarity let us assume that it is the zero function as it is often the case in the machine learning literature.”
MethodSelf-citationTrajectory Optimization using Reinforcement Learning for Map Exploration
Kollar, T., & Roy, N. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364907087426 · Cited by 139 · cited 3×
“However, since the dynamic model is non-differentiable, one cannot use gradient-based stoc hastic approximations [Konda and Tsitsiklis (2003); Singh et a l (2005)] or policy gradient methods [Baxter and Bartlett (20 01); Peters and Schaal (2008a,b)] to update the parameters of the policy. Instead, we propose here the…”
MethodHey, A. M. (1979) · Journal of the Operational Research Society · doi:10.1057/jors.1979.201 · Cited by 136
Active Preference Learning with Discrete Choice Data
Brochu, E., de Freitas, N., & Ghosh, A. (2007) · Cited by 125
Self-citationBayesian Algorithms for One-Dimensional Global Optimization
Locatelli, M. (1997) · Journal of Global Optimization · doi:10.1023/a:1008294716304 · Cited by 122 · cited 4×
“The objective of Bayesian optimization is to find the minimum of the cost function with as few cost evaluations as possible (Kushner 1964; Jones et al. 1998; Locatelli 1997; Mockus et al. 1978).”
BackgroundInfluentialLearning model-free robot control by a Monte Carlo EM algorithm
Vlassis, N., Toussaint, M., Kontes, G., & Piperidis, S. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9132-0 · Cited by 71
Active control for single camera SLAM
Vidal-Calleja, T., Davison, A. J., Andrade-Cetto, J., & Murray, D. W. (2006) · Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006. · doi:10.1109/robot.2006.1641988 · Cited by 65 · cited 1×
“and Williams (2006) for details on Gaussian process regression.”
BackgroundOptimal Sensor Trajectories in Bearings-Only Tracking
Hernandez, M. L. (2004) · Cited by 55
Martínez-Cantín, R., de Freitas, N., & Castellanos, J. A. (2007) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2007.363681 · Cited by 46 · cited 3×
“Path planning also plays a key role in the theoretical and practical convergence of SLAM algorithms (Bailey et al. 2006; Martinez-Cantin et al. 2006 ).”
BackgroundMethodSelf-citationGlobal optimization based on a statistical model and simplicial partitioning
Žilinskas, A., & Žilinskas, J. (2002) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(02)00206-7 · Cited by 44 · cited 1×
“…by the highest infill in the top plot produces the GP fit in the plot shown below. convergence proofs for a simplification of the algorithm using simplicial partitioning in higher dimensions [Zilinskas and Zilinskas (2002)] and, more recently, for GPs with Matern kernels in Vazquez and Bect (2008).”
MethodOptimal observer trajectoryin bearings-only trackingfor manoeuvring sources
Trémois, O., & Cadre, J.-P. L. (1999) · IEE Proceedings - Radar Sonar and Navigation · doi:10.1049/ip-rsn:19990262 · Cited by 34
A novel method for learning policies from variable constraint data
Howard, M., Klanke, S., Gienger, M., Goerick, C., & Vijayakumar, S. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9129-8 · Cited by 30
SMC Samplers for Bayesian Optimal Nonlinear Design
Kück, H., de Freitas, N., & Doucet, A. (2006) · doi:10.1109/nsspw.2006.4378829 · Cited by 30 · cited 1×
“Utility (reward) functions that depend on the belief state are commonplace in the field of experimental design (Chaloner and Verdinelli 1995; Kueck et al. 2006 ).”
BackgroundSelf-citationFinding and transferring policies using stored behaviors
Stolle, M., & Atkeson, C. G. (2010) · Autonomous Robots · doi:10.1007/s10514-010-9191-2 · Cited by 20 · cited 1×
“To overcome these difficulties, we adopt the direct policy search method for solving POMDPs (Williams 1992; Baxter and Bartlett 2001; Ng and Jordan 2000 ); see also the papers of Howard et al. (2009); Stolle and Atkeson (2009) ; Vlassis et al. (2009) in this special issue.”
MethodSimulation-Based Optimal Sensor Scheduling with Application to Observer Trajectory Planning
Singh, S. S., Kantas, N., Arnaud Doucet, B., Vo, B., & Evans, R. J. (2006) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2005.1583338 · Cited by 16
Inferring a probability distribution function for the pose of a sensor network using a mobile robot
Meger, D., Marinakis, D., Rekleitis, I. M., & Dudek, G. (2009) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2009.5152800 · Cited by 15 · cited 2×
“(Bergman 1999; Paris and Le Cadre 2002; Meger et al. 2009; Hernandez et al. 2004; Singh et al. 2005).”
BackgroundOn the convergence of the expected improvement algorithmPreprint
Vázquez, E., & Bect, J. (2007) · arXiv (Cornell University) · Cited by 13
Trajectory planning for multiple robots in bearing-only target localisation
Leung, C., Huang, S., Dissanayake, G., & Furukawa, T. (2005) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2005.1545322 · Cited by 11 · cited 2×
“…full observability is assumed in Paris and Le Cadre (2002) and Sim and Roy (2005), known robot location and discrete actions are assumed in Leung et al (2005) and Singh et al (2007), a small set of actions and myopic planning is adopted in [Stachniss et al (2005), Vidal-Callej et al…”
BackgroundConvergence properties of the expected improvement algorithmPreprint
Vázquez, E., & Bect, J. (2007) · arXiv (Cornell University) · doi:10.48550/arxiv.0712.3744 · Cited by 10
Active map learning for robots:insights into statistical consistency
Martínez-Cantín, R. (2008) · Cited by 10
Planification for Terrain- Aided Navigation
de Beaulieu, C., & Cadre, J. L. (2002) · Cited by 8
Research works printed but not linked: 2
- Gablonsky, J. (2001). Modification of the DIRECT algorithm. PhD thesis, Department of Mathematics, North Carolina State University, Raleigh, North Carolina.
- Martinez-Cantin, R., de Freitas, N., Doucet, A., & Castellanos, J. (2007b). Active policy learning for robot planning and exploration under uncertainty. In Robotics: science and systems (RSS), 2007. · DOI · Paper page
Cited by
Taking the Human Out of the Loop: A Review of Bayesian Optimization
Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & de Freitas, N. (2016) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2494218 · Cited by 6,049
“This interactive Bayesian optimization strategy is particulary effective as humans can be very good at comparing examples, but unable to produce an objective function whose optimum is the example of interest.”
BackgroundPast, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“but tractable solutions for active SLAM include Bayesian Optimization [169] or efficient Gaussian beliefs propagation [195], among others.”
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Brochu, E., Cora, V. M., & de Freitas, N. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.2599 · Cited by 2,011
Self-citationActive learning of inverse models with intrinsically motivated goal exploration in robots
Baranes, A., & Oudeyer, P.-Y. (2013) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2012.05.008 · Cited by 455
Metadata
- Authors
- Martinez-Cantin, R., de Freitas, N., Brochu, E., Castellanos, J., & Doucet, A.
- Affiliations
- University of British Columbia (CA)
- Universidad de Zaragoza (ES)
- Corresponding author
- Brochu, Eric
- ORCID
- Castellanos, José
- Venue
- Autonomous Robots
- Published
- 2009-08
- Volume
- 27
- Issue
- 2
- Pages
- 93-103
- ISSN
- 0929-5593 (ISSN-L)
- 1573-7527
- OpenAlex
- W2096649264
- Cited by
- 217
- References
- 62
- Citations by year
- 2025: 6 · 2024: 9 · 2023: 15 · 2022: 11 · 2021: 27 · 2020: 27 · 2019: 13 · 2018: 20 · 2017: 9 · 2016: 17 · 2015: 13 · 2014: 8 · 2013: 11 · 2012: 11
- FWCI
- 11.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Autonomous Robots (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robotic Path Planning Algorithms
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Keywords
- online path planning, exploration-exploitation balance, Bayesian optimization, partially observable Markov decision processes, mobile robot localization, visual guidance, real-time planning, active vision, sequential design, dynamic sensing, sensor calibration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.