A Bayesian exploration-exploitation approach for optimal online sensing and planning with a visually guided mobile robot
The DOI matches its Crossref record; 60 of the 62 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Ideally, a robot executing an active SLAM algorithm should evaluate every possible action in the robot and map space, but the computational complexity of the evaluation grows exponentially with the search space which proves to be computationally intractable in real applications [24, 169].”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [169]
Citations
Cites
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
(1995) · Cited by 10,558
Efficient Global Optimization of Expensive Black-Box Functions
Jones, D. R., Schonlau, M., & Welch, W. J. (1998) · Journal of Global Optimization · doi:10.1023/a:1008306431147 · Cited by 8,070 · cited 4×
“The objective of Bayesian optimization is to find the minimum of the cost function with as few cost evaluations as possible [Kushner (1964); Jones et al (1998); Locatelli (1997); Mockus et al (1978)].”
BackgroundMethodInfluentialPredictive Control With Constraints
Maciejowski, J. M. (2001) · Cambridge University Engineering Department Publications Database · Cited by 4,368
Simultaneous localization and mapping: part I
Durrant-Whyte, H., & Bailey, T. (2006) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2006.1638022 · Cited by 4,199
Predictive control with constraints
Camacho, E. F. (2003) · Automatica · doi:10.1016/s0005-1098(03)00056-6 · Cited by 3,530
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Optimization—Theory and Applications
Cesari, L. (1983) · Springer eBooks · doi:10.1007/978-1-4613-8165-5 · Cited by 2,559
Simultaneous localization and mapping (SLAM): part II
Bailey, T., & Durrant-Whyte, H. (2006) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2006.1678144 · Cited by 2,554
Marker tracking and HMD calibration for a video-based augmented reality conferencing system
Kato, H., & Billinghurst, M. (2003) · Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99) · doi:10.1109/iwar.1999.803809 · Cited by 2,194 · cited 1×
“The map is built using fiducial landmarks with the ARToolkit Tracker of Kato and Billinghurst (1999).”
Method
Research works printed but not linked: 2
- Gablonsky, J. (2001). Modification of the DIRECT algorithm. PhD thesis, Department of Mathematics, North Carolina State University, Raleigh, North Carolina.
- Martinez-Cantin, R., de Freitas, N., Doucet, A., & Castellanos, J. (2007b). Active policy learning for robot planning and exploration under uncertainty. In Robotics: science and systems (RSS), 2007. · DOI · Paper page
Cited by
Taking the Human Out of the Loop: A Review of Bayesian Optimization
Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & de Freitas, N. (2016) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2494218 · Cited by 6,049
“This interactive Bayesian optimization strategy is particulary effective as humans can be very good at comparing examples, but unable to produce an objective function whose optimum is the example of interest.”
BackgroundPast, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“but tractable solutions for active SLAM include Bayesian Optimization [169] or efficient Gaussian beliefs propagation [195], among others.”
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Brochu, E., Cora, V. M., & de Freitas, N. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.2599 · Cited by 2,011
Self-citationActive learning of inverse models with intrinsically motivated goal exploration in robots
Baranes, A., & Oudeyer, P.-Y. (2013) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2012.05.008 · Cited by 455
Phoenics: A Bayesian Optimizer for Chemistry
Häse, F., Roch, L. M., Kreisbeck, C., & Aspuru-Guzik, A. (2018) · ACS Central Science · doi:10.1021/acscentsci.8b00307 · Cited by 379
A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers
Placed, J. A., Strader, J., Carrillo, H., Atanasov, N., Indelman, V., Carlone, L., & Castellanos, J. A. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3248510 · Cited by 332
“This premise soon led the community to explore DRL, building on existing methods that approached active SLAM with RL [188] and using neural networks to represent the policies or value functions.”
MethodSelf-citationDeliberation for autonomous robots: A survey
Ingrand, F., & Ghallab, M. (2017) · Artificial Intelligence · doi:10.1016/j.artint.2014.11.003 · Cited by 309
“…models and surveyed in [197]; • Environment models , in particular at the geometric level SLAM techniques 209, 11], possibly combined with planning [145], and at the combined geometric and topologic levels, e.g., [45, 125, 184]. sensory-motor and environment the scope of this focus to deliberation.”
BackgroundJuliá, M., Gil, A., & Reinoso, O. (2012) · Autonomous Robots · doi:10.1007/s10514-012-9298-8 · Cited by 256
“Newman et al. (2003) use this representation and select the best navigation points by means of analysing the different features.”
MethodBayesian optimization in high dimensions via random embeddings
Wang, Z., Zoghi, M., Hutter, F., Matheson, D. S., & de Freitas, N. (2013) · UvA-DARE (University of Amsterdam) · Cited by 252
Multi-fidelity Bayesian neural networks: Algorithms and applications
Meng, X., Babaee, H., & Karniadakis, G. E. (2021) · Journal of Computational Physics · doi:10.1016/j.jcp.2021.110361 · Cited by 196
“The GPR uncertainty maps 26 obtaoned could then be utilized for guiding future in-situ measurements using an 27 appropriate acquisition function [10].”
MethodPartially Observable Markov Decision Processes in Robotics: A Survey
Lauri, M., Hsu, D., & Pajarinen, J. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3200138 · Cited by 188
“arbitrary poses, in [170], observations are formed by detecting objects by point cloud segmentation, and in [97], the landmark detections extracted from the image are used as observations.”
BackgroundMethodInfluentialBayesian Estimation of Human Impedance and Motion Intention for Human–Robot Collaboration
Yu, X., He, W., Li, Y., Xue, C., Li, J., Zou, J., & Yang, C. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2940276 · Cited by 162
“The Bayesian estimated methods are widely utilized in dealing with uncertainties in robot motion planning [73] and robot visual tracking [74].”
MethodPortfolio allocation for Bayesian optimization
Hoffman, M. D., Brochu, E., & de Freitas, N. (2011) · Uncertainty in Artificial Intelligence · Cited by 313
Self-citationActive Mapping and Robot Exploration: A Survey
Lluvia, I., Lazkano, E., & Ansuategi, A. (2021) · Sensors · doi:10.3390/s21072445 · Cited by 200
“Overlapping combination of However, in practice, the computational complexity of the evaluation grows exponentially with the search space which proves to be computationally intractable in real applications [39,60,61]. individual tasks.”
BackgroundIndelman, V., Carlone, L., & Dellaert, F. (2015) · The International Journal of Robotics Research · doi:10.1177/0278364914561102 · Cited by 154
Occupancy Anticipation for Efficient Exploration and Navigation
Ramakrishnan, S. K., Al-Halah, Z., & Grauman, K. L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58558-7_24 · Cited by 218
An Optimized Heterogeneous Structure LSTM Network for Electricity Price Forecasting
Zhou, S., Zhou, L., Mao, M., Tai, H., & Wan, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2932999 · Cited by 155
“The expected improvement (EI) strategy [27] is adopted to quantify the probability so that the observed value of x could improve the current optimal objective function value and reflect different improvement amount as well.”
MethodBayesian Optimization of a Free-Electron Laser
Duris, J., Kennedy, D., Hanuka, A., Shtalenkova, J., Edelen, A. L., Baxevanis, P., Egger, A., Cope, T., McIntire, M., Ermon, S., et al. (2020) · Physical Review Letters · doi:10.1103/physrevlett.124.124801 · Cited by 202
“[12, 13], active policy search for reinforcement learning [14], hyper parameter tuning [15], experimental control [16] etc.”
BackgroundLearning Exploration Policies for Navigation
Chen, T., Gupta, S., & Gupta, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.01959 · Cited by 34
“Past works have formulated active SLAM as Partially Observable Markov Decision Processes (POMDPs) (Martinez-Cantin et al., 2009), or as choosing actions that reduce uncertainty in the estimated map (Carrillo et al., 2012).”
BackgroundRobot task planning and explanation in open and uncertain worlds
Hanheide, M., Göbelbecker, M., Horn, G. S., Pronobis, A., Sjöö, K., Aydemir, A., Jensfelt, P., Gretton, C., Dearden, R., Janicek, M., et al. (2017) · Artificial Intelligence · doi:10.1016/j.artint.2015.08.008 · Cited by 145
On the comparison of uncertainty criteria for active SLAM
Carrillo, H., Reid, I., & Castellanos, J. A. (2012) · 2012 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2012.6224890 · Cited by 144
“Recently, a continuous planning approach in states and actions has been proposed by [6].”
BackgroundSelf-citationAccelerating materials discovery with Bayesian optimization and graph deep learning
Zuo, Y., Qin, M., Chen, C., Ye, W., Li, X., Luo, J., & Ong, S. P. (2021) · Materials Today · doi:10.1016/j.mattod.2021.08.012 · Cited by 167
Ergodic Exploration of Distributed Information
Miller, L. M., Silverman, Y., MacIver, M. A., & Murphey, T. D. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2500441 · Cited by 164
“The Bayesian update and the Fisher information matter only in that they allow us to create a map of expected information for the type of parameter estimation problems presented in the examples in Section V. Ergodic exploration could, however, be performed over the expected information calculated…”
BackgroundMethodSergiyenko, O., & Tyrsa, V. V. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3007856 · Cited by 109
“In comparison to Dynamic Bayesian Network in [25]–[27] our kinematic model is not so generalized because of simplifi-cation of task constrains.”
ResultFast charging design for Lithium-ion batteries via Bayesian optimization
Jiang, B., Berliner, M. D., Lai, K., Asinger, P. A., Zhao, H., Herring, P. K., Bazant, M. Z., & Braatz, R. D. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.118244 · Cited by 105
“BO approaches have been proposed for a variety of applications including robot path planning [21], autonomous driving policy determination [22], network sensor placement [23], and hyperparameter tuning for complex machine learning algorithms (e.”
BackgroundPlanning for robotic exploration based on forward simulation
Lauri, M., & Ritala, R. (2016) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2016.06.008 · Cited by 98
“The elements thus defined constitute a discounted reward partially observable Markov decision process (POMDP).”
BackgroundGrado, L. L., Johnson, M. D., & Netoff, T. I. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006606 · Cited by 107
Active SLAM and Exploration with Particle Filters Using Kullback-Leibler Divergence
Carlone, L., Du, J., Kaouk Ng, M., Bona, B., & Indri, M. (2014) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-013-9981-9 · Cited by 94
“In [10], the exploration problem is solved by applying a Bayesian optimization method that allows to reduce the number of simulations needed for planning.”
MethodValencia, R., Valls Miro, J., Dissanayake, G., & Andrade-Cetto, J. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6385637 · Cited by 86
Portfolio Allocation for Bayesian OptimizationPreprint
Brochu, E., Hoffman, M. W., & de Freitas, N. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1009.5419 · Cited by 133
“Examples can be found in robot gait design [26], online path planning [28, 29], intelligent user interfaces for animation [6, 4], algorithm configuration [20], efficient MCMC [34], sensor placement [38, 33], and reinforcement learning [5].”
BackgroundSelf-citationSimultaneous Localization And Mapping: Present, Future, and the Robust-Perception Age.Preprint
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · arXiv (Cornell University) · Cited by 90
Exponential Regret Bounds for Gaussian Process Bandits with Deterministic Observations
de Freitas, N., Smola, A. J., & Zoghi, M. (2012) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1206.6457 · Cited by 76
Bayesian Optimization in High-Dimensional Spaces: A Brief Survey
Malu, M., Dasarathy, G., & Spanias, A. (2021) · International Conference on Information, Intelligence, Systems & Applications (IISA) · doi:10.1109/iisa52424.2021.9555522 · Cited by 105
Learning model-free robot control by a Monte Carlo EM algorithm
Vlassis, N., Toussaint, M., Kontes, G., & Piperidis, S. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9132-0 · Cited by 71
“Reinforcement Learning (RL) is a notable paradigm for robot control with several reported successes in recent years (Tedrake et al., 2005; Abbeel et al., 2007; Kober and Peters, 2009; Riedmiller et al., 2009; Martinez-Cantin et al., 2009).”
BackgroundSpaan, M. T. J., Veiga, T., & Lima, P. U. (2014) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-014-9279-8 · Cited by 80
“One approach is consider non-standard POMDP models for this purpose, where the rewards are belief-based instead of state-based [27,29,56].”
BackgroundPapachristos, C., Mascarich, F., Khattak, S., Dang, T., & Alexis, K. (2019) · Autonomous Robots · doi:10.1007/s10514-019-09864-1 · Cited by 67
Autonomous robotic exploration using a utility function based on Rényi’s general theory of entropy
Carrillo, H., Dames, P., Kumar, V., & Castellanos, J. A. (2017) · Autonomous Robots · doi:10.1007/s10514-017-9662-9 · Cited by 73
“Not surprisingly, this proves to be computationally intractable in real applications (Burgard et al. 2005; Martinez-Cantin et al. 2009) due to the high dimensionality of the joint map and action space.”
BackgroundSelf-citationReal-time adaptive multi-robot exploration with application to underwater map construction
Kapoutsis, A. C., Chatzichristofis, S. A., Doitsidis, L., de Sousa, J. B., Pinto, J., Braga, J., & Kosmatopoulos, E. B. (2015) · Autonomous Robots · doi:10.1007/s10514-015-9510-8 · Cited by 57
“Finally, we mention that another class of approaches for the development of robot exploration approaches employs simulation-based optimization, see e.g. Kollar and Roy (2008), Martinez-Cantin et al. (2009), Peng et al. (2005), and Doucet (1998).”
BackgroundDesign of parallel-jaw gripper tip surfaces for robust grasping
Guo, M., Gealy, D. V., Liang, J., Mahler, J., Goncalves, A., McKinley, S., Ojea, J. A., & Goldberg, K. Y. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989330 · Cited by 67
“Methods to minimize iterations during optimization include MultiArmed Bandits (MABs) [23, 25] and Bayesian Optimization (BO) [16, 26, 27, 35], which allocate fewer samples to regions with a high likelihood of being suboptimal based on samples collected.”
Unscented Bayesian optimization for safe robot grasping
Nogueira, J., Martínez-Cantín, R., Bernardino, A. J. M., & Jamone, L. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759310 · Cited by 58
“autonomous algorithm tuning [13], robot planning [14], [15], control [16], [17], reinforcement learning [18], [5], sensor networks [19], etc.”
Learning-based Path Planning for Autonomous Exploration of Subterranean Environments
Reinhart, R., Dang, T., Hand, E., Papachristos, C., & Alexis, K. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196662 · Cited by 62
“Contributing into the same domain, a set of methods have been proposed to utilize machine learning for the problem of exploration planning [18–20], while a great amount of work has focused on learning for navigation [21].”
MethodHow Effectively Can Adaptive Sampling Methods Capture Spontaneous Ligand Binding?
Betz, R. M., & Dror, R. O. (2019) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.8b00913 · Cited by 55
Transfer Learning for Photovoltaic Power Forecasting with Long Short-Term Memory Neural Network
Zhou, S., Zhou, L., Mao, M., & Xi, X. (2020) · IEEE International Conference on Big Data and Smart Computing (BigComp) · doi:10.1109/bigcomp48618.2020.00-87 · Cited by 58
“…distribution of 1: t D 1 1: max ( ; ) t x t x D (cid:68) (cid:14) (cid:143) (cid:32) In this paper, the exp ected improvement (EI) strategy [24] is utilized, it not only quantifies the probability of the current optimal objective function value could be improved by x , but also reflects…”
MethodSeeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigation
Narasimhan, M., Wijmans, E., Chen, X., Darrell, T., Batra, D., Parikh, D., & Singh, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58523-5_30 · Cited by 65
PLGRIM: Hierarchical Value Learning for Large-scale Exploration in Unknown Environments
Kim, S.-K., Bouman, A., Salhotra, G., Fan, D. D., Otsu, K., Burdick, J. W., & Agha–mohammadi, A. (2021) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v31i1.16014 · Cited by 64
Body schema acquisition through active learning
Martínez-Cantín, R., Lopes, M. C., & Montesano, L. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509406 · Cited by 50
“2) Choose θ ∗ = arg min θ C ( θ ) .”
Online Optimization With Costly and Noisy Measurements Using Random Fourier Expansions
Bliek, L., Verstraete, H. R. G. W., Verhaegen, M., & Wahls, S. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2615134 · Cited by 50
“Bayesian optimization has been successfu lly used in various applications, including active user modeling and reinforcement learning [16], robotics [17], hyperparameter tuning [11], and optics [18].”
MethodOptimal design and operation of Archimedes screw turbines using Bayesian optimization
Lisicki, M., Lubitz, W., & Taylor, G. W. (2016) · Applied Energy · doi:10.1016/j.apenergy.2016.09.084 · Cited by 39
Dealing with asynchronicity in parallel Gaussian Process based global optimization
Ginsbourger, D., Januševskis, J., & Riche, R. L. (2011) · HAL · Cited by 70
Active estimation of object dynamics parameters with tactile sensors
Saal, H. P., Ting, J.-A., & Vijayakumar, S. (2010) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2010.5649191 · Cited by 48
“Another notable use includes the use of GPs to model the cost function, which is then used to trade of exploration and exploitation [13].”
BackgroundBayesian optimization with adaptive kernels for robot control
Martínez-Cantín, R. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989380 · Cited by 55
“In the context of robotics it has been applied for robot planning [7],”
MethodFunneled Bayesian Optimization for Design, Tuning and Control of Autonomous Systems
Martínez-Cantín, R. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2805695 · Cited by 57
Information-based Active SLAM via topological feature graphs
Mu, B., Giamou, M., Paull, L., Agha–mohammadi, A., Leonard, J. J., & How, J. P. (2016) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2016.7799127 · Cited by 46
“There also exists work that maximizes myopic information gain on the next action with partially known maps [6], [7].”
BackgroundA Multi-Objective Exploration Strategy for Mobile Robots Under Operational Constraints
Amanatiadis, A. A., Chatzichristofis, S. A., Charalampous, K., Doitsidis, L., Kosmatopoulos, E. B., Tsalides, P., Gasteratos, A. C., & Roumeliotis, S. I. (2013) · IEEE Access · doi:10.1109/access.2013.2283031 · Cited by 37
“Robot exploration problem is equivalent with the one of dynamically deploying a mobile sensor to learn about an unknown environment [2].”
BackgroundBiswas, A., Morozovska, A. N., Ziatdinov, M. A., Eliseev, E. А., & Kalinin, S. V. (2021) · Journal of Applied Physics · doi:10.1063/5.0068903 · Cited by 44
“This approach has been widely used in many machine learning problems 66 – 70 .”
MethodLopes, M. C., & Oudeyer, P. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2052419 · Cited by 40
“mapping [21], [22], reinforcement learning [23], body schema learning [25], imitation [26], [27], exploration of objects and”
BackgroundDiversity-Guided Multi-Objective Bayesian Optimization With Batch Evaluations
Luković, M. K., Tian, Y., & Matusik, W. (2020) · neural information processing systems · Cited by 123
Multi-robot informative path planning in unknown environments through continuous region partitioning
Dutta, A., Bhattacharya, A., Kreidl, O. P., Ghosh, A., & Dasgupta, P. (2020) · International Journal of Advanced Robotic Systems · doi:10.1177/1729881420970461 · Cited by 37
Mapping, planning and exploration with Pose SLAM
Carreño, R. V. (2013) · UPCommons institutional repository (Universitat Politècnica de Catalunya) · doi:10.5821/dissertation-2117-94927 · Cited by 39
POMDP approach to robotized clothes separation
Purtí, P. M., Alenyà, G., & Torras, C. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6386011 · Cited by 44
“This formulation has been successfully applied to many robotic applications ranging from robot navigation [9], [10] to perception [11], among other applications such as guidance of patients suffering from Alzheimer [12].”
MethodVariable risk control via stochastic optimization
Kuindersma, S., Grupen, R. A., & Barto, A. G. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913476124 · Cited by 38
“…algorithms are a family of global optimization techniques that are well suited to problems where noisy samples of an objective function are expensive to obtain (Lizotte et al., 2007; Frean and Boyle, 2008; Brochu et al., 2009; Martinez-Cantin et al., 2009; Tesch et al., 2011; Wilson et al., 2011).”
Reinforcement Learning-Based Multi-AUV Adaptive Trajectory Planning for Under-Ice Field Estimation
Wang, C., Wei, L., Wang, Z., Song, M., & Mahmoudian, N. (2018) · Sensors · doi:10.3390/s18113859 · Cited by 34
“Specifically, the trajectory planning is often formulated as a partially observable MDP, in which the state is formed by the robot status and the collected field knowledge, and the action corresponds to the waypoints to travel or parameterized continuous trajectories [11–14].”
BackgroundMethodPeralta, F., Reina, D. G., Toral, S. L., Arzamendia, M., & Gregor, D. O. (2021) · IEEE Access · doi:10.1109/access.2021.3050934 · Cited by 50
“A Bayesian Exploration-Exploitation approach for online planning was presented in [25], using a POMDP utility function.”
MethodMultiobjective Optimization Based on Expensive Robotic Experiments under Heteroscedastic Noise
Ariizumi, R., Tesch, M., Kato, K., Choset, H., & Matsuno, F. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2632739 · Cited by 32
“In [37], a hyperprior on a hyperparameter of the kernel function is introduced.”
BackgroundA Successive-Elimination Approach to Adaptive Robotic Source Seeking
Rolf, E., Fridovich-Keil, D., Simchowitz, M., Recht, B., & Tomlin, C. J. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2020.3005537 · Cited by 41
“II, one of the most successful methods for active search in robotics is receding horizon informative path planning, e.g. [14], [15].”
MethodComparison of High-Dimensional Bayesian Optimization Algorithms on BBOB
Santoni, M. L., Raponi, E., Leone, R. D., & Doerr, C. (2024) · ACM Transactions on Evolutionary Learning and Optimization · doi:10.1145/3670683 · Cited by 44
“…et al., 2021], biology, engineering and design [Lam et al., 2018; Raponi et al., 2019, 2021], robotics [Calandra et al., 2016; Junge et al., 2020; Martinez-Cantin et al., 2009], algorithm configuration [Hutter et al., 2011], hyperparameter tuning [Klein et al., 2017; Nguyen, 2019; Turner et al.,…”
Navigating to Objects in Unseen Environments by Distance Prediction
Zhu, M., Zhao, B., & Kong, T. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981766 · Cited by 53
“Some methods [21] formulate this problem as a POMDP (Partially Observable Markov”
A novel method for learning policies from variable constraint data
Howard, M., Klanke, S., Gienger, M., Goerick, C., & Vijayakumar, S. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9129-8 · Cited by 30
“We take a direct policy learning (DPL) approach (Martinez-Cantin et al. 2009; Vlassis et al. 2009; Stolle and Atkeson 2009; Chalodhorn et al. 2006; Nakanishi et al. 2004; Schaal et al. 2003; Atkeson and Schaal 1997; Mussa-Ivaldi 1997) whereby we attempt to learn a continuous model of the policy…”
MethodInformative path planning for anomaly detection in environment exploration and monitoring
Blanchard, A. B., & Sapsis, T. P. (2021) · Ocean Engineering · doi:10.1016/j.oceaneng.2021.110242 · Cited by 39
Tamiya, S., & Yamasaki, H. (2022) · npj Quantum Information · doi:10.1038/s41534-022-00592-6 · Cited by 57
Tanim, A. H., Smith, C., Downey, A. R. J., Imran, J., & Goharian, E. (2024) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2024.106122 · Cited by 34
Utility-Guided Palpation for Locating Tissue Abnormalities
Ayvali, E., Ansari, A., Wang, L., Simaan, N., & Choset, H. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2655619 · Cited by 29
“In this context, GPs are becoming increasingly popular to represent objective functions in planning and decision-making since a continuous model of the function can be formed with few samples [28]–[31].”
BackgroundBatch Bayesian optimization via adaptive local search
Liu, J., Jiang, C., & Zheng, J. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01790-5 · Cited by 28
Duff, A., Rennó‐Costa, C., Marcos, E., Luvizotto, A., Giovannucci, A., Sánchez-Fibla, M., Bernardet, U., & Verschure, P. F. M. J. (2009) · Studies in computational intelligence · doi:10.1007/978-3-642-05181-4_2 · Cited by 27
“Note, that the internal representations formed at the level of the adaptive layer could also serve as a basis for more traditional AI reasoning systems [1, 36] or for localization and mapping methods in autonomous robots [43, 40].”
BackgroundDewangan, R. K., Shukla, A., & Godfrey, W. W. (2020) · Modern Physics Letters B · doi:10.1142/s0217984920501377 · Cited by 26
Wind turbine pitch bearing fault detection with Bayesian augmented temporal convolutional networks
Zhang, C., & Zhang, L. (2023) · Structural Health Monitoring · doi:10.1177/14759217231175886 · Cited by 26
Sample efficient optimization for learning controllers for bipedal locomotion
Antonova, R., Rai, A., & Atkeson, C. G. (2016) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2016.7803249 · Cited by 26
“[13] used BO for online path planning for optimal sensing with a mobile robot.”
MethodOccupancy map building through Bayesian exploration
Francis, G., Ott, L., Marchant, R., & Ramos, F. T. (2019) · The International Journal of Robotics Research · doi:10.1177/0278364919846549 · Cited by 33
“Martinez-Cantin et al. (2007) and more recently Martinez-Cantin et al. (2009) utilised Bayesian optimisation to find control policies that minimise the state error of a robot and landmarks.”
BackgroundMethodIncorporating expert prior in Bayesian optimisation via space warping
Ramachandran, A., Gupta, S., Rana, S., Li, C., & Venkatesh, S. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.105663 · Cited by 51
“Bayesian optimisation has recently become an established and sample-efficient approach for the optimisation of such black-box functions and found several interesting applications in miscellaneous domains: such as reducing the extensive number of experiments that are usually required for a good material design [1], des…”
BackgroundPrediction of concrete abrasion depth and computational design optimization of concrete mixtures
Liu, Q., Andersen, L. V., & Wu, M. (2024) · Cement and Concrete Composites · doi:10.1016/j.cemconcomp.2024.105431 · Cited by 24
Ravi Raj, K. G., & Kos, A. (2023) · Applied Sciences · doi:10.3390/app13179667 · Cited by 23
Gaussian belief space planning with discontinuities in sensing domains
Patil, S., Duan, Y., Schulman, J., Goldberg, K., & Abbeel, P. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907816 · Cited by 24
“For instance, a small set of actions and one-step lookahead (myopic) planning is considered [25], a reinforcement learning approach based on a discretized state and action space is evaluated [13], and active policy search for approximately solving the underlying POMDP has also been explored [16].”
BackgroundBabaee, H., Bastidas, C., DeFilippo, M., Chryssostomidis, C., & Karniadakis, G. E. (2020) · Earth and Space Science · doi:10.1029/2019ea000954 · Cited by 27
“Choosing an optimal kernel can be automated via Bayesian optimization (Gardner et al., 2015; Martinez-Cantin et al., 2009), or via neural-net-induced GP (Pang et al.”
MethodEfficient vision-based navigation
Hornung, A., Bennewitz, M., & Strasdat, H. (2010) · Autonomous Robots · doi:10.1007/s10514-010-9190-3 · Cited by 22
Invertible Particle-Flow-Based Sequential MCMC With Extension to Gaussian Mixture Noise Models
Li, Y., Pal, S., & Coates, M. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2905816 · Cited by 23
“Relevant applications span from robotic learning to financial modelling, from multi-target tracking to weather forecasting [1]–[4].”
BackgroundConstrained Bayesian Optimization for Minimum-Time Charging of Lithium-Ion Batteries
Jiang, B., & Wang, X. (2021) · IEEE Control Systems Letters · doi:10.1109/lcsys.2021.3131269 · Cited by 25
Merino-Casallo, F., Gómez‐Benito, M. J., Juste-Lanas, Y., Martínez-Cantín, R., & García‐Aznar, J. M. (2018) · Frontiers in Physiology · doi:10.3389/fphys.2018.01246 · Cited by 28
Learning Awareness ModelsPreprint
Amos, B., Dinh, L., Cabi, S., Rothörl, T., Colmenarejo, S. G., Muldal, A., Erez, T., Tassa, Y., de Freitas, N., & Denil, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.06318 · Cited by 43
“When stochastic transition and observation models are available, it is possible to use simulation to infer optimal plans for exploring environments (Martinez-Cantin et al., 2009).”
BackgroundOptimal planning for target localization and coverage using range sensing
Miller, L. M., & Murphey, T. D. (2015) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/coase.2015.7294129 · Cited by 18
Planning under uncertainty in the continuous domain: A generalized belief space approach
Indelman, V., Carlone, L., & Dellaert, F. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907858 · Cited by 22
“[23], [24] and Bryson and Sukkarieh [5] investigate planning techniques in conjunction with the use of EKF for estimation.”
MethodYang, Q., Yang, Z. X., Zhang, T., & Hu, G. (2018) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.4658 · Cited by 18
Learning the behavior model of a robot
Infantes, G., Ghallab, M., & Ingrand, F. (2010) · Autonomous Robots · doi:10.1007/s10514-010-9212-1 · Cited by 19
“In [40], the authors use this kind of technique to optimize motion of a robot in order to minimize uncertainty of localization.”
MethodHeteroscedastic Treed Bayesian OptimisationPreprint
Assael, Y., Wang, Z.-Y., Shahriari, B., & de Freitas, N. (2014) · arXiv.org · doi:10.48550/arxiv.1410.7172 · Cited by 48
Self-citationTowards Planning in Generalized Belief Space
Indelman, V., Carlone, L., & Dellaert, F. (2016) · Springer tracts in advanced robotics · doi:10.1007/978-3-319-28872-7_34 · Cited by 17
“[15, 16] and Bryson and Sukkarieh [4] investigate simulation-based approaches in conjunction with the use of EKF as inference engine.”
MethodNguyen, T. D., Gupta, S., Rana, S., & Venkatesh, S. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-57529-2_45 · Cited by 17
Silveira, P. R., Naiff, D., Pereira, C. M. N. A., & Schirru, R. (2017) · Annals of Nuclear Energy · doi:10.1016/j.anucene.2017.11.033 · Cited by 17
Black-box combinatorial optimization using models with integer-valued minima
Bliek, L., Verwer, S., & Weerdt, M. M. D. (2020) · Annals of Mathematics and Artificial Intelligence · doi:10.1007/s10472-020-09712-4 · Cited by 18
Valencia, R., & Andrade‐Cetto, J. (2017) · Springer tracts in advanced robotics · doi:10.1007/978-3-319-60603-3_5 · Cited by 21
Uncertainty-constrained robot exploration: A mixed-integer linear programming approach
Carlone, L., & Lyons, D. J. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6906997 · Cited by 17
“In [13], instead, the problem is solved by applying a Bayesian method that allows reducing the number of simulations needed for planning.”
Method
Metadata
- Authors
- Martinez-Cantin, R., de Freitas, N., Brochu, E., Castellanos, J., & Doucet, A.
- Affiliations
- University of British Columbia (CA)
- Universidad de Zaragoza (ES)
- Corresponding author
- Brochu, Eric
- ORCID
- Castellanos, José
- Venue
- Autonomous Robots
- Published
- 2009-08
- Volume
- 27
- Issue
- 2
- Pages
- 93-103
- ISSN
- 0929-5593 (ISSN-L)
- 1573-7527
- OpenAlex
- W2096649264
- Cited by
- 217
- References
- 62
- Citations by year
- 2025: 6 · 2024: 9 · 2023: 15 · 2022: 11 · 2021: 27 · 2020: 27 · 2019: 13 · 2018: 20 · 2017: 9 · 2016: 17 · 2015: 13 · 2014: 8 · 2013: 11 · 2012: 11
- FWCI
- 11.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Autonomous Robots (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robotic Path Planning Algorithms
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Keywords
- online path planning, exploration-exploitation balance, Bayesian optimization, partially observable Markov decision processes, mobile robot localization, visual guidance, real-time planning, active vision, sequential design, dynamic sensing, sensor calibration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.