The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics
Open accessAbstract
Selecting the right tuning parameters for algorithms is a pravelent problem in machine learning that can significantly affect the performance of algorithms. Data-efficient optimization algorithms, such as Bayesian optimization, have been used to automate this process. During experiments on real-world systems such as robotic platforms these methods can evaluate unsafe parameters that lead to safety-critical system failures and can destroy the system. Recently, a safe Bayesian optimization algorithm, called SafeOpt, has been developed, which guarantees that the performance of the system never falls below a critical value; that is, safety is defined based on the performance function. However, coupling performance and safety is often not desirable in practice, since they are often opposing objectives. In this paper, we present a generalized algorithm that allows for multiple safety constraints separate from the objective. Given an initial set of safe parameters, the algorithm maximizes performance but only evaluates parameters that satisfy safety for all constraints with high probability. To this end, it carefully explores the parameter space by exploiting regularity assumptions in terms of a Gaussian process prior. Moreover, we show how context variables can be used to safely transfer knowledge to new situations and tasks. We provide a theoretical analysis and demonstrate that the proposed algorithm enables fast, automatic, and safe optimization of tuning parameters in experiments on a quadrotor vehicle.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Zhou, K. (1998) · Cited by 3,824
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160
A Taxonomy of Global Optimization Methods Based on Response Surfaces
Jones, D. R. (2001) · Journal of Global Optimization · doi:10.1023/a:1012771025575 · Cited by 2,159
Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting
Srinivas, N., Krause, A., Kakade, S. M., & Seeger, M. W. (2012) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2182033 · Cited by 1,850
Self-citationCompressed Sensing: Theory and Applications
(2012) · Cambridge University Press eBooks · doi:10.1017/cbo9780511794308 · Cited by 1,660
(2020) · Data Mining and Machine Learning · doi:10.1017/9781108564175.026 · Cited by 1,401
Reinforcement learning of motor skills with policy gradients
Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831
Automatic tuning and adaptation for PID controllers - a survey
Åström, K. J., Hägglund, T. J., Hang, C. C., & Ho, W. K. (1993) · Control Engineering Practice · doi:10.1016/0967-0661(93)91394-c · Cited by 686
Kernels for Vector-Valued Functions: a Review
Álvarez, M. A., Rosasco, L., & Lawrence, N. D. (2012) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · Cited by 587
Safe Exploration for Optimization with Gaussian Processes
Sui, Y., Gotovos, A., Burdick, J. W., & Krause, A. (2015) · CaltechAUTHORS (California Institute of Technology) · Cited by 572
Self-citationProvably safe and robust learning-based model predictive control
Aswani, A., Gonzalez, H., Sastry, S. S., & Tomlin, C. (2013) · Automatica · doi:10.1016/j.automatica.2013.02.003 · Cited by 539
Policy Gradient Methods for Robotics
Peters, J., & Schaal, S. (2006) · 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2006.282564 · Cited by 498
Convergence rates of efficient global optimization algorithms
Bull, A. D. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1101.3501 · Cited by 446
Kernels for Vector-Valued Functions: A Review
Álvarez, M. A., Rosasco, L., & Lawrence, N. D. (2012) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000036 · Cited by 415
Bayesian Approach to Global Optimization--Theory and Applications.
Boender, C. G. E., & Mockus, J. (1991) · Mathematics of Computation · doi:10.2307/2008419 · Cited by 409
PID tuning using extremum seeking: online, model-free performance optimization
Killingsworth, N., & Krstić, M. (2006) · IEEE Control Systems · doi:10.1109/mcs.2006.1580155 · Cited by 396
Derivative Observations in Gaussian Process Models of Dynamic Systems
Solak, E., Murray‐Smith, R., Leith, D. J., & Rasmussen, C. E. (2002) · Cambridge University Engineering Department Publications Database · Cited by 353
Bayesian Optimization with Unknown Constraints
Gelbart, M. A., Snoek, J., & Adams, R. P. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.5607 · Cited by 299
Safe Model-based Reinforcement Learning with Stability Guarantees
Berkenkamp, F., Turchetta, M., Schoellig, A. P., & Krause, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08551 · Cited by 299
Self-citationContextual Gaussian Process Bandit Optimization
Krause, A., & Ong, C. S. (2011) · Cited by 267
Self-citationSafe controller optimization for quadrotors with Gaussian processes
Berkenkamp, F., Schoellig, A. P., & Krause, A. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487170 · Cited by 264
Self-citationBayesian optimization in high dimensions via random embeddings
Wang, Z., Zoghi, M., Hutter, F., Matheson, D. S., & de Freitas, N. (2013) · UvA-DARE (University of Amsterdam) · Cited by 252
Reachability-based safe learning with Gaussian processes
Akametalu, A. K., Fisac, J. F., Gillula, J. H., Kaynama, S., Zeilinger, M. N., & Tomlin, C. J. (2014) · 53rd IEEE Conference on Decision and Control · doi:10.1109/cdc.2014.7039601 · Cited by 249
Automatic gait optimization with Gaussian process regression
Lizotte, D. J., Wang, T., Bowling, M., & Schuurmans, D. (2007) · Cited by 237
Robust Constrained Learning-based NMPC enabling reliable mobile robot path tracking
Ostafew, C. J., Schoellig, A. P., & Barfoot, T. D. (2016) · The International Journal of Robotics Research · doi:10.1177/0278364916645661 · Cited by 232
Self-citationA platform for aerial robotics research and demonstration: The Flying Machine Arena
Lupashin, S., Hehn, M., Mueller, M. W., Schoellig, A. P., Sherback, M., & Raffaello D’andrea, B. (2014) · Mechatronics · doi:10.1016/j.mechatronics.2013.11.006 · Cited by 203
Self-citationDuvenaud, D., Nickisch, H., & Rasmussen, C. E. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1112.4394 · Cited by 183
Schaal, S., & Atkeson, C. G. (2010) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2010.936957 · Cited by 177
Safe and robust learning control with Gaussian processes
Berkenkamp, F., & Schoellig, A. P. (2015) · 2015 European Control Conference (ECC) · doi:10.1109/ecc.2015.7330913 · Cited by 169
Self-citationSafe Exploration in Markov Decision Processes
Moldovan, T. M., & Abbeel, P. (2012) · arXiv · doi:10.48550/arxiv.1205.4810 · Cited by 152 · cited 2×
“2017) or the system never leaves a safe subset of the state space (Ostafew et al. 2016; Aswani et al. 2013; Akametalu et al. 2014; Moldovan and Abbeel 2012; Turchetta et al. 2016).”
Safe Exploration of State and Action Spaces in Reinforcement Learning
García, J., & Fernandez, F. (2012) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3761 · Cited by 144
Marco, A., Berkenkamp, F., Hennig, P., Schoellig, A. P., Krause, A., Schaal, S., & Trimpe, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989186 · Cited by 120
Self-citationHigh-Dimensional Gaussian Process Bandits
Djolonga, J., Krause, A., & Cevher, V. (2013) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 116
Self-citationSafe Exploration in Finite Markov Decision Processes with Gaussian ProcessesPreprint
Turchetta, M., Berkenkamp, F., & Krause, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04753 · Cited by 116
Self-citationUsing response surfaces and expected improvement to optimize snake robot gait parameters
Tesch, M., Schneider, J., & Choset, H. (2011) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2011.6095076 · Cited by 110
Stagewise Safe Bayesian Optimization with Gaussian ProcessesPreprint
Sui, Y., Zhuang, V., Burdick, J. W., & Yue, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.07555 · Cited by 102
Safe Exploration for Active Learning with Gaussian Processes
Schreiter, J., Nguyen-Tuong, D., Eberts, M., Bischoff, B., Markert, H., & Toussaint, M. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-23461-8_9 · Cited by 89
An experimental comparison of Bayesian optimization for bipedal locomotion
Calandra, R., Andre Seyfarth, A., Peters, J., & Deisenroth, M. P. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907117 · Cited by 78
Safe Exploration for Interactive Machine LearningPreprint
Turchetta, M., Berkenkamp, F., & Krause, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.13726 · Cited by 61
Self-citationConstrained Bayesian Optimization with Particle Swarms for Safe Adaptive Controller Tuning
Duivenvoorden, R. R. P. R., Berkenkamp, F., Carion, N., Krause, A., & Schoellig, A. P. (2017) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2017.08.1991 · Cited by 43
Self-citationFeasiblity of motion primitives for choreographed quadrocopter flight
Schoellig, A. P., Hehn, M., Lupashin, S., & D’Andrea, R. (2011) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2011.5991482 · Cited by 42
Self-citationFeed-forward parameter identification for precise periodic quadrocopter motions
Schoellig, A. P., Wiltsche, C., & D’Andrea, R. (2012) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2012.6315248 · Cited by 20
Self-citation
Cited by
Concrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314
Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning
Brunke, L., Greeff, M., Hall, A. W., Yuan, Z., Zhou, S., Panerati, J., & Schoellig, A. P. (2022) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-042920-020211 · Cited by 769
Self-citationLearning-Based Model Predictive Control for Safe Exploration
Koller, T., Berkenkamp, F., Turchetta, M., & Krause, A. (2018) · 2018 IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2018.8619572 · Cited by 318
Self-citationSafe controller optimization for quadrotors with Gaussian processes
Berkenkamp, F., Schoellig, A. P., & Krause, A. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487170 · Cited by 264
Self-citationBayesian optimization for maximum power point tracking in photovoltaic power plants
Abdelrahman, H., Berkenkamp, F., Poland, J., & Krause, A. (2016) · 2016 European Control Conference (ECC) · doi:10.1109/ecc.2016.7810598 · Cited by 45
Self-citation
Metadata
- Authors
- Berkenkamp, F., Krause, A., & Schoellig, A. P.
- Venue
- Machine Learning
- Published
- 2021-06-24
- Volume
- 112
- Issue
- 10
- Pages
- 3713-3747
- ISSN
- 0885-6125 (ISSN-L)
- 1573-0565
- OpenAlex
- W2284548247
- Cited by
- 286
- References
- 47
- FWCI
- 24.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.