The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems
Open accessAbstract
The proven efficacy of learning-based control schemes strongly motivates their application to robotic systems operating in the physical world. However, guaranteeing correct operation during the learning process is currently an unresolved issue, which is of vital importance in safety-critical systems. We propose a general safety framework based on Hamilton-Jacobi reachability methods that can work in conjunction with an arbitrary learning algorithm. The method exploits approximate knowledge of the system dynamics to guarantee constraint satisfaction while minimally interfering with the learning process. We further introduce a Bayesian mechanism that refines the safety analysis as the system acquires new evidence, reducing initial conservativeness when appropriate while strengthening guarantees through real-time validation. The result is a least-restrictive, safety-preserving control law that intervenes only when the computed safety guarantees require it, or confidence in the computed guarantees decays in light of new observations. We prove theoretical safety guarantees combining probabilistic and worst-case analysis and demonstrate the proposed framework experimentally on a quadrotor vehicle. Even though safety analysis is based on a simple point-mass model, the quadrotor successfully arrives at a suitable controller by policy-gradient reinforcement learning without ever crashing, and safely retracts away from a strong external disturbance introduced during flight.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Level Set Methods and Dynamic Implicit Surfaces
Osher, S., & Fedkiw, R. (2003) · Applied Mathematical Sciences · doi:10.1007/b98879 · Cited by 4,951
Efficient implementation of essentially non-oscillatory shock-capturing schemes
Shu, C.-W., & Osher, S. (1988) · Journal of Computational Physics · doi:10.1016/0021-9991(88)90177-5 · Cited by 4,886
A time-dependent Hamilton-Jacobi formulation of reachable sets for continuous dynamic games
Mitchell, I. M., Bayen, A. M., & Tomlin, C. J. (2005) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2005.851439 · Cited by 1,152
Self-citation(2015) · Journal of Machine Learning Research · Cited by 1,119
Numerical Computation of Multivariate Normal Probabilities
Genz, A. (1992) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1992.10477010 · Cited by 1,020
Safety Verification of Hybrid Systems Using Barrier Certificates
Prajna, S., & Jadbabaie, A. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24743-2_32 · Cited by 642
Provably safe and robust learning-based model predictive control
Aswani, A., Gonzalez, H., Sastry, S. S., & Tomlin, C. (2013) · Automatica · doi:10.1016/j.automatica.2013.02.003 · Cited by 539
Self-citationA simple learning strategy for high-speed quadrocopter multi-flips
Lupashin, S., Schöllig, A., Sherback, M., & D'Andrea, R. (2010) · 2010 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2010.5509452 · Cited by 278
Reachability-based safe learning with Gaussian processes
Akametalu, A. K., Fisac, J. F., Gillula, J. H., Kaynama, S., Zeilinger, M. N., & Tomlin, C. J. (2014) · 53rd IEEE Conference on Decision and Control · doi:10.1109/cdc.2014.7039601 · Cited by 249
Self-citationLearning for control from multiple demonstrations
Coates, A., Abbeel, P., & Ng, A. Y. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390175 · Cited by 228
Reach-avoid problems with time-varying dynamics, targets and constraints
Fisac, J. F., Chen, M., Tomlin, C. J., & Sastry, S. S. (2015) · Proceedings of the 18th International Conference on Hybrid Systems: Computation and Control · doi:10.1145/2728606.2728612 · Cited by 220
Self-citationRisk-Sensitive Reinforcement Learning Applied to Control under Constraints
Geibel, P., & Wysotzki, F. (2005) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1666 · Cited by 200
Darbon, J., & Osher, S. (2016) · Research in the Mathematical Sciences · doi:10.1186/s40687-016-0068-7 · Cited by 194
Safe learning of regions of attraction for uncertain, nonlinear systems with Gaussian processes
Berkenkamp, F., Moriconi, R., Schoellig, A. P., & Krause, A. (2016) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2016.7798979 · Cited by 189
FaSTrack: A modular framework for fast and guaranteed safe motion planning
Herbert, S. L., Chen, M., Han, S., Bansal, S., Fisac, J. F., & Tomlin, C. J. (2017) · 2017 IEEE 56th Annual Conference on Decision and Control (CDC) · doi:10.1109/cdc.2017.8263867 · Cited by 161
Self-citationSafe Exploration in Markov Decision Processes
Moldovan, T. M., & Abbeel, P. (2012) · arXiv · doi:10.48550/arxiv.1205.4810 · Cited by 152
A Toolbox of Hamilton-Jacobi Solvers for Analysis of Nondeterministic Continuous and Hybrid Systems
Mitchell, I. M., & Templeton, J. A. (2005) · Lecture notes in computer science · doi:10.1007/978-3-540-31954-2_31 · Cited by 145
Guaranteed Safe Online Learning via Reachability: tracking a ground target using a quadrotor
Gillula, J., & Tomlin, C. J. (2012) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2012.6225136 · Cited by 94
Self-citationChen, M., Fisac, J. F., Sastry, S., & Tomlin, C. J. (2015) · European Control Conference (ECC) · doi:10.1109/ecc.2015.7331044 · Cited by 65
Self-citationHobbs, A. N. (2010) · Elsevier eBooks · doi:10.1016/b978-0-12-374518-7.00016-x · Cited by 63
Kolter, J. Z., Plagemann, C., Jackson, D. T., Ng, A. Y., & Thrun, S. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509562 · Cited by 62
Policy search via the signed derivative
Kolter, J. Z., & Ng, A. Y. (2009) · doi:10.15607/rss.2009.v.027 · Cited by 49
Scalable Safety-Preserving Robust Control Synthesis for Continuous-Time Linear Systems∗Preprint
(n.d.) · Cited by 33
Feedback controller parameterizations for Reinforcement Learning
Roberts, J. W., Manchester, I. R., & Tedrake, R. (2011) · IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning · doi:10.1109/adprl.2011.5967370 · Cited by 26
Gillula, J., & Tomlin, C. J. (2012) · Robotics: Science and Systems · doi:10.15607/rss.2012.viii.011 · Cited by 26
Self-citation
Cited by
Learning-Based Model Predictive Control: Toward Safe Learning in Control
Hewing, L., Wabersich, K. P., Menner, M., & Zeilinger, M. N. (2020) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-090419-075625 · Cited by 835
Self-citationSafe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning
Brunke, L., Greeff, M., Hall, A. W., Yuan, Z., Zhou, S., Panerati, J., & Schoellig, A. P. (2022) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-042920-020211 · Cited by 769
Data-Enabled Predictive Control: In the Shallows of the DeePC
Coulson, J., Lygeros, J., & Dorfler, F. (2019) · 2019 18th European Control Conference (ECC) · doi:10.23919/ecc.2019.8795639 · Cited by 628
Cheng, R., Orosz, G., Murray, R. M., & Burdick, J. W. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013387 · Cited by 611
Hamilton-Jacobi reachability: A brief overview and recent advances
Bansal, S., Chen, M., Herbert, S., & Tomlin, C. J. (2017) · 2017 IEEE 56th Annual Conference on Decision and Control (CDC) · doi:10.1109/cdc.2017.8263977 · Cited by 471
Self-citation
Metadata
- Authors
- Fisac, J. F., Akametalu, A. K., Zeilinger, M. N., Kaynama, S., Gillula, J., & Tomlin, C. J.
- Venue
- IEEE Transactions on Automatic Control
- Published
- 2019-07
- Volume
- 64
- Issue
- 7
- Pages
- 2737-2752
- ISSN
- 0018-9286 (ISSN-L)
- 1558-2523
- 2334-3303
- OpenAlex
- W2611136914
- Cited by
- 450
- References
- 26
- FWCI
- 33.38
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.