End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks
Open accessAbstract
Reinforcement Learning (RL) algorithms have found limited success beyond simulated applications, and one main reason is the absence of safety guarantees during the learning process. Real world systems would realistically fail or break before an optimal controller can be learned. To address this issue, we propose a controller architecture that combines (1) a model-free RL-based controller with (2) model-based controllers utilizing control barrier functions (CBFs) and (3) online learning of the unknown system dynamics, in order to ensure safety during learning. Our general framework leverages the success of RL algorithms to learn high-performance controllers, while the CBF-based controllers both guarantee safety and guide the learning process by constraining the set of explorable polices. We utilize Gaussian Processes (GPs) to model the system dynamics and its uncertainties. Our novel controller synthesis algorithm, RL-CBF, guarantees safety with high probability during the learning process, regardless of the RL algorithm used, and demonstrates greater policy exploration efficiency. We test our algorithm on (1) control of an inverted pendulum and (2) autonomous carfollowing with wireless vehicle-to-vehicle communication, and show that our algorithm attains much greater sample efficiency in learning than other state-of-the-art algorithms and maintains safety during the entire learning process.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
RL algorithms have been increasingly adopted by the robotics community over the past years to control complex robots or multi-robot systems, or provide end-to-end policies from perception to control.
“Reinforcement learning (RL) algorithms have been increasingly adopted by the robotics community over the past years to control complex robots or multi-robot systems [1], [2], or provide end-to-end policies from perception to control [3].”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [3]
The Introduction (p. 2) describes an end-to-end safe RL controller for nonlinear control systems, which supports the reference to end-to-end control policies. It does not describe a policy that runs from perception to control; the paper’s “end-to-end” qualifier refers to safety during learning. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705 · see citing works
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · see citing works
(1995) · Cited by 10,558 · see citing works
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578 · cited 13× · see citing works
“Furthermore, if we use TRPO for the RL algorithm, then the control law Proof.”
MethodInfluentialDeterministic Policy Gradient Algorithms
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014) · HAL · Cited by 3,275 · see citing works
Apprenticeship learning via inverse reinforcement learning
Abbeel, P., & Ng, A. Y. (2004) · doi:10.1145/1015330.1015430 · Cited by 2,873 · cited 1× · see citing works
“Previous approaches to safe reinforcement learning include reward-shaping , policy optimization with constraints (Gaskett 2003; Moldovan and Abbeel 2012; Achiam et al. 2017; Wachi et al. 2018), or teacher advice (Abbeel and Ng 2004; Abbeel, Coates, and Ng 2010; Tang et al. 2010).”
BackgroundControl Barrier Function Based Quadratic Programs for Safety Critical Systems
Ames, A. D., Xu, X., Grizzle, J. W., & Tabuada, P. (2016) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2016.2638961 · Cited by 2,682 · cited 1× · see citing works
“The existence of a CBF implies that there exists a deterministic controller u CBF : S → A such that the set C is forward invariant for system (1) (Agrawal and Sreenath 2017; Ames et al. 2017).”
Background(2015) · Journal of Machine Learning Research · Cited by 1,119 · see citing works
Benchmarking Deep Reinforcement Learning for Continuous Control
Duan, Y., Chen, X., Houthooft, R., Schulman, J., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1604.06778 · Cited by 1,114 · cited 1× · see citing works
“This method has been successfully applied to continuous control tasks (Duan et al. 2016; Lillicrap et al. 2015) where controllers have learned to stabilize complex robots (after many policy iterations).”
MethodReinforcement learning of motor skills with policy gradients
Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831 · see citing works
Constrained Policy OptimizationPreprint
Achiam, J., Held, D., Tamar, A., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.10528 · Cited by 706 · cited 1× · see citing works
“Previous approaches to safe reinforcement learning include reward-shaping , policy optimization with constraints (Gaskett 2003; Moldovan and Abbeel 2012; Achiam et al. 2017; Wachi et al. 2018), or teacher advice (Abbeel and Ng 2004; Abbeel, Coates, and Ng 2010; Tang et al. 2010).”
BackgroundAutonomous Helicopter Aerobatics through Apprenticeship Learning
Abbeel, P., Coates, A., & Ng, A. (2010) · Int. J. Robotics Res. · doi:10.1177/0278364910371999 · Cited by 682 · cited 1× · see citing works
“Previous approaches to safe reinforcement learning include reward-shaping , policy optimization with constraints (Gaskett 2003; Moldovan and Abbeel 2012; Achiam et al. 2017; Wachi et al. 2018), or teacher advice (Abbeel and Ng 2004; Abbeel, Coates, and Ng 2010; Tang et al. 2010).”
BackgroundSafe Model-based Reinforcement Learning with Stability GuaranteesPreprint
Berkenkamp, F., Turchetta, M., Schoellig, A. P., & Krause, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08551 · Cited by 555 · cited 1× · see citing works
“…Lyapunov-based methods or model predictive control to guarantee safety under system dynamics during learning (Wang, Theodorou, and Egerstedt 2017; Berkenkamp et al. 2017; Chow et al. 2018; Ohnishi et al. 2018; Koller et al. 2018), but they do not address the issue of exploration and performance…”
MethodA General Safety Framework for Learning-Based Control in Uncertain Robotic Systems
Fisac, J. F., Akametalu, A. K., Zeilinger, M. N., Kaynama, S., Gillula, J., & Tomlin, C. J. (2018) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2018.2876389 · Cited by 460 · see citing works
Agrawal, A., & Sreenath, K. (2017) · doi:10.15607/rss.2017.xiii.073 · Cited by 306 · see citing works
A Lyapunov-based Approach to Safe Reinforcement LearningPreprint
Chow, Y., Nachum, O., Duéñez‐Guzmán, E. A., & Ghavamzadeh, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07708 · Cited by 248 · cited 1× · see citing works
“…methods or model predictive control to guarantee safety under system dynamics during learning (Wang, Theodorou, and Egerstedt 2017; Berkenkamp et al. 2017; Chow et al. 2018; Ohnishi et al. 2018; Koller et al. 2018), but they do not address the issue of exploration and performance optimization.”
MethodLocal and global sparse Gaussian process approximations.
Snelson, E., & Zoubin Ghahramani, A. (2007) · Cited by 236 · see citing works
Simulation Optimization: A Review, New Developments, and Applications
Fu, M. C., Glover, F., & April, J. (2006) · Winter Simulation Conference · doi:10.1109/wsc.2005.1574242 · Cited by 198 · cited 1× · see citing works
“…to maximize long-term reward J ( π ) using (a) policy iteration meth-ods (Bertsekas 2005), (b) derivative-free optimization meth-ods that optimize the return as a function of policy parameters (Fu, Glover, and April 2005), or (c) policy gradient methods (Peters and Schaal 2008; Silver et al. 2014).”
MethodSafe Learning of Quadrotor Dynamics Using Barrier Certificates
Wang, L., Theodorou, E. A., & Egerstedt, M. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460471 · Cited by 175 · see citing works
Safe Exploration in Markov Decision Processes
Moldovan, T. M., & Abbeel, P. (2012) · arXiv · doi:10.48550/arxiv.1205.4810 · Cited by 152 · see citing works
Lyapunov design for safe reinforcement learning
Perkins, T. J., & Barto, A. G. (2003) · doi:10.5555/944919.944955 · Cited by 148 · see citing works
Safe Exploration Algorithms for Reinforcement Learning Controllers
Mannucci, T., Kampen, E.-J. V., de Visser, C. C., & Chu, Q. P. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2654539 · Cited by 104 · cited 1× · see citing works
“Other works guarantee safety by switching between backup controllers (Perkins and Barto 2003; Mannucci et al. 2018), though this overly constrains policy exploration.”
BackgroundGuaranteed Safe Online Learning via Reachability: tracking a ground target using a quadrotor
Gillula, J., & Tomlin, C. J. (2012) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2012.6225136 · Cited by 94 · see citing works
Safe Exploration and Optimization of Constrained MDPs Using Gaussian Processes
Wachi, A., Sui, Y., Yue, Y., & Ono, M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12103 · Cited by 91 · cited 1× · see citing works
“Previous approaches to safe reinforcement learning include reward-shaping , policy optimization with constraints (Gaskett 2003; Moldovan and Abbeel 2012; Achiam et al. 2017; Wachi et al. 2018), or teacher advice (Abbeel and Ng 2004; Abbeel, Coates, and Ng 2010; Tang et al. 2010).”
BackgroundLocal Gaussian process regression for real time online model learning and control
Nguyen-Tuong, D., Peters, J., & Seeger, M. (2008) · TUbilio (Technical University of Darmstadt) · Cited by 87 · see citing works
Learning-based Model Predictive Control for Safe Exploration and Reinforcement LearningPreprint
Koller, T., Berkenkamp, F., Turchetta, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.12189 · Cited by 48 · cited 1× · see citing works
“…methods or model predictive control to guarantee safety under system dynamics during learning (Wang, Theodorou, and Egerstedt 2017; Berkenkamp et al. 2017; Chow et al. 2018; Ohnishi et al. 2018; Koller et al. 2018), but they do not address the issue of exploration and performance optimization.”
MethodSafe Reinforcement Learning: Learning with Supervision Using a Constraint-Admissible Set
Li, Z., Kalabic, U. V., & Chu, T. (2018) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.2018.8430770 · Cited by 45 · see citing works
Parameterized maneuver learning for autonomous helicopter flight
Tang, J., Singh, A., Goehausen, N., et al. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509832 · Cited by 37 · cited 1× · see citing works
“Previous approaches to safe reinforcement learning include reward-shaping , policy optimization with constraints (Gaskett 2003; Moldovan and Abbeel 2012; Achiam et al. 2017; Wachi et al. 2018), or teacher advice (Abbeel and Ng 2004; Abbeel, Coates, and Ng 2010; Tang et al. 2010).”
BackgroundSafe Reinforcement Learning via Shielding
Alshiekh, M., Bloem, R., Ehlers, R., Könighofer, B., Niekum, S., & Topcu, U. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11797 · Cited by 37 · cited 1× · see citing works
“The concept is akin to shielded RL (Alshiekh et al. 2017; Fisac et al. 2018), since the CBF controller compensates for the RL controller to ensure safety, but does not guide exploration of the RL algorithm.”
BackgroundScalable synthesis of safety certificates from data with application to learning-based control
Wabersich, K. P., & Zeilinger, M. N. (2018) · European Control Conference (ECC) · doi:10.23919/ecc.2018.8550288 · Cited by 31 · see citing works
Data-based fuel-economy optimization of connected automated trucks in traffic
He, C. R., Ge, J. I., & Orosz, G. (2018) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.2018.8431512 · Cited by 10 · cited 1× · see citing works
“…methods or model predictive control to guarantee safety under system dynamics during learning (Wang, Theodorou, and Egerstedt 2017; Berkenkamp et al. 2017; Chow et al. 2018; Ohnishi et al. 2018; Koller et al. 2018), but they do not address the issue of exploration and performance optimization.”
MethodSelf-citationSafety-aware Adaptive Reinforcement Learning with Applications to Brushbot NavigationPreprint
Ohnishi, M., Wang, L., Notomista, G., et al. (2018) · arXiv (Cornell University) · Cited by 2 · see citing works
Flagged first, then by citations
2 of the 34 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning
Brunke, L., Greeff, M., Hall, A. W., Yuan, Z., Zhou, S., Panerati, J., & Schoellig, A. P. (2022) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-042920-020211 · Cited by 825 · see citing works
“Input & state (123, 124, 125, 126, 127, 128, 129, 130, 139, 140, 141, 142), State (133, 134, 135) (123, 124, 125, 126, 127, 128, 129, 130, 133, 134, 135, 139, 140, 141) Input & state (119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 139, 140, 141, 142), State (133, 134, 135, 136) (119, 120, 121, 122, 1…”
MethodInfluentialSim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
Zhao, W., Queralta, J. P., & Westerlund, T. (2020) · 2020 IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308468 · Cited by 941 · see citing works
“In this direction, recent works have shown promising results towards safe DRL that is able to ensure convergence even while reducing the exploration space [3].”
BackgroundZhu, M., Wang, Y., Pu, Z., Hu, J., Wang, X., & Ke, R. (2020) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2020.102662 · Cited by 471 · see citing works
Chen, X., Qu, G., Tang, Y., Low, S. H., & Li, N. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3154718 · Cited by 440 · see citing works
“…common ways to handle soft constraints include 1) using barrier functions or penalty functions to integrate the constraints to the reward function [172]; 2) modeling as a chance constraint (i.e., the probability of constraint violation is no larger than a predefined threshold) [173], [174] or a…”
BackgroundA Review of Safe Reinforcement Learning: Methods, Theories, and Applications
Gu, S., Yang, L., Du, Y., Chen, G., Walter, F., Wang, Y.-X., & Knoll, A. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3457538 · Cited by 341 · see citing works
“Moreover,[158]leveragesamodel-free RLandCBFstoguaranteelearningsafetywhileimproving learningefficiency,inwhichCBFsareusedtoconstrainthe searchspacetoenhancelearningsafetyandefficiencybased onaGPlearningmodel.”
Dawson, C., Gao, S., & Fan, C. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3232542 · Cited by 275 · see citing works
“The first class of methods use an a priori given certificate to constrain the actions that the RL agent can take [33], [34], [35], possibly with some learning of model uncertainty as the controller explores [33], [34].”
MethodSafe reinforcement learning: A control barrier function optimization approach
Marvi, Z., & Kiumarsi, B. (2020) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.5132 · Cited by 257 · see citing works
Di, X. S., & Shi, R. (2021) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2021.103008 · Cited by 240 · see citing works
“(Cheng et al., 2019) develops a controller architecture that synthesizes a model-free RL controller and a model-based controller utilizing control barrier functions, which is shown to guarantee safety with high probability.”
BackgroundAdaptive Safety with Control Barrier Functions
Taylor, A. J., & Ames, A. D. (2020) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc45564.2020.9147463 · Cited by 229 · see citing works
“Data-driven methods employing machine learning [19], [5] provide probabilistic safety guarantees, but may require episodic, offline training to improve model estimates [6].”
MethodVariable Impedance Control and Learning—A Review
Abu‐Dakka, F. J., & Saveriano, M. (2020) · Frontiers in Robotics and AI · doi:10.3389/frobt.2020.590681 · Cited by 220 · see citing works
“A possible way to guarantee a safe exploration is to identify a set of safe states where the robot is stable (in the sense of Lyapunov) and to prevent the robot to visit unsafe states (Berkenkamp et al., 2017; Chow et al., 2018; Cheng et al., 2019).”
BackgroundLearning Control Barrier Functions from Expert Demonstrations
Robey, A., Hu, H., Lindemann, L., et al. (2020) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc42340.2020.9303785 · Cited by 181 · see citing works
“Related work on learning and CBFs: Methods using barrier and control barrier functions to ensure safety and guide exploration during episodic supervised learning of uncertain linear dynamics include [11]–[14].”
BackgroundRobust Safety-Critical Control for Dynamic Robotics
Nguyen, Q., & Sreenath, K. (2021) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2021.3059156 · Cited by 172 · see citing works
Robust Control Barrier–Value Functions for Safety-Critical Control
Choi, J. J., Lee, D., Sreenath, K., Tomlin, C. J., & Herbert, S. (2021) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc45484.2021.9683085 · Cited by 131 · see citing works
“Also, it can be used as an automatic safety filter (as opposed to using least-restrictive control) [13], [14], [15].”
MethodSynthesis of Control Barrier Functions Using a Supervised Machine Learning Approach
Srinivasan, M., Dabholkar, A., Coogan, S., et al. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341190 · Cited by 125 · see citing works
“Employing pre-specified barrier functions during the action policy exploration and keeping track of the safety interventions to influence the explored policies, can provide the necessary safety guarantees [9].”
BackgroundControl barrier functions for stochastic systems
Clark, A. (2021) · Automatica · doi:10.1016/j.automatica.2021.109688 · Cited by 119 · see citing works
“Learning-based methods for CBFs in systems with incomplete information due to uncertainties were presented in [8,12,20,7].”
MethodSafe reinforcement learning for power system control: A review
Yu, P., Zhang, H., Song, Y., Wang, Z., Dong, H., & Ji, L. (2025) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2025.116022 · Cited by 117 · see citing works
Safe exploration in model-based reinforcement learning using control barrier functions
Cohen, M. H., & Belta, C. A. (2022) · Automatica · doi:10.1016/j.automatica.2022.110684 · Cited by 107 · see citing works
A formal methods approach to interpretable reinforcement learning for robotic planning
Li, X., Serlin, Z., Yang, G., & Belta, C. A. (2019) · Science Robotics · doi:10.1126/scirobotics.aay6276 · Cited by 104 · see citing works
“Combining CBF with RL has been looked at by the authors of (25) and (26), whereas the authors of (27) made an effort to integrate temporal logic with CBF.”
BackgroundHu, Y., Fu, J., & Wen, G. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3233592 · Cited by 104 · see citing works
“By leveraging model-based conditions, the CBF-based safe RL approach has been proposed, where the farsighted safe actions can be produced to ensure the forward invariance of safe sets [15], [16].”
BackgroundMethodZhang, Y., Liang, X., Li, D., Ge, S. S., Gao, B., Chen, H., & Lee, T. H. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3186528 · Cited by 97 · see citing works
“First, describe the following helpful result about CBF from [14].”
BackgroundMethodInfluentialHow to certify machine learning based safety-critical systems? A systematic literature review
Tambon, F., Laberge, G., An, L., Nikanjam, A., Mindom, P. S. N., Pequignot, Y., Khomh, F., Antoniol, G., Merlo, E., & Laviolette, F. (2022) · Automated Software Engineering · doi:10.1007/s10515-022-00337-x · Cited by 100 · see citing works
“Similarly, RL-CBF framework [40] is a combination of a model-free RL-based controller, model-based controllers utilizing CBFs and real-time learning of the unknown system dynamics.”
MethodSafe Multi-Agent Interaction through Robust Control Barrier Functions with Learned Uncertainties
Cheng, R., Khojasteh, M. J., Ames, A. D., et al. (2020) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc42340.2020.9304395 · Cited by 89 · see citing works
Self-citationHobbs, K. L., Mote, M., Abate, M., Coogan, S., & Féron, É. (2023) · IEEE Control Systems · doi:10.1109/mcs.2023.3234380 · Cited by 90 · see citing works
“A variety of RTA approaches have been explored in the literature including human-like intervention [190], Lyapunov-based approaches [191], [192], barrier functions [193], and formal verification of safety constraints [194], [195].”
BackgroundBayesian Learning-Based Adaptive Control for Safety Critical Systems
Fan, D. D., Nguyen, J., Thakker, R., et al. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196709 · Cited by 85 · see citing works
Control Barrier Functions for Unknown Nonlinear Systems using Gaussian Processes
Jagtap, P., Pappas, G. J., & Zamani, M. (2020) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc42340.2020.9303847 · Cited by 82 · see citing works
“Assuming a prior knowledge of control barrier functions, the results in [19] and [20] provide the safe online learning of the Gaussian process model and safe learning of the reinforcement learning policy, respectively.”
BackgroundPhan, D. T., Grosu, R., Jansen, N., Paoletti, N., Smolka, S. A., & Stoller, S. D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-55754-6_6 · Cited by 82 · see citing works
“CBFs are also used in [7].”
A survey on physics informed reinforcement learning: Review and open problems
Banerjee, C., Nguyen, K., Fookes, C., & Raissi, M. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128166 · Cited by 85 · see citing works
“[19, 11] is made in recent safety critical RL applications.”
InfluentialExtended Dynamic Mode Decomposition with Learned Koopman Eigenfunctions for Prediction and Control
Folkestad, C., Pastor, D., Mezić, I., Mohr, R., Fonoberova, M., & Burdick, J. W. (2020) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc45564.2020.9147729 · Cited by 81 · see citing works
“Moreover, while safety during MFRL is now possible [8], [9], one cannot yet guarantee that learned policies will satisfy performance requirements or state and actuator limits.”
BackgroundSelf-citationSoft Tissue Simulation Environment to Learn Manipulation Tasks in Autonomous Robotic Surgery
Tagliabue, E., Pore, A., Dall’Alba, D., Magnabosco, E., Piccinelli, M., & Fiorini, P. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341710 · Cited by 79 · see citing works
“Although safe learning methods have been proposed, their exploitation in the surgical robotic field remains limited, due to the impracticality to acquire many training trials from real SRSs [3].”
BackgroundInfluentialOn reliability of reinforcement learning based production scheduling systems: a comparative survey
de Puiseau, C. W., Meyes, R., & Meisen, T. (2022) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-022-01915-2 · Cited by 68 · see citing works
“Some also separately measure the average penalty (Bohez et al., 2019) or the average and variance of constraint values (Cheng, Orosz, et al., 2019; Cheng, Verma, et al., 2019; Chow, Nachum, et al., 2018) or the cumulative constraint violation values (Dalal et al., 2012; Yang et al., 2020).”
BackgroundPore, A., Li, Z., Dall’Alba, D., Hernansanz, A., Momi, E. D., Menciassi, A., Casals, A., Dankelman, J., Fiorini, P., & Poorten, E. V. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3269384 · Cited by 77 · see citing works
“Somestudies haveproposedsafeRLframeworksforsafety-criticalparadigms usingbarrierfunctionstorestricttherobotactuationinasafe workspace[160],[180]anditsbehaviorsisformallyverified to guarantee safety [161], [162].”
Zhang, Y., Liang, X., Li, D., et al. (2023) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2023.3283771 · Cited by 76 · see citing works
“Therefore, for learning-based control incorporating such a safety-critical control system, safety constraints are required to be ensured even during the learning period [6], [7].”
BackgroundMethodLearning Safe Multi-Agent Control with Decentralized Neural Barrier CertificatesPreprint
Qin, Z., Zhang, K., Chen, Y., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.05436 · Cited by 75 · see citing works
“…safe control via CBFs, which primarily consist of two categories: learning CBFs from data (Saveriano & Lee, 2020; Srinivasan et al., 2020; Jin et al., 2020; Boffi et al., 2020), and CBF-based approach for controlling unknown systems (Wang et al., 2017; 2018; Cheng et al., 2019; Taylor et al., 2020).”
BackgroundActive learning in robotics: A review of control principles
Taylor, A., Berrueta, T. A., & Murphey, T. D. (2021) · Mechatronics · doi:10.1016/j.mechatronics.2021.102576 · Cited by 75 · see citing works
Safe Reinforcement Learning Using Robust Control Barrier Functions
Emam, Y., Notomista, G., Glotfelter, P., Kira, Z., & Egerstedt, M. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3216996 · Cited by 74 · see citing works
“The first baseline only differs from MF SAC-RCBF in that it does not utilize a differentiable safety layer, and the second baseline leverages the compensator from [10], which is a neural-net trained to learn the behavior of the RCBF-QP during training.”
InfluentialSafe Nonlinear Control Using Robust Neural Lyapunov-Barrier FunctionsPreprint
Dawson, C., Qin, Z., Gao, S., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.06697 · Cited by 72 · see citing works
“The most recent set of methods promising general-purpose controller synthesis come from the field of learning for control; for instance, using reinforcement learning [6, 7] or supervised learning [8, 9, 10, 11].”
BackgroundHan, M., Tian, Y., Zhang, L., Wang, Y.-X., & Pan, W. (2021) · Automatica · doi:10.1016/j.automatica.2021.109689 · Cited by 71 · see citing works
Verification and Synthesis of Control Barrier Functions
Clark, A. (2021) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc45484.2021.9683520 · Cited by 66 · see citing works
“CBFs have been generalized to high-degree [6], stochastic [7], [8], and uncertain [9], [10] systems, and applied in diverse domains such as bipedal locomotion [11], automotive control [12], [13], and UAVs [14], [15].”
BackgroundZeng, J., Zhang, B., Li, Z., et al. (2021) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc50511.2021.9482626 · Cited by 66 · see citing works
Control Barriers in Bayesian Learning of System Dynamics
Dhiman, V., Khojasteh, M. J., Franceschetti, M., et al. (2021) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2021.3137059 · Cited by 65 · see citing works
“The works in [52], [53], [55], [56] study time-triggered CBFbased control of relative-degree-one systems with additive uncertainties in the drift part of the dynamics.”
BackgroundXu, J., Wang, J., Rao, J., et al. (2021) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.5955 · Cited by 65 · see citing works
Impacts of Connected Automated Vehicles on Freeway Traffic Patterns at Different Penetration Levels
Avedisov, S. S., Bansal, G., & Orosz, G. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3043323 · Cited by 62 · see citing works
Self-citationShao, Y., Chen, C., Kousik, S., & Vasudevan, R. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3063989 · Cited by 62 · see citing works
“On the other hand, one can construct a Control Barrier Function (CBF) to serve as a backup controller to safely guide model learning [7].”
BackgroundModel-Free Safe Reinforcement Learning Through Neural Barrier Certificate
Yang, Y., Jiang, Y., Liu, Y., Chen, J., & Li, S. E. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3238656 · Cited by 62 · see citing works
“However, the limitations of these algorithms are that they either require strong assumptions of safety certificates [7], [16], [18], [19] or rely on environment models [17], [21], [22], [23], [24].”
BackgroundInfluentialConfidence-aware reinforcement learning for energy management of electrified vehicles
Wu, J., Huang, C., He, H., & Huang, H. (2023) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2023.114154 · Cited by 62 · see citing works
Navigating the Landscape of Deep Reinforcement Learning for Power System Stability Control: A Review
Massaoudi, M., Abu‐Rub, H., & Ghrayeb, A. (2023) · IEEE Access · doi:10.1109/access.2023.3337118 · Cited by 61 · see citing works
“RL algorithms do not require explicit modeling of the probability distributions or constraint sets of uncertainties [11].”
BackgroundDisturbance Observer-based Robust Control Barrier Functions
Wang, Y., & Xu, X. (2023) · 2023 American Control Conference (ACC) · doi:10.23919/acc55779.2023.10156095 · Cited by 64 · see citing works
“Recently, some robust CBF control schemes based on disturbance estimation and compensation techniques were proposed with the goal of reducing the conservatism of the re-lated safe controllers [8], [9], [10], [11], [12].”
BackgroundMethodAvoiding Dense and Dynamic Obstacles in Enclosed Spaces: Application to Moving in Crowds
Huber, L., Slotine, J.-J. E., & Billard, A. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3164789 · Cited by 57 · see citing works
“Recent works have used such control barrier function in the context of safe learning by demonstration [44], [45] or reinforcement learning [46].”
BackgroundYin, Y., Chen, Z., Liu, G., Yin, J., & Guo, J. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123202 · Cited by 57 · see citing works
Guaranteeing Safety of Learned Perception Modules via Measurement-Robust Control Barrier FunctionsPreprint
Dean, S., Taylor, A. J., Cosner, R. K., Recht, B., & Ames, A. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.16001 · Cited by 55 · see citing works
“Furthermore, the integration of CBFs with machine-learning approaches for reducing model uncertainty has shown great promise theoretically [7, 8, 9] and in application [10, 11].”
BackgroundSafe-State Enhancement Method for Autonomous Driving via Direct Hierarchical Reinforcement Learning
Gu, Z., Gao, L., Ma, H., Li, S. E., Zheng, S., Jing, W., & Chen, J. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3271642 · Cited by 55 · see citing works
“[42], they still fall back to the model-based setting, lacking effectiveness in model-free environments.”
BackgroundPeng, C., Liu, X., & Ma, J. (2023) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2023.3288826 · Cited by 52 · see citing works
“Some approaches learn model uncertainty in the dynamics through high-probabilistic models, such as Gaussian processes (GPs) [30], [33].”
BackgroundMethodHan, S., Zhou, S., Wang, J., Pepin, L., Ding, C., Fu, J., & Miao, F. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3336670 · Cited by 53 · see citing works
Episodic Learning with Control Lyapunov Functions for Uncertain Robotic Systems
Taylor, A. J., Dorobantu, V. D., Le, H., Yue, Y., & Ames, A. D. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967820 · Cited by 72 · see citing works
Neural Certificates for Safe Control PoliciesPreprint
Jin, W., Wang, Z., Yang, Z., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08465 · Cited by 52 · see citing works
“In the learning field, barrier functions are recently employed to guarantee safety in learning of a policy or dynamics, such as [18–20].”
MethodConstrained model-free reinforcement learning for process optimization
Pan, E., Petsagkourakis, P., Mowbray, M., et al. (2021) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2021.107462 · Cited by 51 · see citing works
“Such augmented systems are broadly constituted by barrier function [7, 38, 6] and safety filter methods [10, 41, 46].”
BackgroundStability-Guaranteed Reinforcement Learning for Contact-Rich Manipulation
Khader, S. A., Yin, H., Falco, P., & Kragić, D. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3028529 · Cited by 49 · see citing works
“The method in [29] also uses GP and is further limited to learning only the unactuated part of the dynamics.”
MethodLearning from Demonstrations for Autonomous Soft-tissue Retraction
Pore, A., Tagliabue, E., Piccinelli, M., Dall’Alba, D., Casals, A., & Fiorini, P. (2021) · International Symposium on Medical Robotics · doi:10.1109/ismr48346.2021.9661514 · Cited by 48 · see citing works
“Hence, in subsequent work, we will incorporate safety constraints through a Safe-Reinforcement learning technique [22].”
MethodVerifiably safe exploration for end-to-end reinforcement learning
Hunt, N., Fulton, N., Magliacane, S., Hoang, T. N., Das, S., & Solar-Lezama, A. (2021) · International Conference on Hybrid Systems: computation and control · doi:10.1145/3447928.3456653 · Cited by 48 · see citing works
“Many recent safe exploration methods focus on safety guarantees that hold in expectation (e.g., (Schulman et al., 2015; Achiam et al., 2017)) or with high probability (e.g., (Berkenkamp et al., 2017; Dalal et al., 2018; Koller et al., 2018; Cheng et al., 2019).”
Ensuring Safety of Learning-Based Motion Planners Using Control Barrier Functions
Wang, X. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3152313 · Cited by 47 · see citing works
“Although the approaches in [6]–[10] have shown good performance in their experimental setups, they are based on simplified models, simplified assumptions, or simplified safety constraints, thus they cannot be applied to general highway scenarios directly.”
InfluentialHu, B., Xiao, Y., Zhang, S., & Liu, B. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3213026 · Cited by 46 · see citing works
“to safety critical EMS applications [20].”
Kochdumper, N., Krasowski, H., Wang, X., Bak, S., & Althoff, M. (2023) · IEEE Open Journal of Control Systems · doi:10.1109/ojcsys.2023.3256305 · Cited by 47 · see citing works
“for action projection are control barrier functions [21], [22], model predictive control [23], [24], and parameterized reachability analysis [25].”
BackgroundRobust Safety under Stochastic Uncertainty with Discrete-Time Control Barrier Functions
Cosner, R. K., Culbertson, P., Taylor, A., et al. (2023) · doi:10.15607/rss.2023.xix.084 · Cited by 44 · see citing works
Sim-to-Lab-to-Real: Safe reinforcement learning with shielding and generalization guarantees
Hsu, K., Ren, A. Z., Nguyen, D. P., et al. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2022.103811 · Cited by 43 · see citing works
“[26] propose a similar shielding framework, utilizing the related control barrier function (CBF) concept.”
MethodA Control Barrier Perspective on Episodic Learning via Projection-to-State Safety
Taylor, A. J., Singletary, A., Yue, Y., et al. (2020) · IEEE Control Systems Letters · doi:10.1109/lcsys.2020.3009082 · Cited by 43 · see citing works
“Due to its flexibility, it is increasingly popular to incorporate learning into safe controller synthesis [7], [8], [16], [23].”
BackgroundModel-free Safe Control for Zero-Violation Reinforcement Learning
Zhao, W., He, T., & Liu, C. (2021) · Cited by 41 · see citing works
Formal synthesis of controllers for safety-critical autonomous systems: Developments and challenges
Yin, X., Gao, B., & Yu, X. P. (2024) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2024.100940 · Cited by 40 · see citing works
“To address this issue, safe RL techniques, such as online shields Alshiekh et al. (2018), Cheng et al. (2019), Carr et al. (2023), can be further leveraged to guarantee safety during exploration.”
BackgroundChen, X., Qu, G., Tang, Y., Low, S. H., & Li, N. (2021) · CaltechAUTHORS (California Institute of Technology) · Cited by 42 · see citing works
Stabilizing Neural Control Using Self-Learned Almost Lyapunov Critics
Chang, Y.-C., & Gao, S. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560886 · Cited by 42 · see citing works
“There has been exciting progress towards introducing control-theoretic approaches for enhancing stability in reinforcement learning and imitation learning, such as using Lyapunov methods [1]–[8], control barrier functions [9]–[11], or control-theoretic regularization [12]– [14].”
BackgroundLiu, S., Liu, L., & Yu, Z. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.11.006 · Cited by 39 · see citing works
Tang, C., & Lai, Y.-C. (2020) · International Conference on Unmanned Aircraft Systems (ICUAS) · doi:10.1109/icuas48674.2020.9213987 · Cited by 39 · see citing works
“Many new different training methods, such as dynamic weights of reward have also been proposed [27, 28], providing new prospects to the field of this study.”
BackgroundLearning safe neural network controllers with barrier certificates
Zhao, H., Zeng, X. L., Chen, T., Liu, Z., & Woodcock, J. (2021) · Formal Aspects of Computing · doi:10.1007/s00165-021-00544-5 · Cited by 38 · see citing works
“A verificationin-the-loop RL algorithm was proposed in [8] to learn safe NN controllers for known system dynamics using CBFs; an end-to-end safe RL architecture was developed by combining model-free RL control, model-based CBF control, and model learning in [5]; CLFs and CBFs are integrated into the episodic learning…”
BackgroundMethodAdaptive Safety with Multiple Barrier Functions Using Integral Concurrent Learning
Isaly, A., Patil, O. S., Sanfelice, R. G., et al. (2021) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc50511.2021.9483208 · Cited by 38 · see citing works
“The idea of using an estimate of the unknown dynamics to reduce conservativeness in a BF-induced input constraint was also explored in [17] and [18], where the authors used Gaussian processes (GP).”
MethodCan You Trust Your Autonomous Car? Interpretable and Verifiably Safe Reinforcement Learning
Schmidt, L. M., Kontes, G. D., Plinge, A., & Mutschler, C. (2021) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv48863.2021.9575328 · Cited by 39 · see citing works
“Recent work [19], [23] augments the generated policies with components that can reject actions if they are deemed unsafe, building on previous work in general RL research [24], [25].”
BackgroundExploration-Exploitation in Constrained MDPsPreprint
Efroni, Y., Mannor, S., & Pirotta, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.02189 · Cited by 38 · see citing works
“Other approaches use Gaussian processes to model the dynamics and/or the value function [Berkenkamp et al., 2017, Wachi et al., 2018, Koller et al., 2018, Cheng et al., 2019] in order to be able to estimate the constraints and (approximately) guarantee safety over learning.”
MethodHu, Y., Fu, J., & Wen, G. (2022) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2022.3203928 · Cited by 37 · see citing works
“As a vanilla extension of the approaches in [5] and [29], Cheng et al.”
BackgroundMethodInfluentialEpisodic Learning with Control Lyapunov Functions for Uncertain Robotic Systems
Taylor, A. J., Dorobantu, V. D., Le, H., Yue, Y., & Ames, A. D. (2019) · CaltechAUTHORS (California Institute of Technology) · Cited by 36 · see citing works
Zhao, T., Wang, J., & Yue, M. (2023) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2022.3233770 · Cited by 36 · see citing works
“Inspired by [28], this work formulates a barrier-certification system to guide the RL control.”
BackgroundMethodResultInfluentialSafe Learning for Control using Control Lyapunov Functions and Control Barrier Functions: A Review
Anand, A. S., Seel, K., Gjærum, V. B., et al. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.09.173 · Cited by 35 · see citing works
Online shielding for reinforcement learning
Könighofer, B., Rudolf, J., Palmisano, A., et al. (2022) · Innovations in Systems and Software Engineering · doi:10.1007/s11334-022-00480-4 · Cited by 35 · see citing works
“In the continuous domain, model-based approaches have utilised Lyapunov-based methods [8] and control-barrier functions [7, 28] to enable safe learning under continuous system dynamics.”
MethodFormal Verification of Unknown Discrete- and Continuous-Time Systems: A Data-Driven Approach
Nejati, A., Lavaei, A., Jagtap, P., et al. (2023) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2023.3255141 · Cited by 35 · see citing works
Converse Barrier Functions via Lyapunov Functions
Liu, J. (2021) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2021.3085419 · Cited by 35 · see citing works
Robust Regression for Safe Exploration in ControlPreprint
Liu, A., Shi, G., Chung, S., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05819 · Cited by 35 · see citing works
“One could blend GP-based modeling with general function approximations (such as deep learning) (Berkenkamp et al., 2017; Cheng et al., 2019a), but the resulting optimization-based control problem can be challenging to solve.”
BackgroundA survey of demonstration learning
Correia, A., & Alexandre, L. A. (2024) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2024.104812 · Cited by 34 · see citing works
Towards Robust Data-Driven Control Synthesis for Nonlinear Systems with Actuation Uncertainty
Taylor, A. J., Dorobantu, V. D., Dean, S., Recht, B., Yue, Y., & Ames, A. D. (2021) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc45484.2021.9683511 · Cited by 36 · see citing works
“and machine learning based approaches have shown great promise for controlling systems with an uncertain model or with no model at all (Kober et al., 2013; Shi et al., 2019; Cheng et al., 2019; Lee et al., 2020).”
Learning Robust Output Control Barrier Functions From Safe Expert Demonstrations
Lindemann, L., Robey, A., Jiang, L., Das, S., Tu, S., & Matni, N. (2024) · IEEE Open Journal of Control Systems · doi:10.1109/ojcsys.2024.3385348 · Cited by 33 · see citing works
“Further work in this direction can be found in [27], [28], [29].”
BackgroundHakobyan, A., & Yang, I. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3200156 · Cited by 32 · see citing works
“5], μ̃ t,k,2 y = [8, 6] and Σ̃ y = diag[0.”
Barrier Function-based Safe Reinforcement Learning for Emergency Control of Power Systems
Vu, T. L., Mukherjee, S., Huang, R., et al. (2021) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc45484.2021.9683573 · Cited by 32 · see citing works
“In a related work [10], model-based control Barrier Lyapunov function was used to design a control to compensate for”
MethodWang, Y., & Chapman, M. P. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2022.103743 · Cited by 32 · see citing works
“The class of policies is constrained using control barrier functions, which generalize Lyapunov functions, in [129, 130].”
BackgroundSablas: Learning Safe Control for Black-Box Dynamical Systems
Qin, Z., Sun, D., & Fan, C. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3142743 · Cited by 32 · see citing works
“Control Barrier Function [11], [12], [13], [14], [15], [16], [17], [39] ensures the existence of a controller that keep the system inside a safe invariant set.”
BackgroundTrainify: A CEGAR-Driven Training and Verification Framework for Safe Deep Reinforcement Learning
Jin, P., Tian, J., Zhi, D., Wen, X., & Zhang, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-13185-1_10 · Cited by 32 · see citing works
SRL-ORCA: A Socially Aware Multi-Agent Mapless Navigation Algorithm in Complex Dynamic Scenes
Qin, J., Qin, J., Qiu, J., Liu, Q., Li, M., & Ma, Q. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3331621 · Cited by 33 · see citing works
“Thestudyofsafereinforcement learning(SRL)[4]iscriticalinDRLnavigation[5],[6],[7],[8].”
BackgroundCao, H., Xiong, H., Zeng, W., et al. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3330183 · Cited by 31 · see citing works
“Safe RL approaches based on integrating RL with shields based on CBFs have been proposed in [26], [36], [37], [38], and [39].”
BackgroundMethodInfluentialScalable Learning of Safety Guarantees for Autonomous Systems using Hamilton-Jacobi Reachability
Herbert, S., Choi, J. J., Sanjeev, S., Gibson, M., Sreenath, K., & Tomlin, C. J. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561561 · Cited by 31 · see citing works
Safe and Stable RL (S2RL) Driving Policies Using Control Barrier and Control Lyapunov Functions
Gangopadhyay, B., Dasgupta, P., & Dey, S. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3160202 · Cited by 33 · see citing works
Convex synthesis and verification of control-Lyapunov and barrier functions with input constraints
Dai, H., & Permenter, F. (2023) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc55779.2023.10156043 · Cited by 31 · see citing works
“Recently, they have been used to guide learning-based methods [8], [9], [17].”
MethodTemporal Logic Guided Safe Reinforcement Learning Using Control Barrier FunctionsPreprint
Li, X., & Belta, C. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.09885 · Cited by 34 · see citing works
“The authors of [17] and [18] utilize control barrier functions to safeguard exploration.”
BackgroundControl for smart systems: Challenges and trends in smart cities
Jia, Q., Panetto, H., Macchi, M., Siri, S., Weichhart, G., & Xu, Z. (2022) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2022.04.010 · Cited by 30 · see citing works
Safe Reinforcement Learning Using Black-Box Reachability Analysis
Selim, M., Alanwar, A., Kousik, S., et al. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3192205 · Cited by 30 · see citing works
“One can also combine data-driven approaches with model-free barrier or intervention functions [22]–[24], or use robust CBFs to model uncertain parts of a system [25], [26].”
BackgroundAutonomous Driving Based on Approximate Safe Action
Wang, X., Zhang, J., Hou, D., et al. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3292253 · Cited by 30 · see citing works
“For example, some model-based safe exploration methods use Lyapunov function or control barrier function to guide the policy optimization within a safe set [31], [32], [33].”
Method
- Authors
- Cheng, R., Orosz, G., Murray, R. M., & Burdick, J. W.
- Affiliations
- California Institute of Technology (US)
- University of Michigan (US)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2019-07-17
- Volume
- 33
- Issue
- 01
- Pages
- 3387-3395
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2966735560
- Semantic Scholar
- adcd4bfd88213da0c33d3cb7057411dd15b72c7a
- Cited by
- 611
- References
- 34
- Citations by year
- 2026: 79 · 2025: 84 · 2024: 79 · 2023: 102 · 2022: 84 · 2021: 112 · 2020: 60 · 2019: 10 · 2018: 1
- FWCI
- 25.44
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- www.aaai.org/ (unspecified)
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- CaltechAUTHORS (California Institute of Technology) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Adaptive Dynamic Programming Control
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Formal Methods in Verification
- Keywords
- safe reinforcement learning, control barrier functions, safety-critical control, continuous control, model-based control, policy exploration, sample efficiency, inverted pendulum, vehicle-to-vehicle communication
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
