The DOI matches its Crossref record; 124 of the 124 references it cites were found in a registry or an index.
A Review of Safe Reinforcement Learning: Methods, Theories, and Applications
Open accessAbstract
Reinforcement Learning (RL) has achieved tremendous success in many complex decision-making tasks. However, safety concerns are raised during deploying RL in real-world applications, leading to a growing demand for safe RL algorithms, such as in autonomous driving and robotics scenarios. While safe control has a long history, the study of safe RL algorithms is still in the early stages. To establish a good foundation for future safe RL research, in this paper, we provide a review of safe RL from the perspectives of methods, theories, and applications. First, we review the progress of safe RL from five dimensions and come up with five crucial problems for safe RL being deployed in real-world applications, coined as "2H3W". Second, we analyze the algorithm and theory progress from the perspectives of answering the "2H3W" problems. Particularly, the sample complexity of safe RL algorithms is reviewed and discussed, followed by an introduction to the applications and benchmarks of safe RL algorithms. Finally, we open the discussion of the challenging problems in safe RL, hoping to inspire future research on this thread. To advance the study of safe RL algorithms, we release an open-sourced repository containing major safe RL algorithms at the link.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
A Stochastic Approximation Method
Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643
Geurts, P., Ernst, D., & Wehenkel, L. (2006) · Machine Learning · doi:10.1007/s10994-006-6226-1 · Cited by 9,300
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Constrained model predictive control: Stability and optimality
Mayne, D. Q., Rawlings, J. B., Rao, C. V., & Scokaert, P. O. M. (2000) · Automatica · doi:10.1016/s0005-1098(99)00214-9 · Cited by 8,736 · cited 1×
“Inthis section,wewillinvestigatemodel-basedsafeRLmethodsfrom differentperspectives:policyoptimization-basedapproaches, controltheory-basedapproaches,formalmethods-basedap-proaches, and Gaussian process-based approaches.”
Recent advances in convolutional neural networks
Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., et al. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.013 · Cited by 6,576
Soft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594 · cited 1×
“In contrast to [127], [128] not just use CBFs, but also robust CBFs [131], they develop a safe RL method based on SAC [132] and robust CBFs. Specifically, they consider safety in robust settings by estimating the disturbed dynamics systems with GP models.”
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767
Microscopic Traffic Simulation using SUMO
Lopez, P. A., Wiessner, E., Behrisch, M., Bieker-Walz, L., Erdmann, J., Flotterod, Y.-P., Hilbrich, R., Lucken, L., Rummel, J., & Wagner, P. (2018) · 2018 21st International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569938 · Cited by 3,297
A Survey of Monte Carlo Tree Search Methods
Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., & Colton, S. (2012) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2012.2186810 · Cited by 2,970 · cited 2×
“…method [63] using CBFs, Cao et al. [212] improve the safety of autonomous driving in low-level decision-making settings by integrating a rule-based policy, e.g., a Gipps car-following model [213], into RLframework,andaMonteCarlotreesearchingmethod[214]is usedtogeneratetheirRLframeworkpolicies.”
A behavioural car-following model for computer simulation
Gipps, P. G. (1981) · Transportation Research Part B: Methodological · doi:10.1016/0191-2615(81)90037-0 · Cited by 2,357
Constructive Nonlinear Control
Sepulchre, R., Janković, M., & Kokotović, P. V. (1997) · Communications and Control Engineering · doi:10.1007/978-1-4471-0967-9 · Cited by 2,297
Control Barrier Functions: Theory and Applications
Ames, A. D., Coogan, S., Egerstedt, M., Notomista, G., Sreenath, K., & Tabuada, P. (2019) · 2019 18th European Control Conference (ECC) · doi:10.23919/ecc.2019.8796030 · Cited by 1,936
Temporal difference learning and TD-Gammon
Tesauro, G. (1995) · Communications of the ACM · doi:10.1145/203330.203343 · Cited by 1,618 · cited 1×
“For example, TD learning is used to solve problems including backgammon playing [200], [201], job-shop scheduling and elevator group control [202]; a stochastic approximation algorithm with RL properties is utilized to solve pricing options for financial derivatives in two-player and zero-sum games…”
Krajewski, R., Bock, J., Kloeker, L., & Eckstein, L. (2018) · 2018 21st International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569552 · Cited by 1,401
The Problem of Abortion and the Doctrine of the Double Effect
Foot, P. (2002) · Virtues and Vices · doi:10.1093/0199252866.003.0002 · Cited by 1,364
Killing, Letting Die, and the Trolley Problem
Thomson, J. J., & The Hegeler Institute (1976) · Monist · doi:10.5840/monist197659224 · Cited by 1,341
Concrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
Zhang, K., Yang, Z., & Başar, T. (2021) · Studies in Systems, Decision and Control · doi:10.1007/978-3-030-60990-0_12 · Cited by 1,264
An Introduction to Numerical Analysis
Süli, E., & Mayers, D. F. (2003) · Cambridge University Press eBooks · doi:10.1017/cbo9780511801181 · Cited by 1,255
Envelope Theorems for Arbitrary Choice Sets
Milgrom, P., & Segal, I. (2002) · Econometrica · doi:10.1111/1468-0262.00296 · Cited by 1,235
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · Nature · doi:10.1038/s41586-020-03051-4 · Cited by 1,214 · cited 1×
“In video compression, Mandhane et al. [70] leverage the Muzero [237], one of the alpha series algorithms, to solve the safe RL problem in video compression.”
(2015) · Journal of Machine Learning Research · Cited by 1,119
TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play
Tesauro, G. (1994) · Neural Computation · doi:10.1162/neco.1994.6.2.215 · Cited by 787
Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning
Brunke, L., Greeff, M., Hall, A. W., Yuan, Z., Zhou, S., Panerati, J., & Schoellig, A. P. (2022) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-042920-020211 · Cited by 769
An analog of the minimax theorem for vector payoffs
Blackwell, D. (1956) · Pacific Journal of Mathematics · doi:10.2140/pjm.1956.6.1 · Cited by 740
(2018) · Springer eBooks · doi:10.1007/978-3-319-10575-8 · Cited by 725
Kendall, A., Hawke, J., Janz, D., Mazur, P., Reda, D., Allen, J.-M., Lam, V.-D., Bewley, A., & Shah, A. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793742 · Cited by 694 · cited 2×
“Kendall et al. [60] provides a model-free based RL method, which is combined by variational autoencoder [219] and DDPG [220].”
Robust Control of Markov Decision Processes with Uncertain Transition Matrices
Nilim, A., & El Ghaoui, L. (2005) · Operations Research · doi:10.1287/opre.1050.0216 · Cited by 694
Challenges of real-world reinforcement learning: definitions, benchmarks and analysis
Dulac-Arnold, G., Levine, N., Mankowitz, D. J., Li, J., Paduraru, C., Gowal, S., & Hester, T. (2021) · Machine Learning · doi:10.1007/s10994-021-05961-4 · Cited by 645
Cheng, R., Orosz, G., Murray, R. M., & Burdick, J. W. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013387 · Cited by 611 · cited 1×
“Moreover,[158]leveragesamodel-free RLandCBFstoguaranteelearningsafetywhileimproving learningefficiency,inwhichCBFsareusedtoconstrainthe searchspacetoenhancelearningsafetyandefficiencybased onaGPlearningmodel.”
Autonomous Helicopter Aerobatics through Apprenticeship Learning
Abbeel, P., Coates, A., & Ng, A. Y. (2010) · The International Journal of Robotics Research · doi:10.1177/0278364910371999 · Cited by 584
Cautious Model Predictive Control Using Gaussian Process Regression
Hewing, L., Kabzan, J., & Zeilinger, M. N. (2020) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2019.2949757 · Cited by 577
Provably safe and robust learning-based model predictive control
Aswani, A., Gonzalez, H., Sastry, S. S., & Tomlin, C. (2013) · Automatica · doi:10.1016/j.automatica.2013.02.003 · Cited by 539
Risk-Sensitive Markov Decision Processes
Howard, R. A., & Matheson, J. E. (1972) · Management Science · doi:10.1287/mnsc.18.7.356 · Cited by 501
Relative Entropy Policy Search
Peters, J., Mulling, K., & Altun, Y. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7727 · Cited by 455
A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems
Fisac, J. F., Akametalu, A. K., Zeilinger, M. N., Kaynama, S., Gillula, J., & Tomlin, C. J. (2019) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2018.2876389 · Cited by 450
Constrained EV Charging Scheduling Based on Safe Deep Reinforcement Learning
Li, H., Wan, Z., & He, H. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2955437 · Cited by 428
Cooperative Inverse Reinforcement Learning
Hadfield-Menell, D., Dragan, A., Abbeel, P., & Russell, S. (2016) · arXiv · doi:10.48550/arxiv.1606.03137 · Cited by 413
Safe Reinforcement Learning Using Robust MPC
Zanon, M., & Gros, S. (2021) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2020.3024161 · Cited by 338
Atakishiyev, S., Salameh, M., Yao, H., & Goebel, R. (2021) · arXiv · doi:10.48550/arxiv.2112.11561 · Cited by 336
Tsitsiklis, J. N., & van Roy, B. (1999) · IEEE Transactions on Automatic Control · doi:10.1109/9.793723 · Cited by 326
Atakishiyev, S., Salameh, M., Yao, H., & Goebel, R. (2024) · IEEE Access · doi:10.1109/access.2024.3431437 · Cited by 323
Learning-Based Model Predictive Control for Safe Exploration
Koller, T., Berkenkamp, F., Turchetta, M., & Krause, A. (2018) · 2018 IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2018.8619572 · Cited by 318
YouTube UGC Dataset for Video Compression Research
Wang, Y., Inguva, S., & Adsumilli, B. (2019) · 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2019.8901772 · Cited by 282
Self-citationElevator Group Control Using Multiple Reinforcement Learning Agents
Crites, R. H., & Barto, A. G. (1998) · Machine Learning · doi:10.1023/a:1007518724497 · Cited by 275
Safe reinforcement learning: A control barrier function optimization approach
Marvi, Z., & Kiumarsi, B. (2021) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.5132 · Cited by 254
Reachability-based safe learning with Gaussian processes
Akametalu, A. K., Fisac, J. F., Gillula, J. H., Kaynama, S., Zeilinger, M. N., & Tomlin, C. J. (2014) · 53rd IEEE Conference on Decision and Control · doi:10.1109/cdc.2014.7039601 · Cited by 249
Distributed Coordination Control for Multi-Robot Networks Using Lyapunov-Like Barrier Functions
Panagou, D., Stipanovic, D. M., & Voulgaris, P. G. (2016) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2015.2444131 · Cited by 248
Dynamic stochastic electric vehicle routing with safe reinforcement learning
Basso, R., Kulcsár, B., Sánchez-Díaz, I., & Qu, X. (2021) · Transportation Research Part E Logistics and Transportation Review · doi:10.1016/j.tre.2021.102496 · Cited by 217
Multi-criteria Reinforcement Learning
Gábor, Z., Kalmár, Z., & Szepesvári, C. (1998) · Cited by 205
Mirchevska, B., Pek, C., Werling, M., Althoff, M., & Boedecker, J. (2018) · 2018 21st International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569448 · Cited by 198
Choi, J., Castañeda, F., Tomlin, C., & Sreenath, K. (2020) · Robotics: Science and Systems XVI · doi:10.15607/rss.2020.xvi.088 · Cited by 194
The review unmanned surface vehicle path planning: Based on multi-modality constraint
Zhou, C., Gu, S., Wen, Y., Du, Z., Xiao, C., Huang, L., & Zhu, M. (2020) · Ocean Engineering · doi:10.1016/j.oceaneng.2020.107043 · Cited by 190
Self-citationSafe Reinforcement Learning via Formal Methods: Toward Safe Control Through Proof and Learning
Fulton, N., & Platzer, A. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12107 · Cited by 188
DEAR: Deep Reinforcement Learning for Online Advertising Impression in Recommender Systems
Zhao, X., Gu, C., Zhang, H., Yang, X., Liu, X., Tang, J., & Liu, H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i1.16156 · Cited by 185
Stochastic Successive Convex Approximation for Non-Convex Constrained Stochastic Optimization
Liu, A., Lau, V. K. N., & Kananian, B. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2925601 · Cited by 179
Consideration of Risk in Reinforcement Learning
Heger, M. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50021-0 · Cited by 174
An actor-critic algorithm for constrained Markov decision processes
Borkar, V. S. (2005) · Systems & Control Letters · doi:10.1016/j.sysconle.2004.08.007 · Cited by 172
Safe Reinforcement Learning With Model Uncertainty Estimates
Lutjens, B., Everett, M., & How, J. P. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793611 · Cited by 171
Safe and robust learning control with Gaussian processes
Berkenkamp, F., & Schoellig, A. P. (2015) · 2015 European Control Conference (ECC) · doi:10.1109/ecc.2015.7330913 · Cited by 169
Safe multi-agent reinforcement learning for multi-robot control
Gu, S., Kuba, J. G., Chen, Y., Du, Y., Yang, L., Knoll, A., & Yang, Y. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103905 · Cited by 167
Self-citationSafe Reinforcement Learning Using Robust Control Barrier Functions
Emam, Y., Notomista, G., Glotfelter, P., Kira, Z., & Egerstedt, M. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3216996 · Cited by 166
IPO: Interior-Point Policy Optimization under Constraints
Liu, Y., Ding, J., & Liu, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5932 · Cited by 163
Reinforcement Learning-Based Mobile Offloading for Edge Computing Against Jamming and Interference
Xiao, L., Lu, X., Xu, T., Wan, X., Ji, W., & Zhang, Y. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3007742 · Cited by 160
PAC Bounds for Discounted MDPs
Lattimore, T., & Hütter, M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34106-9_26 · Cited by 156
Safe Exploration in Markov Decision Processes
Moldovan, T. M., & Abbeel, P. (2012) · arXiv · doi:10.48550/arxiv.1205.4810 · Cited by 152 · cited 1×
“Fromtheperspectiveofknowledgeutilization, somemethodsconsidersafetyintheRLexplorationprocessby leveragingexternalknowledge,e.g.,[34],[100],[101],[102], [103],[104],[105].”
Minimax PAC bounds on the sample complexity of reinforcement learning with a generative model
Azar, M. G., Munos, R., & Kappen, H. J. (2013) · Machine Learning · doi:10.1007/s10994-013-5368-1 · Cited by 149
Safe Exploration of State and Action Spaces in Reinforcement Learning
García, J., & Fernandez, F. (2012) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3761 · Cited by 144
OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World
Pham, T.-H., Magistris, G. D., & Tachibana, R. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460547 · Cited by 140
Inequity aversion improves cooperation in intertemporal social dilemmas
Hughes, E., Leibo, J. Z., Phillips, M. G., Tuyls, K., Duéñez-Guzmán, E. A., Castañeda, A. G., Dunning, I., Zhu, T., McKee, K. R., Koster, R., et al. (2018) · arXiv · doi:10.48550/arxiv.1803.08884 · Cited by 134
Safe Reinforcement Learning on Autonomous Vehicles
Isele, D., Nakhaei, A., & Fujimura, K. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593420 · Cited by 127
An inductive synthesis framework for verifiable reinforcement learning
Zhu, H., Xiong, Z., Magill, S., & Jagannathan, S. (2019) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/3314221.3314638 · Cited by 127
Q-Learning for Risk-Sensitive Control
Borkar, V. S. (2002) · Mathematics of Operations Research · doi:10.1287/moor.27.2.294.324 · Cited by 126
Minimax Regret Bounds for Reinforcement Learning
Azar, M. G., Osband, I., & Munos, R. (2017) · arXiv · doi:10.48550/arxiv.1703.05449 · Cited by 124
A Teaching Method for Reinforcement Learning
Clouse, J. A., & Utgoff, P. E. (1992) · Elsevier eBooks · doi:10.1016/b978-1-55860-247-2.50017-6 · Cited by 123
Yuan, Z., Hall, A. W., Zhou, S., Brunke, L., Greeff, M., Panerati, J., & Schoellig, A. P. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3196132 · Cited by 123
Scalable agent alignment via reward modeling: a research direction
Leike, J., Krueger, D., Everitt, T., Martic, M., Maini, V., & Legg, S. (2018) · arXiv · doi:10.48550/arxiv.1811.07871 · Cited by 121 · cited 1×
“The fourth type of safety definition: agents are considered to be “safe” by some research [6], [7], [8] if “they act, reason, and generalize obeying human desires”.”
Policy Learning with Constraints in Model-free Reinforcement Learning: A Survey
Liu, Y., Halev, A., & Liu, X. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/614 · Cited by 113
Policy Gradients with Variance Related Risk CriteriaPreprint
Castro, D. D., Tamar, A., & Mannor, S. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6404 · Cited by 108
Safe exploration in model-based reinforcement learning using control barrier functions
Cohen, M. H., & Belta, C. A. (2022) · Automatica · doi:10.1016/j.automatica.2022.110684 · Cited by 107
Trustworthy safety improvement for autonomous driving using reinforcement learning
Cao, Z. L., Xu, S., Jiao, X., Peng, H., & Yang, D. (2022) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2022.103656 · Cited by 102
Safe Exploration and Optimization of Constrained MDPs Using Gaussian Processes
Wachi, A., Sui, Y., Yue, Y., & Ono, M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12103 · Cited by 99
Unsolved Problems in ML Safety
Hendrycks, D., Carlini, N., Schulman, J., & Steinhardt, J. (2021) · arXiv · doi:10.48550/arxiv.2109.13916 · Cited by 95
Pek, C., Zahn, P. B., & Althoff, M. (2017) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2017.7995918 · Cited by 92
Open-Sourced Reinforcement Learning Environments for Surgical Robotics
Richter, F., Orosco, R. K., & Yip, M. C. (2019) · arXiv · doi:10.48550/arxiv.1903.02090 · Cited by 92
Han, M., Tian, Y., Zhang, L., Wang, Y.-X., & Pan, W. (2021) · Automatica · doi:10.1016/j.automatica.2021.109689 · Cited by 89
Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Primal-Dual Approach
Bai, Q., Bedi, A. S., Agarwal, M., Koppel, A., & Aggarwal, V. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i4.20281 · Cited by 86
UAV Anti-Jamming Video Transmissions With QoE Guarantee: A Reinforcement Learning-Based Approach
Xiao, L., Ding, Y., Huang, J., Liu, S., Tang, Y., & Dai, H. (2021) · IEEE Transactions on Communications · doi:10.1109/tcomm.2021.3087787 · Cited by 84
Computing VaR and CVaR using stochastic approximation and adaptive unconstrained importance sampling
Bardou, O., Frikha, N., & Pagès, G. (2009) · Monte Carlo Methods and Applications · doi:10.1515/mcma.2009.011 · Cited by 82
Safe Reinforcement Learning for Autonomous Vehicles through Parallel Constrained Policy Optimization
Wen, L., Duan, J., Li, S. E., Xu, S., & Peng, H. (2020) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc45102.2020.9294262 · Cited by 78
An empirical investigation of the challenges of real-world reinforcement learning
Dulac-Arnold, G., Levine, N., Mankowitz, D. J., Li, J., Paduraru, C., Gowal, S., & Hester, T. (2020) · arXiv · doi:10.48550/arxiv.2003.11881 · Cited by 78
Bhatnagar, S., & Lakshmanan, K. S. (2012) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-012-9989-5 · Cited by 77
Verifiably safe exploration for end-to-end reinforcement learning
Hunt, N., Fulton, N., Magliacane, S., Hoang, T. N., Das, S., & Solar-Lezama, A. (2021) · International Conference on Hybrid Systems: computation and control · doi:10.1145/3447928.3456653 · Cited by 77
Teaching a humanoid robot to walk faster through Safe Reinforcement Learning
García, J., & Shafie, D. (2019) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2019.103360 · Cited by 76
Safe Reinforcement Learning for Autonomous Lane Changing Using Set-Based Prediction
Krasowski, H., Wang, X., & Althoff, M. (2020) · 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc45102.2020.9294259 · Cited by 76
Lu, X., Xiao, L., Niu, G., Ji, X., & Wang, Q. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3149396 · Cited by 74
Irving, G., Christiano, P., & Amodei, D. (2018) · arXiv · doi:10.48550/arxiv.1805.00899 · Cited by 71
Model-Free Safe Reinforcement Learning Through Neural Barrier Certificate
Yang, Y., Jiang, Y., Liu, Y., Chen, J., & Li, S. E. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3238656 · Cited by 69
TD Algorithm for the Variance of Return and Mean-Variance Reinforcement Learning
Sato, M., Kimura, H., & Kobayashi, S. (2001) · Transactions of the Japanese Society for Artificial Intelligence · doi:10.1527/tjsai.16.353 · Cited by 66
Agents teaching agents: a survey on inter-agent transfer learning
da Silva, F. L., Warnell, G., Costa, A. H. R., & Stone, P. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09430-0 · Cited by 64
Ensuring Safety of Learning-Based Motion Planners Using Control Barrier Functions
Wang, X. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3152313 · Cited by 63
SAMBA: safe model-based & active reinforcement learning
Cowen-Rivers, A. I., Palenicek, D., Moens, V., Abdullah, M. A., Sootla, A., Wang, Y.-X., & Ammar, H. B. (2022) · Machine Learning · doi:10.1007/s10994-021-06103-6 · Cited by 56
A Learning Algorithm for Risk-Sensitive Cost
Basu, A., Bhattacharyya, T., & Borkar, V. S. (2008) · Mathematics of Operations Research · doi:10.1287/moor.1080.0324 · Cited by 51
Parameterized maneuver learning for autonomous helicopter flight
Tang, J., Singh, A., Goehausen, N., & Abbeel, P. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509832 · Cited by 50
Geramifard, A., Redding, J. D., & How, J. P. (2013) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-013-9826-6 · Cited by 43
Gu, S., Chen, G., Zhang, L., Hou, J. C., Hu, Y., & Knoll, A. (2022) · Robotics · doi:10.3390/robotics11040081 · Cited by 41
Self-citationDiscounted Markov decision processes with utility constraints
Kadota, Y., Kurano, M., & Yasuda, M. (2006) · Computers & Mathematics with Applications · doi:10.1016/j.camwa.2005.11.013 · Cited by 39
Kamran, D., Engelgeh, T., Busch, M., Fischer, J., & Stiller, C. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636847 · Cited by 39
Delage, E., & Mannor, S. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273525 · Cited by 36
Run Time Assured Reinforcement Learning for Safe Satellite Docking
Dunlap, K. M., Mote, M., Delsing, K., & Hobbs, K. L. (2022) · Journal of Aerospace Information Systems · doi:10.2514/1.i011126 · Cited by 36
Barrier Function-based Safe Reinforcement Learning for Emergency Control of Power Systems
Vu, T. L., Mukherjee, S., Huang, R., & Huang, Q. (2021) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc45484.2021.9683573 · Cited by 35
Faster algorithm and sharper analysis for constrained Markov decision process
Li, T., Guan, Z., Zou, S., Xu, T., Liang, Y., & Lan, G. (2024) · Operations Research Letters · doi:10.1016/j.orl.2024.107107 · Cited by 34
A Sample-Efficient Algorithm for Episodic Finite-Horizon MDP with Constraints
Kalagarla, K. C., Jain, R., & Nuzzo, P. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i9.16979 · Cited by 30
Safe Continuous Control with Constrained Model-Based Policy Optimization
Zanger, M. A., Daaboul, K., & Zöllner, J. M. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635984 · Cited by 23
Safe Off-policy Reinforcement Learning Using Barrier Functions
Marvi, Z., & Kiumarsi, B. (2020) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc45564.2020.9147584 · Cited by 21
Reinforcement Learning with Quantitative Verification for Assured Multi-Agent Policies
Riley, J., Călinescu, R., Paterson, C., Kudenko⋆, D., & Banks, A. (2021) · International Conference on Agents and Artificial Intelligence (ICAART) · doi:10.5220/0010258102370245 · Cited by 15
Sample Complexity of Policy Gradient Finding Second-Order Stationary Points
Yang, L., Zheng, Q., & Pan, G. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17271 · Cited by 13
Self-citationModel-free safe reinforcement learning for chemical processes using Gaussian processes
Savage, T., Zhang, D., Mowbray, M., et al. (2021) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2021.08.292 · Cited by 13
Formal Methods Assisted Training of Safe Reinforcement Learning Agents
Murugesan, A., Moghadamfalahi, M., & Chattopadhyay, A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20652-9_22 · Cited by 6
HasanzadeZonuzy, A., Kalathil, D. M., & Shakkottai, S. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/347 · Cited by 6
Multi-Principal Assistance Games
Fickinger, A., Zhuang, S., Hadfield-Menell, D., & Russell, S. (2020) · arXiv · doi:10.48550/arxiv.2007.09540 · Cited by 5
Cited by
Hua, M., Qi, X., Chen, D., Jiang, K., Liu, Z. E., Sun, H., Zhou, Q., & Xu, H. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3574280 · Cited by 88
“Current research has explored various safety-enhancing strategies, including safety-constrained policy training, cooperative decision-making frameworks, and hierarchical MARL methods [168].”
Metadata
- Authors
- Gu, S., Yang, L., Du, Y., Chen, G., Walter, F., Wang, J., & Knoll, A.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2024-12
- Volume
- 46
- Issue
- 12
- Pages
- 11216-11235
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W4402402057
- Cited by
- 315
- References
- 124
- FWCI
- 46.43
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.