Safe Reinforcement Learning via Probabilistic Logic Shields
Open accessAbstract
Safe Reinforcement learning (Safe RL) aims at learning optimal policies while staying safe. A popular solution to Safe RL is shielding, which uses a logical safety specification to prevent an RL agent from taking unsafe actions. However, traditional shielding techniques are difficult to integrate with continuous, end-to-end deep RL methods. To this end, we introduce Probabilistic Logic Policy Gradient (PLPG). PLPG is a model-based Safe RL technique that uses probabilistic logic programming to model logical safety constraints as differentiable functions. Therefore, PLPG can be seamlessly applied to any policy gradient algorithm while still providing the same convergence guarantees. In our experiments, we show that PLPG learns safer and more rewarding policies compared to other state-of-the-art shielding techniques.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Cites
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · see citing works
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,821 · see citing works
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · see citing works
Monte Carlo Statistical Methods
Kim, H., Robert, C. P., & Casella, G. (2000) · Technometrics · doi:10.2307/1270959 · Cited by 5,536 · see citing works
(1999) · Cited by 5,147 · see citing works
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578 · see citing works
High-Dimensional Continuous Control Using Generalized Advantage Estimation
Schulman, J., Moritz, P., Levine, S., Jordan, M., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1506.02438 · Cited by 2,103 · see citing works
Concrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314 · see citing works
(2015) · Journal of Machine Learning Research · Cited by 1,119 · see citing works
Constrained Policy OptimizationPreprint
Achiam, J., Held, D., Tamar, A., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.10528 · Cited by 706 · see citing works
A differential approach to inference in Bayesian networks
Darwiche, A. Y. (2003) · Journal of the ACM · doi:10.1145/765568.765570 · Cited by 477 · see citing works
Reward Constrained Policy OptimizationPreprint
Tessler, C., Mankowitz, D. J., & Mannor, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11074 · Cited by 399 · see citing works
Safe Exploration in Continuous Action SpacesPreprint
Dalal, G., Dvijotham, K., Vecerík, M., Hester, T., Păduraru, C., & Tassa, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.08757 · Cited by 284 · see citing works
Probabilistic policy reuse in a reinforcement learning agent
Fernández, F., & Veloso, M. (2006) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/1160633.1160762 · Cited by 274 · see citing works
A Semantic Loss Function for Deep Learning with Symbolic KnowledgePreprint
Xu, J., Zhang, Z., Friedman, T., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.11157 · Cited by 260 · see citing works
Statistical Relational Artificial Intelligence
Raedt, L. D., Kersting, K., Natarajan, S., & Poole, D. (2016) · Synthesis lectures on artificial intelligence and machine learning · doi:10.2200/s00692ed1v01y201601aim032 · Cited by 255 · see citing works
Introduction to Probability, Statistics, and Random Processes
Pishro-Nik, H. (2018) · doi:10.25334/q40h8j · Cited by 202 · see citing works
Inference and learning in probabilistic logic programs using weighted Boolean formulas
Daan Fierens, A., den Broeck, G. V., Renkens, J., Shterionov, D., Gutmann, B., Thon, I., Janssens, G., & Raedt, L. D. (2015) · eScholarship (California Digital Library) · Cited by 201 · see citing works
Probabilistic (logic) programming concepts
Raedt, L. D., & Kimmig, A. (2015) · Machine Learning · doi:10.1007/s10994-015-5494-z · Cited by 190 · see citing works
Verifiable Reinforcement Learning via Policy ExtractionPreprint
Bastani, O., Pu, Y., & Solar-Lezama, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08328 · Cited by 188 · see citing works
LTL and Beyond: Formal Languages for Reward Function Specification in Reinforcement Learning
Camacho, A., Toro Icarte, R., Klassen, T. Q., Valenzano, R., & McIlraith, S. A. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/840 · Cited by 157 · see citing works
Safe Exploration in Markov Decision Processes
Moldovan, T. M., & Abbeel, P. (2012) · arXiv · doi:10.48550/arxiv.1205.4810 · Cited by 152 · see citing works
OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World
Pham, T.-H., Magistris, G. D., & Tachibana, R. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460547 · Cited by 123 · see citing works
Neurosymbolic Reinforcement Learning with Formally Verified ExplorationPreprint
Anderson, G., Verma, A., Dillig, I., & Chaudhuri, S. (2020) · Neural Information Processing Systems · doi:10.48550/arxiv.2009.12612 · Cited by 95 · see citing works
Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamics
Li, S., & Bastani, O. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196867 · Cited by 75 · see citing works
QMDP-Net: Deep Learning for Planning under Partial ObservabilityPreprint
Karkus, P., Hsu, D., & Lee, W. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.06692 · Cited by 74 · see citing works
Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf Restraining Specifications
Giacomo, G. D., Iocchi, L., Favorito, M., & Patrizi, F. (2019) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v29i1.3549 · Cited by 69 · see citing works
Self-Supervised Online Reward Shaping in Sparse-Reward Environments
Memarian, F., Goo, W., Lioutikov, R., Niekum, S., & Topcu, U. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636020 · Cited by 49 · see citing works
Verifiably safe exploration for end-to-end reinforcement learning
Hunt, N., Fulton, N., Magliacane, S., Hoang, T. N., Das, S., & Solar-Lezama, A. (2021) · International Conference on Hybrid Systems: computation and control · doi:10.1145/3447928.3456653 · Cited by 48 · see citing works
Temporal-Logic-Based Reward Shaping for Continuing Reinforcement Learning Tasks
Jiang, Y., Bharadwaj, S., Wu, B., Shah, R., Topcu, U., & Stone, P. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i9.16975 · Cited by 39 · see citing works
Safe Reinforcement Learning via Shielding
Alshiekh, M., Bloem, R., Ehlers, R., Könighofer, B., Niekum, S., & Topcu, U. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11797 · Cited by 37 · see citing works
Probabilistic logic programming for hybrid relational domains
Nitti, D., Laet, T. D., & Raedt, L. D. (2016) · Machine Learning · doi:10.1007/s10994-016-5558-8 · Cited by 36 · see citing works
Semantic Probabilistic Layers for Neuro-Symbolic Learning
Ahmed, K., Teso, S., CHANG, K., Broeck, G. V. D., & Vergari, A. (2022) · neural information processing systems · doi:10.52202/068431-2171 · Cited by 29 · see citing works
Imitation-Projected Programmatic Reinforcement LearningPreprint
Verma, A., Le, H. M., Yue, Y., & Chaudhuri, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.05431 · Cited by 28 · see citing works
Deep Inverse Q-learning with ConstraintsPreprint
Kalweit, G., Hügle, M., Werling, M., & Boedecker, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.01712 · Cited by 26 · see citing works
Spacecraft Command and Control with Safety Guarantees using Shielded Deep Reinforcement Learning
Harris, A. T., & Schaub, H. (2020) · AIAA SciTech Forum · doi:10.2514/6.2020-0386 · Cited by 15 · see citing works
Planning for potential: efficient safe reinforcement learning
den Hengst, F., François-Lavet, V., Hoogendoorn, M., et al. (2022) · Machine Learning · doi:10.1007/s10994-022-06143-6 · Cited by 14 · see citing works
Shielding Atari Games with Bounded Prescience
Giacobbe, M., Hasanbeig, M., Kroening, D., & Wijk, H. (2021) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/qyee4979 · Cited by 13 · see citing works
Probabilistic Policy Reuse for Safe Reinforcement Learning
García, J., & Fernández, F. (2018) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3310090 · Cited by 12 · see citing works
Reinforcement Learning with External Knowledge by using Logical Neural NetworksPreprint
Kimura, D., Chaudhury, S., Wachi, A., Kohita, R., Munawar, A., Tatsubori, M., & Gray, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.02363 · Cited by 8 · see citing works
Enforcing Almost-Sure Reachability in POMDPs
Junges, S., Jansen, N., & Seshia, S. A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-81688-9_28 · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Still being checked.
- Authors
- Yang, W.-C., Marra, G., Rens, G., & De Raedt, L.
- Affiliations
- KU Leuven (BE)
- Stellenbosch University (ZA)
- Leuven AI, KU Leuven, Belgium
- Venue
- Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence
- Published
- 2023-08
- Pages
- 5739-5749
- OpenAlex
- W4385764501
- Cited by
- 34
- References
- 41
- Citations by year
- 2026: 13 · 2025: 12 · 2024: 4 · 2023: 5
- FWCI
- 6.04
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Joint Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Lirias (KU Leuven) (submittedVersion, other-oa)
- Lirias (KU Leuven) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Fonds Wetenschappelijk Onderzoek (1239422N)
- KU Leuven
- Vlaamse regering
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Formal Methods in Verification
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Adaptive Dynamic Programming Control
- Keywords
- safe reinforcement learning, shielding, probabilistic logic programming, policy gradient, model-based reinforcement learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-11-01.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
