Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning
Open accessAbstract
Deep reinforcement learning (DRL) has numerous applications in the real world, thanks to its ability to achieve high performance in a range of environments with little manual oversight. Despite its great advantages, DRL is susceptible to adversarial attacks, which precludes its use in real-life critical systems and applications (e.g., smart grids, traffic controls, and autonomous vehicles) unless its vulnerabilities are addressed and mitigated. To address this problem, we provide a comprehensive survey that discusses emerging attacks on DRL-based systems and the potential countermeasures to defend against these attacks. We first review the fundamental background on DRL and present emerging adversarial attacks on machine learning techniques. We then investigate the vulnerabilities that an adversary can exploit to attack DRL along with state-of-the-art countermeasures to prevent such attacks. Finally, we highlight open issues and research challenges for developing solutions to deal with attacks on DRL-based intelligent systems.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Enhanced adversarial training, incorporating diverse adversarial examples during model training, and deploying defensive mechanisms such as adversarial loss can improve resilience against intentional manipulations.
“Enhanced adversarial training, incorporating diverse adversarial examples during model training, and deploying defensive mechanisms such as adversarial loss can improve resilience against intentional manipulations [69].”
From A Comprehensive Review of DeepFake Detection Using Advanced Machine Learning and Fusion Methods (2023) · cites this paper as [69]
Cites
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · see citing works
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859 · see citing works
The Limitations of Deep Learning in Adversarial Settings
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016) · 2016 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2016.36 · Cited by 3,993 · see citing works
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018) · Science · doi:10.1126/science.aar6404 · Cited by 3,562 · cited 1× · see citing works
“In 2017, DRL beat the human champions in the game of Go [13] and most recently a team of five DRL agents has beaten the world champion human team in Dota2 matches [14].”
BackgroundDeep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318 · see citing works
Universal Adversarial Perturbations
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Frossard, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.17 · Cited by 2,680 · see citing works
Dota 2 with Large Scale Deep Reinforcement Learning
OpenAI, :, Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., Farhi, D., Fischer, Q., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.06680 · Cited by 1,065 · see citing works
Transfer Learning in Deep Reinforcement Learning: A Survey
Zhu, Z., Lin, K., Jain, A. K., & Zhou, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292075 · Cited by 861 · see citing works
Deep Direct Reinforcement Learning for Financial Signal Representation and Trading
Deng, Y., Bao, F., Kong, Y., Ren, Z., & Dai, Q. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2522401 · Cited by 822 · see citing works
Virtual to Real Reinforcement Learning for Autonomous Driving
Pan, X., You, Y., Wang, Z., & Lu, C. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.11 · Cited by 326 · cited 3× · see citing works
“They test their approach on a self-driving environment using the TORCS simulator and show that a risk-averse agent handles the risk better and leads to fewer crashes than a normal agent trained in a similar environment.”
BackgroundInfluentialQuery-Efficient Hard-label Black-box Attack:An Optimization-based Approach
Cheng, M., Le, T., Chen, P.-Y., Yi, J., Zhang, H., & Hsieh, C.-J. (2018) · arXiv · doi:10.48550/arxiv.1807.04457 · Cited by 269 · see citing works
Deep Reinforcement Learning for Trading
Zhang, Z., Zohren, S., & Roberts, S. (2020) · The Journal of Financial Data Science · doi:10.3905/jfds.2020.1.030 · Cited by 227 · see citing works
Thinking Fast and Slow with Deep Learning and Tree Search
Anthony, T., Tian, Z., & Barber, D. (2017) · arXiv · doi:10.48550/arxiv.1705.08439 · Cited by 198 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Zeb, S., Mahmood, A., Hassan, S. A., Piran, M. J., Gidlund, M., & Guizani, M. (2022) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2021.103309 · Cited by 132 · see citing works
A Comprehensive Review of DeepFake Detection Using Advanced Machine Learning and Fusion Methods
Gupta, G., Raja, K., Gupta, M., Jan, T., Whiteside, S. T., & Prasad, M. (2023) · Electronics · doi:10.3390/electronics13010095 · Cited by 99 · see citing works
- Authors
- Ilahi, I., Usama, M., Qadir, J., Janjua, M. U., Al-Fuqaha, A., Hoang, D. T., & Niyato, D.
- Affiliations
- Information Technology University (PK)
- Hamad Bin Khalifa University, Doha, Qatar
- University of Technology Sydney (AU)
- Nanyang Technological University (SG)
- ORCID
- Venue
- IEEE Transactions on Artificial Intelligence
- Published
- 2022-04
- Volume
- 3
- Issue
- 2
- Pages
- 90-109
- ISSN
- 2691-4581 (ISSN-L)
- OpenAlex
- W3001548133
- Cited by
- 170
- References
- 13
- Citations by year
- 2026: 23 · 2025: 42 · 2024: 34 · 2023: 34 · 2022: 19 · 2021: 12 · 2020: 6
- FWCI
- 8.75
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DR-NTU (Nanyang Technological University) (submittedVersion)
- Qatar University QSpace (Qatar University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Qatar National Research Fund
- Qatar Foundation
- National Priorities Research Program (13S-0206-200273)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Keywords
- adversarial attacks, deep reinforcement learning, adversarial robustness, vulnerability (computer security), autonomous vehicles, smart grid, traffic management
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
