PreprintCited by 194Reinforcement Learning in RoboticsAdvanced Bandit Algorithms ResearchArtificial Intelligence in Games

The DOI matches its DataCite record; 208 of the 217 references it cites were found in a registry or an index.

Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain

Open access

Hao, Jianye, Yang, Tianpei, Tang, Hongyao, Bai, Chenjia, Liu, Jinyi, Meng, Zhaopeng, Liu, Peng, Wang, Zhen·2021·arXiv

DOI 10.48550/arxiv.2109.06668arXiv 2109.06668

Open this paper in CoChat — free account Open PDF

Abstract

Deep Reinforcement Learning (DRL) and Deep Multi-agent Reinforcement Learning (MARL) have achieved significant successes across a wide range of domains, including game AI, autonomous vehicles, robotics, and so on. However, DRL and deep MARL agents are widely known to be sample inefficient that millions of interactions are usually needed even for relatively simple problem settings, thus preventing the wide application and deployment in real-industry scenarios. One bottleneck challenge behind is the well-known exploration problem, i.e., how efficiently exploring the environment and collecting informative experiences that could benefit policy learning towards the optimal ones. This problem becomes more challenging in complex environments with sparse rewards, noisy distractions, long horizons, and non-stationary co-learners. In this paper, we conduct a comprehensive survey on existing exploration methods for both single-agent and multi-agent RL. We start the survey by identifying several key challenges to efficient exploration. Beyond the above two main branches, we also include other notable exploration methods with different ideas and techniques. In addition to algorithmic analysis, we provide a comprehensive and unified empirical comparison of different exploration methods for DRL on a set of commonly used benchmarks. According to our algorithmic and empirical investigation, we finally summarize the open problems of exploration in DRL and deep MARL and point out a few future directions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2109.06668

Cites

208 of 208 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Attention Is All You Need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions
Ryan, R. M., & Deci, E. L. (2000) · Contemporary Educational Psychology · doi:10.1006/ceps.1999.1020 · Cited by 18,666
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578

Show all 217 references

Research works printed but not linked: 26

[10][10] S. S. Mousavi, M. Schukat, and E. Howley, 'Deep reinforcement learning: an overview,' in SAI Intelligent Systems Conference , 2016. -
[20][20] T. Lattimore and C. Szepesvári, Bandit Algorithms . Cambridge University Press, 2020. -
[32][32] D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. A. Riedmiller, 'Deterministic policy gradient algorithms,' in ICML , 2014. -
[33][33] R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, 'Multiagent actor-critic for mixed cooperative-competitive environments,' in NeurIPS , 2017. -
[38][38] N. Srinivas, A. Krause, S. M. Kakade, and M. W. Seeger, 'Gaussian process optimization in the bandit setting: No regret and experimental design,' in ICML , 2010. -
[46][46] Y. Burda, H. Edwards, A. J. Storkey, and O. Klimov, 'Exploration by random network distillation,' in ICLR , 2019. -
[69][69] R. Dearden, N. Friedman, and S. J. Russell, 'Bayesian q-learning,' in AAAI , 1998. -
[70][70] P. Auer, N. Cesa-Bianchi, and P. Fischer, 'Finite-time analysis of the multiarmed bandit problem,' Machine learning , vol. 47, no. 2-3, pp. 235-256, 2002. -
[72][72] P. Auer, 'Using confidence bounds for exploitation-exploration tradeoffs,' JMLR , vol. 3, pp. 397-422, 2002. -
[79][79] N. Nikolov, J. Kirschner, F. Berkenkamp, and A. Krause, 'Informationdirected exploration for deep reinforcement learning,' in ICLR , 2019. -
[84][84] C. Oh and A. Cavallaro, 'Learning action representations for selfsupervised visual exploration,' in ICRA , 2019. -
[93][93] T. Zhang, P. Rashidinejad, J. Jiao, Y. Tian, J. Gonzalez, and S. Russell, 'Made: Exploration via maximizing deviation from explored regions,' arXiv preprint arXiv:2106.10268 , 2021. - · arXiv
[97][97] M. Klissarov, R. Islam, K. Khetarpal, and D. Precup, 'Variational state encoding as intrinsic motivation in reinforcement learning,' in TARL Workshop on ICLR , 2019. -
[134][134] S. L. Herbert, S. Bansal, S. Ghosh, and C. J. Tomlin, 'Reachabilitybased safety guarantees using efficient initializations,' in CDC , 2019, pp. 4810-4816. -
[137][137] Y. Guo, J. Choi, M. Moczulski, S. Feng, S. Bengio, M. Norouzi, and H. Lee, 'Memory based trajectory-conditioned policies for learning from sparse rewards,' in NeurIPS , 2020. -
[146][146] J. Zhao, M. Yang, X. Hu, W. Zhou, and H. Li, 'DQMIX: A distributional perspective on multi-agent reinforcement learning,' CoRR , vol. abs/2202.10134, 2022. - · arXiv
[150][150] L. Zheng, J. Chen, J. Wang, J. He, Y. Hu, Y. Chen, C. Fan, Y. Gao, and C. Zhang, 'Episodic multi-agent reinforcement learning with curiositydriven exploration,' in NeurIPS , M. Ranzato, A. Beygelzimer, Y. N. Dauphin, P. Liang, and J. W. Vaughan, Eds., 2021, pp. 3757-3769. -
[156][156] D. Carmel and S. Markovitch, 'Exploration strategies for model-based learning in multi-agent systems: Exploration strategies,' JAAMAS , vol. 2, no. 2, pp. 141-172, 1999. -
[168][168] M. Samvelyan, T. Rashid, C. S. de Witt, G. Farquhar, N. Nardelli, T. G. J. Rudner, C. Hung, P. H. S. Torr, J. N. Foerster, and S. Whiteson, 'The starcraft multi-agent challenge,' pp. 2186-2188, 2019. -
[169][169] G. Farquhar, L. Gustafson, Z. Lin, S. Whiteson, N. Usunier, and G. Synnaeve, 'Growing action spaces,' in ICML , 2020. -
[179][179] K. H. Kim, M. Sano, J. De Freitas, N. Haber, and D. Yamins, 'Active world model learning in agent-rich environments with progress curiosity,' in ICML , 2020. -
[186][186] Z. C. Lipton, X. Li, J. Gao, L. Li, F. Ahmed, and L. Deng, 'Bbqnetworks: Efficient exploration in deep reinforcement learning for taskoriented dialogue systems,' in AAAI , 2018. -
[210][210] R. Zhao, X. Sun, and V. Tresp, 'Maximum entropy-regularized multigoal reinforcement learning,' in ICML , 2019. -
[218][218] V. H. Pong, M. Dalal, S. Lin, A. Nair, S. Bahl, and S. Levine, 'Skew-fit: State-covering self-supervised reinforcement learning,' in ICML , 2020. -
[219][219] A. Sharma, S. Gu, S. Levine, V. Kumar, and K. Hausman, 'Dynamicsaware unsupervised discovery of skills,' in ICLR , 2020. -
[221][221] T. Haarnoja, H. Tang, P. Abbeel, and S. Levine, 'Reinforcement learning with deep energy-based policies,' in ICML , ser. Proceedings of Machine Learning Research, vol. 70, 2017, pp. 1352-1361. -

Sorted by citations

Theoretical approaches to AI in supply chain optimization: Pathways to efficiency and resilience
Abaku, E. A., Edunjobi, T. E., & Odimarha, A. C. (2024) · International Journal of Science and Technology Research Archive · doi:10.53771/ijstra.2024.6.1.0033 · Cited by 149
Multi-agent reinforcement learning for resources allocation optimization: a survey
Hady, M. A., Hu, S., Pratama, M., Cao, Z., & Kowalczyk, R. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11340-5 · Cited by 131
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15217 · Cited by 101
Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithms
Li, P., Hao, J., Tang, H., Fu, X., Zheng, Y., & Tang, K. (2024) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2024.3443913 · Cited by 102
Self-citation
Energy management for demand response in networked greenhouses with multi-agent deep reinforcement learning
Ajagekar, A., Decardi-Nelson, B., & You, F. (2024) · Applied Energy · doi:10.1016/j.apenergy.2023.122349 · Cited by 82
Deep Reinforcement Learning for Solving Vehicle Routing Problems With Backhauls
Wang, C., Cao, Z., Wu, Y., Teng, L., & Wu, G. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3371781 · Cited by 84
The State of AI-Empowered Backscatter Communications: A Comprehensive Survey
Xu, F., Hussain, T., Ahmed, M., Ali, K., Mirza, M. A., Khan, W. U., Ihsan, A., & Han, Z. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3299210 · Cited by 67
TF-DDRL: A Transformer-Enhanced Distributed DRL Technique for Scheduling IoT Applications in Edge and Cloud Computing Environments
Wang, Z., Goudarzi, M., & Buyya, R. (2025) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2025.3528346 · Cited by 55
Self-citation
Physics-informed deep reinforcement learning for enhancement on tunnel boring machine's advance speed and stability
Lin, P., Wu, M., Xiao, Z., Tiong, R. L. K., & Zhang, L. (2023) · Automation in Construction · doi:10.1016/j.autcon.2023.105234 · Cited by 47
Federated Deep Reinforcement Learning for Prediction-Based Network Slice Mobility in 6G Mobile Networks
Ming, Z., Yu, H., & Taleb, T. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3404125 · Cited by 44
Deep Reinforcement Learning for AoI Minimization in UAV-Aided Data Collection for WSN and IoT Applications: A Survey
Amodu, O. A., Jarray, C., Mahmood, R. A. R., Althumali, H., Bukar, U. A., Nordin, R., Abdullah, N. F., & Luong, N. C. (2024) · IEEE Access · doi:10.1109/access.2024.3425497 · Cited by 34
Improve exploration in deep reinforcement learning for UAV path planning using state and action entropy
Lv, H. P., Chen, Y., Li, S., Zhu, B., & Li, M. (2024) · Measurement Science and Technology · doi:10.1088/1361-6501/ad2663 · Cited by 31
Performance enhancement of artificial intelligence: A survey
Krichen, M., & Abdalzaher, M. S. (2024) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2024.104034 · Cited by 35
A Review of Smart Grid Evolution and Reinforcement Learning: Applications, Challenges and Future Directions
Xu, N., Tang, Z., Si, C., Bian, J., & Mu, C. (2025) · Energies · doi:10.3390/en18071837 · Cited by 30
Progress and summary of reinforcement learning on energy management of MPS-EV
Lin, Y., Chu, L., Hu, J., Hou, Z., Li, J., Jiang, J., & Zhang, Y. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e23014 · Cited by 28
When to Switch: Planning and Learning for Partially Observable Multi-Agent Pathfinding
Skrynnik, A., Андреевич, А. А., Yakovlev, K., & Panov, A. I. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3303502 · Cited by 29
Transient gas path fault diagnosis of aero-engine based on domain adaptive offline reinforcement learning
Xu, J., Wang, Y., Wang, Z., Wang, X., & Zhao, Y. (2024) · Aerospace Science and Technology · doi:10.1016/j.ast.2024.109701 · Cited by 28
Self-citation
Improved exploration–exploitation trade-off through adaptive prioritized experience replay
Hassani, H., Nikan, S., & Shami, A. A. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128836 · Cited by 24
A systematic survey and meta-analysis of the segment anything model in remote sensing image processing: Challenges, advances, applications, and opportunities
Wan, Z., Wang, S., Han, W., Wang, Y., Huang, X., Zhang, X., Chen, X., & Chen, Y. (2025) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2025.08.023 · Cited by 26
Intelligent games meeting with multi-agent deep reinforcement learning: a comprehensive review
Wang, Y., Wang, Y., Tian, F., Ma, J., & Jin, Q. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11166-1 · Cited by 25
LSTM-augmented DRL for generalisable energy management of hydrogen-hybrid ship propulsion systems
Fan, A., Liu, H., Wu, P., Yang, L., Guan, C., Li, T., Bucknall, R. W. G., & Liu, Y. (2025) · eTransportation · doi:10.1016/j.etran.2025.100442 · Cited by 20
Balancing exploration and exploitation in episodic reinforcement learning
Chen, Q., Zhang, Q., & Liu, Y. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120801 · Cited by 19
A Reinforcement Learning Approach Combined With Scope Loss Function for Crime Prediction on Twitter (X)
Yang, L., Guofan, J., Yixin, Z., Qianze, W., Jian, Z., Alizadehsani, R., & Pławiak, P. (2024) · IEEE Access · doi:10.1109/access.2024.3473296 · Cited by 17
Environment-Aware Adaptive Reinforcement Learning-Based Routing for Vehicular Ad Hoc Networks
Jiang, Y., Zhu, J., & Yang, K. (2023) · Sensors · doi:10.3390/s24010040 · Cited by 16
A novel inverse design method for morphing airfoil based on deep reinforcement learning
Su, J., Sun, G., & Tao, J. (2024) · Aerospace Science and Technology · doi:10.1016/j.ast.2024.108895 · Cited by 16
Dynamic Bottleneck for Robust Self-Supervised ExplorationPreprint
Bai, C., Wang, L., Han, L., Garg, A., Hao, J., Liu, P., & Wang, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.10735 · Cited by 16
Self-citation
End-to-End Autonomous Driving in CARLA: A Survey
Ozaibi, Y. A., Hina, M. D., & Ramdane-Chérif, A. (2024) · IEEE Access · doi:10.1109/access.2024.3473611 · Cited by 20
Traffic navigation via reinforcement learning with episodic-guided prioritized experience replay
Hassani, H., Nikan, S., & Shami, A. A. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109147 · Cited by 15
VL-TGS: Trajectory Generation and Selection Using Vision Language Models in Mapless Outdoor Environments
Song, D., Liang, J., Xiao, X., & Manocha, D. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3559822 · Cited by 15
A Hierarchical Resource Scheduling Method for Satellite Control System Based on Deep Reinforcement Learning
Li, Y., Guo, X., Meng, Z., Qin, J., Li, X., Ma, X., Ren, S., & Yang, J. Y. (2023) · Electronics · doi:10.3390/electronics12193991 · Cited by 13
Self-citation
A Coverage-Guided Fuzzing Method for Automatic Software Vulnerability Detection Using Reinforcement Learning-Enabled Multi-Level Input Mutation
Pham, V.-H., Hien, D. T. T., Chuong, N. P., Thai, P. T., & Duy, P. T. (2024) · IEEE Access · doi:10.1109/access.2024.3421989 · Cited by 16
Overview of Generative AI Techniques for Cybersecurity
Sindiramutty, S. R., Prabagaran, K. R. V., Akbar, R., Hussain, M., & Malik, N. A. (2024) · Advances in information security, privacy, and ethics book series · doi:10.4018/979-8-3693-5415-5.ch001 · Cited by 14
Efficient surrogate-based optimization framework integrating physics-informed neural networks, deep active learning and deep reinforcement learning: Multilayer thin films case study
Zheng, J., Huang, J., Lin, A., Li, F., & Huang, H. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.113609 · Cited by 12
WToE: Learning When to Explore in Multiagent Reinforcement Learning
Dong, S., Mao, H., Yang, S., Zhu, S., Li, W., Hao, J., & Gao, Y. (2023) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2023.3328732 · Cited by 12
Self-citation
Robust Multi-Agent Reinforcement Learning by Mutual Information Regularization
Li, S., Xu, R., Xiu, J., Zheng, Y., Feng, P., Ma, Y., An, B., Yang, Y., & Liu, X. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3577259 · Cited by 17
An enhanced deep reinforcement learning approach for efficient, effective, and equitable disaster relief distribution
Ahmad, M., Tayyab, M., & Habib, M. S. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110002 · Cited by 11
Historical Decision-Making Regularized Maximum Entropy Reinforcement Learning
Dong, B., Huang, L., Pang, N., Chen, H., & Zhang, W. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3481887 · Cited by 10
Entropy-driven multi agent deep reinforcement learning for resilient distribution networks: coordinating MESS and microgrids
Cai, C., Gan, F., Cui, Y., Li, B., & Hou, S. (2025) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2025.110968 · Cited by 10
Multi-Agent Deep reinforcement learning for EV aggregator bidding in Renewable-Dominated electricity markets
Zhang, Y., Meng, L., de Souza, A. C. Z., Hu, Q., Liu, H., Lebedev, A., & Shotorbani, A. M. (2025) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2025.111444 · Cited by 9
Mutual information oriented deep skill chaining for multi‐agent reinforcement learning
Xie, Z., Ji, C., Qiao, C. Y., Song, W., Li, Z., Zhang, Y., & Zhang, Y. (2024) · CAAI Transactions on Intelligence Technology · doi:10.1049/cit2.12322 · Cited by 9
Multi-agent generalized cooperative optimization scheduling for multi-energy complementarity in microgrids
Xu, N., Mu, C., Wang, K., Ma, L., & Liu, Z. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-025-11354-z · Cited by 8
Multi-agent cooperation policy gradient method based on enhanced exploration for cooperative tasks
Zhao, L.-Y., Chang, T.-Q., Zhang, L., Zhang, X., & Wang, J. (2023) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-023-01976-6 · Cited by 8
Improving the Traffic Engineering of SDN networks by using Local Multi-Agent Deep Reinforcement Learning
Gómez‐delaHiz, J., & Galán–Jiménez, J. (2024) · IEEE/IFIP Network Operations and Management Symposium (NOMS) · doi:10.1109/noms59830.2024.10575465 · Cited by 8
Cooperative multi-agent game based on reinforcement learning
Liu, H. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100205 · Cited by 7
Reinforcement learning for fuselage shape control during aircraft assembly
Lutz, T., Wang, Y., Yue, X., & Camelio, J. A. (2024) · IISE Transactions · doi:10.1080/24725854.2024.2413136 · Cited by 7
Optimization methods in fully cooperative scenarios: a review of multiagent reinforcement learning
Yang, T., Shi, X., Zeng, Q., Yang, Y., Xu, C., & Liu, H. (2025) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2400259 · Cited by 7
Self-citation
OpenBench: A New Benchmark and Baseline for Semantic Navigation in Smart Logistics
Wang, J., Huo, D., Xu, Z., Shi, Y., Yan, Y., Wang, Y., Gao, C., Qiao, Y., & Zhou, G. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127476 · Cited by 7
Reinforcement Learning Review: Past Acts, Present Facts and Future Prospects
Kommey, B., Isaac, O. J., Tamakloe, E., & Opoku, D. (2024) · IT JOURNAL RESEARCH AND DEVELOPMENT · doi:10.25299/itjrd.2023.13474 · Cited by 10
OVD-Explorer: Optimism Should Not Be the Sole Pursuit of Exploration in Noisy Environments
Liu, J., Wang, Z., Zheng, Y., Hao, J., Bai, C., Ye, J., Wang, Z., Piao, H., & Sun, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i12.29303 · Cited by 7
Self-citation
PORTAL: Automatic Curricula Generation for Multiagent Reinforcement Learning
Wu, J., Hao, J., Yang, T., Hao, X., Zheng, Y., Wang, W., & Taylor, M. E. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i14.29524 · Cited by 8
Self-citation
VLD-Net: Localization and Detection of the Vertebrae From X-Ray Images by Reinforcement Learning With Adaptive Exploration Mechanism and Spine Anatomy Information
Xiang, S., Zhang, L., Wang, Y., Zhou, S., Zhao, X., Zhang, T., & Li, S. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3553935 · Cited by 7
BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation
Li, C., Zhang, R., Wong, J., Gokmen, C., Srivastava, S., Martín-Martín, R., Wang, C., Levine, G., Ai, W., Martinez, B., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.09227 · Cited by 6
Robust Multi‐Agent Reinforcement Learning Against Adversarial Attacks for Cooperative Self‐Driving Vehicles
Wang, C., Wang, Z., & Aouf, N. (2025) · IET Radar Sonar & Navigation · doi:10.1049/rsn2.70033 · Cited by 11
Self-citation
Task Phasing: Automated Curriculum Learning from Demonstrations
Bajaj, V., Sharon, G., & Stone, P. (2023) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v33i1.27235 · Cited by 6
Settling Decentralized Multi-Agent Coordinated Exploration by Novelty Sharing
Jiang, H., Ding, Z., & Lu, Z. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29693 · Cited by 8
A unified data-driven approach under deep reinforcement learning with direct control responses for microgrid operations
Yao, F., Zhao, W., Forshaw, M., & Zhou, W. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113844 · Cited by 6
Exploring into the Unseen: Enhancing Language-Conditioned Policy Generalization with Behavioral Information
Cao, L., Wang, C., Qi, J., & Peng, Y. (2023) · Cyborg and Bionic Systems · doi:10.34133/cbsystems.0084 · Cited by 6
Inverse RL Scene Dynamics Learning for Nonlinear Predictive Control in Autonomous Vehicles
Grigorescu, S. M., & Zaha, M. V. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3549816 · Cited by 7
NM-TD3: A Hybrid Noise-Driven TD3 Algorithm With Long-Term Reward Propagation for Mobile Robot Path Planning
Yang, X., Wang, Q., Li, J., & Jiang, X. (2025) · IEEE Access · doi:10.1109/access.2025.3602286 · Cited by 5
Guiding Multiagent Multitask Reinforcement Learning by a Hierarchical Framework With Logical Reward Shaping
Liu, C., Cong, J., Chen, B., Jin, Y., & Zhu, E. (2025) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2025.3631239 · Cited by 6
Cooperative Multiagent Learning and Exploration With Min–Max Intrinsic Motivation
Hou, Y., Kang, J., Piao, H., Zeng, Y., Ong, Y.-S., Jin, Y., & Zhang, Q. (2025) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2025.3557694 · Cited by 5
Multi-fidelity reinforcement learning for time-optimal quadrotor re-planning
Ryou, G., Wang, G., & Karaman, S. (2025) · The International Journal of Robotics Research · doi:10.1177/02783649251364393 · Cited by 9
Curiosity-driven learning in artificial intelligence and its applications
Sun, C. (2023) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/172831 · Cited by 6
The impact of intrinsic rewards on exploration in Reinforcement Learning
Kayal, A., Pignatelli, E., & Toni, L. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11340-0 · Cited by 13
Dynamics of Q -Learning in Networked Stochastic Games
Yuan, Z., Jiang, G., Hu, S., Perc, M., Chu, C., & Liu, J. (2026) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3641365 · Cited by 5
Self-citation
FPGA Divide-and-Conquer Placement using Deep Reinforcement Learning
Wang, S., Mamillapalli, D. R. S., Yang, T., & Taylor, M. E. (2024) · International Symposium of Electronics Design Automation (ISEDA) · doi:10.1109/iseda62518.2024.10617677 · Cited by 5
Self-citation
Efficient and stable deep reinforcement learning: selective priority timing entropy
Huo, L., Mao, J., San, H., Zhang, S., Li, R., & Fu, L. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05705-6 · Cited by 5
A Deep Reinforcement Learning-Based Evolutionary Algorithm for Distributed Heterogeneous Green Hybrid Flowshop Scheduling
Xu, H., Huang, L., Tao, J., Zhang, C., & Zheng, J. (2025) · Processes · doi:10.3390/pr13030728 · Cited by 4
Curiosity-driven exploration based on hierarchical vision transformer for deep reinforcement learning with sparse rewards
Jiang, W., Liu, G., Leng, Q., & Guo, N. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130252 · Cited by 4
CADRL: Category-Aware Dual-Agent Reinforcement Learning for Explainable Recommendations over Knowledge Graphs
Zheng, S., Yin, H., Chen, T., Kong, X., Hou, J., & Zhao, P. (2025) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde65448.2025.00017 · Cited by 4
Generative subgoal oriented multi-agent reinforcement learning through potential field
Li, S., Jiang, H., Liu, Y., Zhang, J., Xu, X., & Liu, D. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106552 · Cited by 4
GenSafe: A Generalizable Safety Enhancer for Safe Reinforcement Learning Algorithms Based on Reduced Order Markov Decision Process Model
Zhou, Z., Xie, X., Song, J., Shu, Z., & Ma, L. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3496492 · Cited by 4
Multi-agent Reinforcement Learning for Dynamic Dispatching in Material Handling Systems
Lee, X. Y., Wang, H., KATSUMATA, D., Matsui, T., & Gupta, C. (2024) · IEEE International Conference on Cognitive Machine Intelligence (CogMI) · doi:10.1109/cogmi62246.2024.00031 · Cited by 4
Road preview method for active suspension based on reinforcement learning
Wang, G., Kou, F., Liu, P., Lv, W., & Xing, L. (2025) · Measurement Science and Technology · doi:10.1088/1361-6501/adad09 · Cited by 4
Self-citation
Multi-Agent Deep Q-Network with Layer-Based Communication Channel for Autonomous Internal Logistics Vehicle Scheduling in Smart Manufacturing
Feizabadi, M., Hosseini, A., & Yahouni, Z. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-80760-2_1 · Cited by 4
A deep reinforcement learning model for large-scale traffic signal control based on graph meta-learning using local subgraphs
Zhou, Z., Zhang, Y., & Li, X. (2025) · Science China Information Sciences · doi:10.1007/s11432-023-4280-6 · Cited by 4
Discretizing Continuous Action Space With Unimodal Probability Distributions for On-Policy Reinforcement Learning
Zhu, Y., Wang, Z., Zhu, Y., Chen, C., & Zhao, D. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3446371 · Cited by 8
Self-citation
Representation-driven sampling and adaptive policy resetting for improving multi-Agent reinforcement learning
Jin, W., Tian, X., Wang, N., Wu, B., Shi, B., Zhao, B., & Yang, G. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107875 · Cited by 5
A cooperative jamming mode adjustment method based on Multi-Agent reinforcement learning
Wang, J., Liu, Y., Li, C., Yang, D., & Yin, Q. (2025) · Ain Shams Engineering Journal · doi:10.1016/j.asej.2025.103672 · Cited by 3
Subgoal-Based Hierarchical Reinforcement Learning for Multiagent Collaboration
Xu, C., Shi, Y., Zhang, C., Wang, R., Duan, S., Wan, Y., & Zhang, X. (2026) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2025.3646451 · Cited by 7
Diffusion-DDPG: An Edge Intelligence Offloading Algorithm Empowering Agentic AI for Autonomous Decision-Making
Yuan, X., Wang, S., Wang, A., Zhang, N., Ye, Q., Yan, L., & Jin, X. (2026) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2026.3697300 · Cited by 3
Reinforcement learning with thermal fluctuations at the nanoscale
Boccardo, F., & Pierre-Louis, O. (2024) · Physical review. E · doi:10.1103/physreve.110.l023301 · Cited by 3
Dynamic data partitioning strategy for distributed learning on heterogeneous edge system
Yu, K., & Zhang, W. (2025) · Computer Communications · doi:10.1016/j.comcom.2025.108262 · Cited by 3
Efficient Exploration in Large State-Action Space Through Structured Action Space for Learning Multirobots Motion Planning
Mu, C., Jiang, Z., Zhang, J., Wang, K., Xu, X., & Yi, J. (2025) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2025.3619986 · Cited by 3
Q-CMAPO: A quantum-classical framework for balancing exploration and exploitation in multi-agent reinforcement learning
Taghavi, M., & Vahidi, J. (2026) · Quantum Machine Intelligence · doi:10.1007/s42484-026-00361-0 · Cited by 7
A Unified Uncertainty-Aware Exploration: Combining Epistemic and Aleatory Uncertainty
Malekzadeh, P., Hou, M., & Plataniotis, K. N. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095594 · Cited by 3
Deep Reinforcement Learning based channel allocation (DRLCA) in Cognitive Radio Networks
Pavan, M. N. (2024) · Journal of Electrical Systems · doi:10.52783/jes.2774 · Cited by 3
Research on Intelligent Matching Technology of Power Grid Dispatching Automation Emergency Plan Based on Large Model and Small Sample LoRA Fine-Tuning Algorithm
Wang, D., Yin, C., Liu, L., Zhang, Z., She, Y., & Yu, Y. (2024) · Asia Conference on Power and Electrical Engineering (ACPEE) · doi:10.1109/acpee60788.2024.10532664 · Cited by 3
Rule-Based Assessment of Reinforcement Learning Practices Using Large Language Models
Ntentos, E., Warnett, S. J., & Zdun, U. (2025) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain66642.2025.00009 · Cited by 3
The State of AI-Empowered Backscatter Communications: A Comprehensive SurveyPreprint
Ahmed, M., Hussain, T., Ali, K., Mirza, M. A., Khan, W. U., Ihsan, A., Xu, F., & Han, Z. (2023) · doi:10.36227/techrxiv.22245958.v1 · Cited by 3
Boosting Weak-to-Strong Agents in Multiagent Reinforcement Learning via Balanced PPO
Huang, S., Chen, H., Piao, H., Sun, Z., Chang, Y., Sun, L., & Yang, B. X. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3437366 · Cited by 3
Cognition-Oriented Multiagent Reinforcement Learning
Qiu, T., Wu, S., Liu, Z., Pu, Z., Yi, J., Zhao, Y., & Luo, B. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3502176 · Cited by 2
State-novelty guided action persistence in deep reinforcement learning
Hu, J., Weng, P., & Ban, Y. (2025) · Machine Learning · doi:10.1007/s10994-024-06675-z · Cited by 3
Enhanced exploration in reinforcement learning using graph neural network based intrinsic reward mechanism
Pandian, J. A., Thirunavukarasu, R., & Nagarajan, R. (2025) · Scientific Reports · doi:10.1038/s41598-025-23769-3 · Cited by 2
Research on Microgrid Resilience in Highway Service Areas Based on Federated Multi-Agent Deep Reinforcement Learning
Li, J., Cheng, Z., Peng, Y., Huang, H., & Ye, C. (2026) · Sustainability · doi:10.3390/su18021027 · Cited by 2
A Policy Resonance Approach to Solve the Problem of Responsibility Diffusion in Multiagent Reinforcement Learning
Fu, Q., Qiu, T., Yi, J., Pu, Z., Ai, X., & Yuan, W. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3423417 · Cited by 2
Rethinking Exploration and Experience Exploitation in Value-Based Multi-Agent Reinforcement Learning
Borzilov, A., Skrynnik, A., & Panov, A. I. (2025) · IEEE Access · doi:10.1109/access.2025.3530974 · Cited by 2
Collaborative path penetration in 5G-IoT networks: A multi-agent deep reinforcement learning approach
Shen, H., Li, X. D., Wang, Y., Wang, T., & Bai, G. (2025) · Peer-to-Peer Networking and Applications · doi:10.1007/s12083-025-01939-4 · Cited by 4
DERRIC: Decentralized Reinforced RAN Intelligent Controller Orchestration for 6G Networks
Nezhad, E. H., Maio, A. D., & Braun, T. I. (2025) · IEEE Wireless Communications and Networking Conference · doi:10.1109/wcnc61545.2025.10978840 · Cited by 2
Adaptive Kalman-Based Hybrid Car Following Strategy Using Td3 and CaccPreprint
Zheng, Y., Yan, R., Jia, B., Jiang, R., Tapus, A., Chen, X., Zheng, S., & Ying, S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4681766 · Cited by 2

Authors: Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z.
Affiliations: Tianjin University (CN)
Shanghai Artificial Intelligence Laboratory (CN)
Harbin Institute of Technology (CN)
Northwestern Polytechnical University (CN)
ORCID: Hao, Jianye
Yang, Tianpei
Tang, Hongyao
Bai, Chenjia
Liu, Jinyi
Meng, Zhaopeng
Liu, Peng
Wang, Zhen
Venue: arXiv
Publisher: arXiv
Published: 2021
Volume: 35
Issue: 7
Pages: 8762-8782
ISSN: 2162-237X (ISSN-L)
DOI: 10.48550/arxiv.2109.06668
arXiv: 2109.06668
OpenAlex: W4319165238
Cited by: 192
References: 217
Citations by year: 2026: 42 · 2025: 77 · 2024: 48 · 2023: 23 · 2022: 1 · 2021: 1
FWCI: 17.91
Type: journal article
Language: en
Open access: green
arxiv.org
Licence: arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations: IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
PubMed (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: Tencent
National Natural Science Foundation of China (U22B2036, 11931015, 62025602, U1836214)
National Science Fund for Distinguished Young Scholars (62025602)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
Keywords: sample inefficiency, intrinsic motivation, sparse rewards

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks