PreprintCited by 194Reinforcement Learning in RoboticsAdvanced Bandit Algorithms ResearchArtificial Intelligence in Games

The DOI matches its DataCite record; 208 of the 217 references it cites were found in a registry or an index.

Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain

Open access

Hao, Jianye, Yang, Tianpei, Tang, Hongyao, Bai, Chenjia, Liu, Jinyi, Meng, Zhaopeng, Liu, Peng, Wang, Zhen·2021·arXiv

DOI 10.48550/arxiv.2109.06668arXiv 2109.06668

Open this paper in CoChat — free account Open PDF

Abstract

Deep Reinforcement Learning (DRL) and Deep Multi-agent Reinforcement Learning (MARL) have achieved significant successes across a wide range of domains, including game AI, autonomous vehicles, robotics, and so on. However, DRL and deep MARL agents are widely known to be sample inefficient that millions of interactions are usually needed even for relatively simple problem settings, thus preventing the wide application and deployment in real-industry scenarios. One bottleneck challenge behind is the well-known exploration problem, i.e., how efficiently exploring the environment and collecting informative experiences that could benefit policy learning towards the optimal ones. This problem becomes more challenging in complex environments with sparse rewards, noisy distractions, long horizons, and non-stationary co-learners. In this paper, we conduct a comprehensive survey on existing exploration methods for both single-agent and multi-agent RL. We start the survey by identifying several key challenges to efficient exploration. Beyond the above two main branches, we also include other notable exploration methods with different ideas and techniques. In addition to algorithmic analysis, we provide a comprehensive and unified empirical comparison of different exploration methods for DRL on a set of commonly used benchmarks. According to our algorithmic and empirical investigation, we finally summarize the open problems of exploration in DRL and deep MARL and point out a few future directions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2109.06668

Cites

208 of 208 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Attention Is All You Need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions
Ryan, R. M., & Deci, E. L. (2000) · Contemporary Educational Psychology · doi:10.1006/ceps.1999.1020 · Cited by 18,666
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578

Show all 217 references

Research works printed but not linked: 26

[10][10] S. S. Mousavi, M. Schukat, and E. Howley, 'Deep reinforcement learning: an overview,' in SAI Intelligent Systems Conference , 2016. -
[20][20] T. Lattimore and C. Szepesvári, Bandit Algorithms . Cambridge University Press, 2020. -
[32][32] D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. A. Riedmiller, 'Deterministic policy gradient algorithms,' in ICML , 2014. -
[33][33] R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, 'Multiagent actor-critic for mixed cooperative-competitive environments,' in NeurIPS , 2017. -
[38][38] N. Srinivas, A. Krause, S. M. Kakade, and M. W. Seeger, 'Gaussian process optimization in the bandit setting: No regret and experimental design,' in ICML , 2010. -
[46][46] Y. Burda, H. Edwards, A. J. Storkey, and O. Klimov, 'Exploration by random network distillation,' in ICLR , 2019. -
[69][69] R. Dearden, N. Friedman, and S. J. Russell, 'Bayesian q-learning,' in AAAI , 1998. -
[70][70] P. Auer, N. Cesa-Bianchi, and P. Fischer, 'Finite-time analysis of the multiarmed bandit problem,' Machine learning , vol. 47, no. 2-3, pp. 235-256, 2002. -
[72][72] P. Auer, 'Using confidence bounds for exploitation-exploration tradeoffs,' JMLR , vol. 3, pp. 397-422, 2002. -
[79][79] N. Nikolov, J. Kirschner, F. Berkenkamp, and A. Krause, 'Informationdirected exploration for deep reinforcement learning,' in ICLR , 2019. -
[84][84] C. Oh and A. Cavallaro, 'Learning action representations for selfsupervised visual exploration,' in ICRA , 2019. -
[93][93] T. Zhang, P. Rashidinejad, J. Jiao, Y. Tian, J. Gonzalez, and S. Russell, 'Made: Exploration via maximizing deviation from explored regions,' arXiv preprint arXiv:2106.10268 , 2021. - · arXiv
[97][97] M. Klissarov, R. Islam, K. Khetarpal, and D. Precup, 'Variational state encoding as intrinsic motivation in reinforcement learning,' in TARL Workshop on ICLR , 2019. -
[134][134] S. L. Herbert, S. Bansal, S. Ghosh, and C. J. Tomlin, 'Reachabilitybased safety guarantees using efficient initializations,' in CDC , 2019, pp. 4810-4816. -
[137][137] Y. Guo, J. Choi, M. Moczulski, S. Feng, S. Bengio, M. Norouzi, and H. Lee, 'Memory based trajectory-conditioned policies for learning from sparse rewards,' in NeurIPS , 2020. -
[146][146] J. Zhao, M. Yang, X. Hu, W. Zhou, and H. Li, 'DQMIX: A distributional perspective on multi-agent reinforcement learning,' CoRR , vol. abs/2202.10134, 2022. - · arXiv
[150][150] L. Zheng, J. Chen, J. Wang, J. He, Y. Hu, Y. Chen, C. Fan, Y. Gao, and C. Zhang, 'Episodic multi-agent reinforcement learning with curiositydriven exploration,' in NeurIPS , M. Ranzato, A. Beygelzimer, Y. N. Dauphin, P. Liang, and J. W. Vaughan, Eds., 2021, pp. 3757-3769. -
[156][156] D. Carmel and S. Markovitch, 'Exploration strategies for model-based learning in multi-agent systems: Exploration strategies,' JAAMAS , vol. 2, no. 2, pp. 141-172, 1999. -
[168][168] M. Samvelyan, T. Rashid, C. S. de Witt, G. Farquhar, N. Nardelli, T. G. J. Rudner, C. Hung, P. H. S. Torr, J. N. Foerster, and S. Whiteson, 'The starcraft multi-agent challenge,' pp. 2186-2188, 2019. -
[169][169] G. Farquhar, L. Gustafson, Z. Lin, S. Whiteson, N. Usunier, and G. Synnaeve, 'Growing action spaces,' in ICML , 2020. -
[179][179] K. H. Kim, M. Sano, J. De Freitas, N. Haber, and D. Yamins, 'Active world model learning in agent-rich environments with progress curiosity,' in ICML , 2020. -
[186][186] Z. C. Lipton, X. Li, J. Gao, L. Li, F. Ahmed, and L. Deng, 'Bbqnetworks: Efficient exploration in deep reinforcement learning for taskoriented dialogue systems,' in AAAI , 2018. -
[210][210] R. Zhao, X. Sun, and V. Tresp, 'Maximum entropy-regularized multigoal reinforcement learning,' in ICML , 2019. -
[218][218] V. H. Pong, M. Dalal, S. Lin, A. Nair, S. Bahl, and S. Levine, 'Skew-fit: State-covering self-supervised reinforcement learning,' in ICML , 2020. -
[219][219] A. Sharma, S. Gu, S. Levine, V. Kumar, and K. Hausman, 'Dynamicsaware unsupervised discovery of skills,' in ICLR , 2020. -
[221][221] T. Haarnoja, H. Tang, P. Abbeel, and S. Levine, 'Reinforcement learning with deep energy-based policies,' in ICML , ser. Proceedings of Machine Learning Research, vol. 70, 2017, pp. 1352-1361. -

Sorted by citations

Items 101–194 of 194

Policy Selection and Scheduling of Cyber-Physical Systems with Denial-of-Service Attacks via Reinforcement Learning
Jin, Z., Li, Q., Zhang, H., Liu, Z., & Wang, Z. (2024) · Journal of Advanced Computational Intelligence and Intelligent Informatics · doi:10.20965/jaciii.2024.p0962 · Cited by 1
Self-citation
Phased Continuous Exploration Method for Cooperative Multi-Agent Reinforcement Learning
Kang, J., Hou, Y., Zeng, Y., Chen, Y., Tong, X., Xu, X., & Zhang, Q. (2024) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai59869.2024.00196 · Cited by 1
Research and Application Analysis of Reinforcement Learning Algorithms
Dou, X., & Zhou, R. (2024) · Applied and Computational Engineering · doi:10.54254/2755-2721/2025.18525 · Cited by 1
Meta-Offline and Distributional Multi-Agent RL for Risk-Aware Decision-Making
Eldeeb, E., & Alves, H. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463052 · Cited by 1
Adventurer: exploration with BiGAN for deep reinforcement learning
Liu, Y., & Liu, X. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06600-4 · Cited by 1
Energy aware DBSCAN and mobility aware balanced q-learning based opportunistic routing protocol in MANET
Das, A. D. G., & Devi, E. A. (2025) · Peer-to-Peer Networking and Applications · doi:10.1007/s12083-025-02004-w · Cited by 1
Emergent Braitenberg-style Behaviours for Navigating the ViZDoom 'My Way Home' Labyrinth
Bayer, C., Smith, R. J., & Heywood, M. I. (2025) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3712256.3726331 · Cited by 1
Communication in Multiagent Reinforcement Learning via Counterfactual Message Value
Gao, Z., Qu, C., Hao, Y., & Ke, L. (2025) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2025.3604230 · Cited by 1
Investigating Primacy Bias in Multiagent Reinforcement Learning: An Empirical Study
Li, J., Yang, Y., He, Z., Wu, H., Zhao, C., & Hwang, K. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3560617 · Cited by 1
A safety modulator actor-critic method in model-free safe reinforcement learning and application in UAV hovering
Qi, Q., Yang, X., Ho, D. W. C., & Sun, Y. (2025) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2025.108158 · Cited by 1
Quantum-Inspired Reinforcement Learning for High-Dimensional Optimization Problems
Mutheeswaran, U., Mudalkar, P. K., Gopinath, D., Prathik, A., Gujar, V. B., & Tanu, T. (2025) · International Conference on Innovative Mechanisms for Industry Applications · doi:10.1109/icimia67127.2025.11200933 · Cited by 1
Adaptive model splitting with sample-efficient reinforcement learning for federated learning
Zhang, N. N., Zeng, K., Wen, H., & Shen, T. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.130875 · Cited by 1
Reinforcement Learning and Digital Human Modeling for Multi-objective Factory Layout Planning
Elango, V., Lind, A., Joseph, M. S., Makkar, A., Sandblad, J., Hanson, L., Syberfeldt, A., & Forsman, M. (2025) · Springer series in design and innovation · doi:10.1007/978-981-96-9334-4_44 · Cited by 1
Computing efficiency and latency optimization in intelligent reflecting surface-enhanced device-to-device mobile edge computing via lagrange-dual deep learning
Chhea, K., & Lee, J.-R. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00427-2 · Cited by 1
Hammurabi: Establish Cooperative Order From Pre-Trained Policies in Multi-UAV Networks
Chen, D., He, H., Qi, Q., Wang, J., Han, R., He, B., Zhuang, Z., Fu, Q., Liao, J., & Han, Z. (2026) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2026.3654605 · Cited by 1
Multi-Agent Transfer Learning Based on Evolutionary Algorithms and Dynamic Grid Structures for Industrial Applications
Löppenberg, M., Yuwono, S., & Schwung, A. (2026) · AI · doi:10.3390/ai7020062 · Cited by 1
Prioritizing Latency with Profit: A DRL-Based Admission Control for 5G Network Slices
Chakraborty, P., Asrar, A., Sengupta, J., & Bit, S. D. (2025) · IEEE International Conference on Advanced Networks and Telecommunications Systems · doi:10.1109/ants66931.2025.11430111 · Cited by 1
Learning Optimal Policies With Local Observations for Cooperative Multiagent Reinforcement Learning
Kong, H., Xing, Q., Wang, Q., Chen, H., Niu, R., Duan, Z., Wang, S., Chang, Y., & King, I. (2026) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2026.3673692 · Cited by 1
Utilizing Deep Reinforcement Learning and QLearning algorithms for Improved Ethereum Cybersecurity
Kabanda, G., Chipfumbu, C. T., & Chingoriw, T. (2023) · International Journal of Advanced Networking and Applications · doi:10.35444/ijana.2023.14612 · Cited by 1
Mnemonic Dictionary Learning for Intrinsic Motivation in Reinforcement Learning
Yan, R., Wu, Z., Zhan, Y., Tao, P., Wang, Z., Cai, Y., & Xing, J. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191424 · Cited by 1
Self-citation
Delay Optimization of IoT-Edge Computing in Smart Grid Using Deep Reinforcement Learning
bin Jafri, M. A., Asuhaimi, F. A., Dahlan, A. F., & Hatta, M. K. M. (2023) · IEEE International Conference on Artificial Intelligence in Engineering and Technology (IICAIET) · doi:10.1109/iicaiet59451.2023.10292063 · Cited by 1
Action Prediction for Cooperative Exploration in Multi-agent Reinforcement Learning
Zhang, Y., Feng, D., & Ding, B. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8082-6_28 · Cited by 1
Optimization of Urban Target Area Accessibility for Multi-UAV Data Gathering Based on Deep Reinforcement Learning
Jin, Z., Chen, R., Wu, K., Yu, T., & Fu, L. (2024) · Drones · doi:10.3390/drones8090462 · Cited by 1
Curious Explorer: A Provable Exploration Strategy in Policy Learning
Miani, M., Parton, M., & Romito, M. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3460972 · Cited by 1
Improving Cooperation via Joint Intrinsic Motivation Exploration in Multi-Agent Reinforcement Learning
Du, H., Yang, Z., Wu, Y., & Xie, N. (2024) · International Conference on Data-driven Optimization of Complex Systems (DOCS) · doi:10.1109/docs63458.2024.10704285 · Cited by 2
Robust adaptive maximum-entropy linear quadratic regulator
Kamel, A., Esmzad, R., Niknejad, N., & Modares, H. (2025) · IFAC Journal of Systems and Control · doi:10.1016/j.ifacsc.2025.100305 · Cited by 1
Learning to Coordinate With Different Teammates via Team Probing
Ding, H., Jia, C., Zhang, Z., Guan, C., Chen, F., Yuan, L., & Yu, Y. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3563773 · Cited by 1
CoMARS: Coordinated Multi-Agent Rollout Strategy for Distributed Channel Access in Clustered IoT Networks
Xia, N., Wang, Z., Yin, Y., Wan, J., Wang, Y., Chen, L., & Wei, S. (2025) · IEEE Transactions on Green Communications and Networking · doi:10.1109/tgcn.2025.3574231 · Cited by 1
Self-citation
AI-in-the-Loop Resilience Enhancement of Cyber–Physical Electric Power Systems Against Electromagnetic Pulse Attacks
Zhang, R., Dehghanian, P., & Li, Y. (2025) · IEEE Transactions on Reliability · doi:10.1109/tr.2025.3575116 · Cited by 1
Collaborative Anti-Jamming for Multi-UAV Communications Based on Deep Reinforcement Learning
Yan, B. S., Jia, W., Xie, W., Wang, X., Xiong, T., & Ke, Z. (2025) · International Conference on Computer and Communication Systems (ICCCS) · doi:10.1109/icccs65393.2025.11069790 · Cited by 1
Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: Present, Challenges, and Future ProspectsPreprint
Kadi, H. A., & Terzić, K. (2022) · Preprints.org · doi:10.20944/preprints202212.0305.v1 · Cited by 1
Deep Reinforcement Learning for AoI Minimization in UAV-aided Data Collection for WSN and IoT: a SurveyPreprint
Amodu, O. A., Jarray, C., Mahmood, R. A. R., Althumali, H., Bukar, U. A., Nordin, R., & Luong, N. C. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4725427 · Cited by 1
The State of AI-Empowered Backscatter Communications: A Comprehensive SurveyPreprint
Ahmed, M., Hussain, T., Ali, K., Mirza, M. A., Khan, W. U., Ihsan, A., Xu, F., & Han, Z. (2023) · doi:10.36227/techrxiv.22245958
Projection Exploration for Multi-agent Reinforcement LearningPreprint
Tang, H., Liu, J., Wang, Z., & Gao, Z. (2023) · Research Square · doi:10.21203/rs.3.rs-2759603/v1
Self-citation
PRACM: Predictive Rewards for Actor-Critic with Mixing Function in Multi-Agent Reinforcement Learning
Yu, S., Liu, B., Zhu, W., & Liu, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40292-0_7
Generative Subgoal Oriented Multi-Agent Reinforcement Learning Through Potential FieldPreprint
Jiang, H., Li, S., Liu, Y., Zhang, J., Xu, X., & Liu, D. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4612788
Field Robot Self-Exploration Based on Deep Reinforcement Learning and Safety Control MechanismPreprint
Kang, J., Zhang, Z., He, L., & Chen, X. (2024) · Research Square · doi:10.21203/rs.3.rs-4254664/v1
Field Robot Self-Exploration Based on Deep Reinforcement Learning and Obstacle Avoidance StrategyPreprint
Kang, J., Zhang, Z., He, L., & Chen, X. (2024) · Research Square · doi:10.21203/rs.3.rs-4343274/v1
DHAA: Distributed heuristic action aware multi-agent path finding in high density scene
Zhou, D., Pang, Z., & Li, W. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19951-w
Brain-Inspired Parallel Inference Learning for Complex Decision-Making
Jia, T., Fan, C., Wang, Q., Han, Y., & Wu, X. (2025) · Lecture notes in computer science · doi:10.1007/978-981-97-8705-0_28
An ADP-based robust control scheme for nonaffine nonlinear systems with uncertainties and input constraints
Luo, S., Zhang, K., & Xue, W. (2025) · Chinese Physics B · doi:10.1088/1674-1056/adbee1
CPIG: Leveraging Consistency Policy With Intention Guidance for Multiagent Exploration
Fu, Y., Zhu, Y., Li, H., Zhao, Z., Chai, J., & Zhao, D. (2025) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2025.3578001 · Cited by 1
navTD3: An End-to-End Learning Approach for Indoor Navigation
Mutlu, F. B., Cicek, D. C., Guler, R. B., & Kozat, S. S. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-04329-8
Provably efficient information-directed sampling algorithms for multi-agent reinforcement learning
Zhang, Q. E., Bai, C., Hu, S., Wang, Z., & Li, X. (2025) · Artificial Intelligence · doi:10.1016/j.artint.2025.104392 · Cited by 2
Self-citation
How does stochasticity in learning impact the accumulation of knowledge and the evolution of learning?
Maisonneuve, L., & Lehmann, L. (2026) · Evolutionary Human Sciences · doi:10.1017/ehs.2026.10044
AI for ESG Governance in South Africa: A Systematic Review of Financial Intelligence ApproachesPreprint
Mathebula, M., Modupe, A., Marivate, V. N., & du Toit, E. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5443142
EMERALD-O: efficient multi-agent reinforcement learning framework for optimised deep learning hyperparameter tuning and selection
Akhila, V. H., Chacko, A. M., & Kumaraguru, P. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06878-4
The Role of Environment Complexity in Exploration Strategies: A Comparative Analysis of CartPole and Lunar Lander
Chowdhury, A., & Joshi, P. (2025) · IEEE Wireless Antenna and Microwave Symposium (WAMS) · doi:10.1109/wams64402.2025.11158892
Reinforcement learning for traffic signal control: advancing efficiency through hybrid exploration strategies
Thadikamalla, S., Joshi, P., & Gangadharan, D. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07892-6 · Cited by 1
Multi-Agent Cooperative Exploration via Weighted Conditional Mutual Information in Sparse Reward Tasks
Liu, J., & Li, F. (2025) · Chinese Control Conference (CCC) · doi:10.23919/ccc64809.2025.11178392
Self-citation
Enhancing the Policy Generalization on OOD Tasks via Latent Variable Distribution Enhancement Sampler
Li, S., Lin, J., Yang, X., Zhang, H., & Zhao, P. F. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3614724
Value of Information-Enhanced Exploration in Bootstrapped DQN
Plataniotis, S., Akasiadis, C., & Chalkiadakis, G. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11229044
A2CHoldem: An Intelligent Agent for Texas Hold’em Based on Deep Reinforcement Learning
Guo, Q., Wang, Y., Wang, S., & Zhao, T. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227788
Universal Stabilization for Maximum Entropy Optimization in Reinforcement Learning
Chen, X., Li, Y., Cao, X., Chen, H., Yao, H., An, B., & Chang, Y. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3626050
Gradient-driven exploration and pattern matching experience replay for efficient UAV path planning: Flying over or around?
Jin, Z., Chen, R., Wu, K., Liang, D., & Yan, L. (2025) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2025.102156
From Blind Exploration to Structured Search: A LLM-Enhanced MARL Framework via Subgoal Planning Forest
Zhang, J., Wang, H., Lv, K., Lin, Y., & Han, S. (2025) · IEEE International Conference on Unmanned Systems (ICUS) · doi:10.1109/icus66297.2025.11294042
Fine-tuning of large language models for obstacle circumvention in path planning via chain-of-thought reasoning
Wang, R., Li, B., Shi, X., Wen, Y., Cheng, R., Yu, Y., Zhang, N., Tang, H., Zheng, Y., & Hao, J. (2025) · Scientia Sinica Technologica · doi:10.1360/sst-2025-0219
Self-citation
Deep Reinforcement Learning for UAV Wireless Charging and Trajectory Planning: A Review
Shaikh, P. W., & Mouftah, H. T. (2025) · IEEE Vehicular Technology Conference · doi:10.1109/vtc2025-fall65116.2025.11310168
CAUSALNAV: A Long-Term Embodied Navigation System for Autonomous Mobile Robots in Dynamic Outdoor Scenarios
Duan, H., Luo, S., Deng, Z., Chen, Y., Chiang, Y., Liu, Y., Liu, F., & Wang, X. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3653283
A Systematic Mapping on Applications of Deep Reinforcement Learning in Medical Imaging using Bibliometric Analysis
Kharb, A., & Chaudhary, P. (2025) · International Conference on Computational Intelligence and Computing Applications (ICCICA) · doi:10.1109/iccica67008.2025.11337435
Unveiling New Approaches to Hybrid Intrinsic-Extrinsic Rewards in Actor-Critic Reinforcement Learning: A Systematic Review of Engagement, Performance, and Generalization
Hatta, Utami, Setyanto, & Kusrini (2025) · International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS) · doi:10.1109/icimcis68501.2025.11327330
Hierarchical Deep Deterministic Policy Gradient for Autonomous Maze Navigation of Mobile RobotsPreprint
WENJIE, H., Zhou, Y., & Ho, H. W. (2026) · Research Square · doi:10.21203/rs.3.rs-7976730/v1
TNCOA: Efficient Exploration via Observation‐Action Constraint on Trajectory‐Based Intrinsic Reward
Ma, J., Ma, H., & Zhang, Y. (2026) · CAAI Transactions on Intelligence Technology · doi:10.1049/cit2.70100
Conflict Prioritized-Based Experience Replay Soft Actor-Critic Algorithm for Unsignalized Intersections Coordination
Dai, J., Cui, Y., Liu, W., & Li, P. (2025) · CAA International Conference on Vehicular Control and Intelligence (CVCI) · doi:10.1109/cvci66304.2025.11348538
Adaptive Exploration in Deep Reinforcement Learning via Uncertainty-Aware State Representation
Kolla, T., Babu, T., & Nair, R. R. (2025) · International Conference on Advancements in Computing (ICAC) · doi:10.1109/icac69156.2025.11361472
Nested Actors and Critics With Uncertainty Parameter in Robot Path Planning
Gu, Y., Meng, F., Chen, Y., Miao, R., Hao, J., & Lv, J. (2026) · Journal of Field Robotics · doi:10.1002/rob.70175
Self-citation
Reasoning Under Uncertainty in Multi-Agent Reinforcement Learning (MARL) with Upper Confidence Bound (UCB) Technique
Islam, K. S., Dutta, A. K., Anwar, A. S., & Carvalho, M. M. (2026) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc67433.2026.11393838
Adaptive Incentive Mechanism for Multi-Agent Systems in Adversarial Environments
Chen, M., Zhang, X., Wang, L., & Zhang, J. (2025) · International Conference on Computer, Internet of Things and Smart City (CIoTSC) · doi:10.1109/ciotsc67482.2025.11413055
Intelligent SDN Traffic Engineering with MADRL
Dev, A., Rout, S., Patnaik, H. K., Singh, S., & Patra, S. S. (2026) · International Conference on Mobile Computing and Sustainable Informatics (ICMCSI) · doi:10.1109/icmcsi67283.2026.11412947
A Multi-USV-UAV Environmental Exploration Method Based on Cross-Modal Attention Fusion
Chen, C., Liu, W., & Cai, L. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3675926
People’s livelihood demand prediction based on improved generative adversarial networks and reinforcement learning
Ding, X., Luo, Y., Tuo, C., & Gao, J. (2026) · Journal of Information and Telecommunication · doi:10.1080/24751839.2026.2643530
Improving Cooperation in Multi-agent Deep Reinforcement Learning for Mobile Robotics
Toquebiau, M. (2024) · theses.fr (ABES) · doi:10.70675/3385d0bbzc66cz4fceza9e0z8696ff61ff4e
Deep reinforcement learning for integrated data assimilation and valve control in water distribution systems
Jeung, M., Jang, I., Bae, H.-S., Kim, K., Yu, J., Kang, M.-S., Her, Y., Hong, S. M., & Baek, S.-S. (2026) · Water Research X · doi:10.1016/j.wroa.2026.100540
Adversarial imitation reinforcement learning energy management for range-extended electric vehicles with quantitative support from an improved unified fuel model
Song, D., Liu, C., Wang, X., Li, J., & Zeng, X. (2026) · Applied Thermal Engineering · doi:10.1016/j.applthermaleng.2026.130955
Dynamic Channel Optimization of Reconfigurable Intelligent Surface in Communications
Zhang, Y. (2026) · International Journal of Interdisciplinary Telecommunications and Networking · doi:10.4018/ijitn.409338
Experience Extractor for Adaptive Tradeoff Between Exploration and Exploitation in Reinforcement Learning
Yi, Z., Wu, Z., Xie, Y., Chen, M., & Dai, Y. (2026) · Mathematics · doi:10.3390/math14101624
Hierarchical Reinforcement Learning-Based Multi-UAV Dynamic Game Cooperative Decision-Making Optimization in Game-Theoretic Frameworks
Shen, Y., Hu, R., & Zhang, Y. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-95-6557-3_54
Hybrid-Constrained Multi-Agent Reinforcement Learning for Collaborative Dual-Aircraft Scenarios
Liang, H., Gao, Y., Chang, H., Zhao, Z., & Xi, S. (2026) · IEEE Aerospace Conference (AERO) · doi:10.1109/aero66936.2026.11519866
ALA-MARL: Multiobjective Edge Server Placement Based on Artificial Lemming Algorithm and Multiagent Reinforcement Learning in Mobile Edge Computing
Zhou, Z., Zheng, T., & Liu, G. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3697572
Collaborative Torsional Vibration Suppression Strategy for Hybrid Electric Vehicles based on Harmonic-Residual Decoupling and Multi-Agent Deep Reinforcement LearningPreprint
Huang, Y., Yan, Z., Tan, Z., Yao, M., Du, H., & Zhang, N. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6839607
Autonomous Obstacle Avoidance and Path Planning for Mobile Robots Based on Reinforcement Learning
Yang, B., Zhang, Q. G., Liu, Y., & Miao, X. (2026) · International Annual Conference on Complex Systems and Intelligent Science (CSIS-IAC) · doi:10.1109/csis-iac70275.2026.11585118
A comprehensive survey on deep reinforcement learning for autonomous vehicles urban driving: Approaches, challenges, and future directions
Amraouy, E. H., Lablali, O., Yahyaouy, A., Chaoui, H., Faquir, S., & Gualous, H. (2026) · Scientific African · doi:10.1016/j.sciaf.2026.e03522
Design for manufacturing: a manufacturability knowledge-integrated reinforcement learning framework for free-form pipe routing in aeroengines
Wang, C., Wang, Z., Zhang, S., Xiang, Y., Li, Z., Li, L., & Tan, J. (2026) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2026.101166
Self-citation
Adaptive hybrid storage and retrieval management in blockchain-IPFS systems: a multi-agent deep reinforcement learning approach
Bahrampour, S., Rasouli, M. R., & Fathian, M. (2026) · Cluster Computing · doi:10.1007/s10586-026-06165-y
An improved soft actor-retrospective critic algorithm based on the dynamic fusion priority experience replay for UAV path planning
Wang, L., Meng, C., Li, G., Kong, X., & Лин, Г. (2026) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2026.105639
Adversarial Geometric Traps: A DRL-Driven ROS 2 Framework for Robust Mapless Navigation in Dead-End Scenarios
Farouq, A. A., Hosoda, Y., & Lee, J.-H. (2026) · International Conference on Ubiquitous Robots (UR) · doi:10.1109/ur69537.2026.11626792
Multi-Armed Bandits Convergence Acceleration Using Cluster Normalizations
Strenadko, V., Kiselev, D., & Makarov, I. (2026) · IEEE Access · doi:10.1109/access.2026.3718576
A modular tool-driven RAG framework for engineering test documentation
Trappey, A. J. C., Huang, W.-Y., & Chien, C.-H. (2026) · Journal of Engineering Design · doi:10.1080/09544828.2026.2698389
Joint AoI and SWIPT-Aware Scheduling via Multi- Agent Deep Reinforcement Learning
Liu, K.-T., Liu, J., & Chang, W.-L. (2026) · International Conference on Electronics, Communications and Computer Science (ICECCS) · doi:10.1109/iceccs70432.2026.11651135
Self-citation
Task-Guided Multi-UAV Cooperative Multi-Target Tracking with Gaussian Process-Based Value Correction
Li, W., Chen, X., & Li, X. (2026) · Drones · doi:10.3390/drones10090676
Joint Optimization of Delay and Energy Efficiency for UAV Task Offloading and Cooperative Scheduling
Zhang, T., Xu, T., Liu, Z., & Ben, C. (2026) · Journal of Circuits Systems and Computers · doi:10.1142/s0218126626502695
Autonomous Decision‐Making for Multi‐Spacecraft Orbital pursuit–evasion Games via Regularized MADDPG
Yu, L., Peng, C., Wang, H., & Ren, W. (2026) · IET Control Theory and Applications · doi:10.1049/cth2.70186
Shared Value Representations Drive Exploration-Induced Cooperation Collapse in Spatial Social DilemmasPreprint
Weng, Y.-N., & Lee, H. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7629485
Relation-aware evolutionary reinforcement learning for scheduling reconfigurable manufacturing systems with multi-module configurations
Li, H., Li, X., Gao, L., Liu, Q., & Zhang, C. (2026) · Advanced Engineering Informatics · doi:10.1016/j.aei.2026.105429

Authors: Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z.
Affiliations: Tianjin University (CN)
Shanghai Artificial Intelligence Laboratory (CN)
Harbin Institute of Technology (CN)
Northwestern Polytechnical University (CN)
ORCID: Hao, Jianye
Yang, Tianpei
Tang, Hongyao
Bai, Chenjia
Liu, Jinyi
Meng, Zhaopeng
Liu, Peng
Wang, Zhen
Venue: arXiv
Publisher: arXiv
Published: 2021
Volume: 35
Issue: 7
Pages: 8762-8782
ISSN: 2162-237X (ISSN-L)
DOI: 10.48550/arxiv.2109.06668
arXiv: 2109.06668
OpenAlex: W4319165238
Cited by: 192
References: 217
Citations by year: 2026: 42 · 2025: 77 · 2024: 48 · 2023: 23 · 2022: 1 · 2021: 1
FWCI: 17.91
Type: journal article
Language: en
Open access: green
arxiv.org
Licence: arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations: IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
PubMed (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: Tencent
National Natural Science Foundation of China (U22B2036, 11931015, 62025602, U1836214)
National Science Fund for Distinguished Young Scholars (62025602)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
Keywords: sample inefficiency, intrinsic motivation, sparse rewards

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks