Hierarchical Reinforcement Learning: A Survey and Open Research Challenges
Open accessThe DOI matches its Crossref record; 208 of the 208 references it cites were found in a registry or an index.
Abstract
Reinforcement learning (RL) allows an agent to solve sequential decision-making problems by interacting with an environment in a trial-and-error fashion. When these environments are very complex, pure random exploration of possible solutions often fails, or is very sample inefficient, requiring an unreasonable amount of interaction with the environment. Hierarchical reinforcement learning (HRL) utilizes forms of temporal- and state-abstractions in order to tackle these challenges, while simultaneously paving the road for behavior reuse and increased interpretability of RL systems. In this survey paper we first introduce a selection of problem-specific approaches, which provided insight in how to utilize often handcrafted abstractions in specific task settings. We then introduce the Options framework, which provides a more generic approach, allowing abstractions to be discovered and learned semi-automatically. Afterwards we introduce the goal-conditional approach, which allows sub-behaviors to be embedded in a continuous space. In order to further advance the development of HRL agents, capable of simultaneously learning abstractions and how to use them, solely from interaction with complex high dimensional environments, we also identify a set of promising research directions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Pretraining is a common approach to increase the stability of the policy in hierarchical reinforcement learning [24].”
From Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation (2023) · cites this paper as [24]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Research works printed but not linked: 53
- Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press.
- Li, Y. (2017). Deep Reinforcement Learning: An Overview. arXiv.
- Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv.
- Badia, A.P., Piot, B., Kapturowski, S., Sprechmann, P., Vitvitskyi, A., Guo, D., and Blundell, C. (2020). Agent57: Outperforming the Atari Human Benchmark. arXiv.
- OpenAI (2019). Dota 2 with Large Scale Deep Reinforcement Learning. arXiv.
- OpenAI (2019). Learning Dexterous In-Hand Manipulation. arXiv.
- Kalashnikov, D., Irpan, A., Pastor, P., Ibarz, J., Herzog, A., Jang, E., Quillen, D., Holly, E., Kalakrishnan, M., and Vanhoucke, V. (2018). QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation. arXiv.
- Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., and Mané, D. (2016). Concrete problems in AI safety. arXiv.
- Burda, Y., Edwards, H., Storkey, A., and Klimov, O. (2018). Exploration by Random Network Distillation. arXiv.
- Bellemare, M.G., Srinivasan, S., Ostrovski, G., Schaul, T., Saxton, D., and Munos, R. (2016). Unifying Count-Based Exploration and Intrinsic Motivation. arXiv.
- Nachum, O., Tang, H., Lu, X., Gu, S., Lee, H., and Levine, S. (2019). Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?. arXiv.
- Mott, A., Zoran, D., Chrzanowski, M., Wierstra, D., and Rezende, D.J. (2019). Towards Interpretable Reinforcement Learning Using Attention Augmented Agents. arXiv.
- Garnelo, M., Arulkumaran, K., and Shanahan, M. (2016). Towards Deep Symbolic Reinforcement Learning. arXiv.
- Houthooft, R., Chen, X., Chen, X., Duan, Y., Schulman, J., Turck, F.D., and Abbeel, P. (2016). VIME: Variational Information Maximizing Exploration. arXiv.
- Harutyunyan, A., Dabney, W., Mesnard, T., Heess, N., Azar, M.G., Piot, B., van Hasselt, H., Singh, S., Wayne, G., and Precup, D. (2019). Hindsight Credit Assignment. arXiv.
- Rummery, G.A., and Niranjan, M. (1994). On-Line Q-Learning Using Connectionist Systems, University of Cambridge, Department of Engineering.
- Donahue, J., and Simonyan, K. (2019). Large Scale Adversarial Representation Learning. arXiv.
- Radford, A., Metz, L., and Chintala, S. (2016). Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. arXiv.
- van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K. (2016). WaveNet: A Generative Model for Raw Audio. arXiv.
- Sutskever, I., Vinyals, O., and Le, Q.V. (2014). Sequence to Sequence Learning with Neural Networks. arXiv.
- Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to Align and Translate. arXiv.
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention Is All You Need. arXiv.
- Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8–13). Generative Adversarial Nets. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
- Van Hasselt, H., Doron, Y., Strub, F., Hessel, M., Sonnerat, N., and Modayil, J. (2018). Deep Reinforcement Learning and the Deadly Triad. arXiv.
- Zhang, S., and Sutton, R.S. (2017). A Deeper Look at Experience Replay. arXiv.
- Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2016). Prioritized Experience Replay. arXiv.
- Dabney, W., Rowland, M., Bellemare, M., and Munos, R. (2018, January 2–7). Distributional Reinforcement Learning with Quantile Regression. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA. · DOI · Paper page
- Ferns, N., Panangaden, P., and Precup, D. (2004, January 9–11). Metrics for Finite Markov Decision Processes. Proceedings of the Twentieth Conference on Uncertainty in Artificial Intelligence, Banff, AB, Canada.
- McGovern, A., Sutton, R.S., and Fagg, A.H. (1997, January 19–21). Roles of macro-actions in accelerating reinforcement learning. Proceedings of the Grace Hopper Celebration of Women in Computing, San Jose, CA, USA.
- Heess, N., Wayne, G., Tassa, Y., Lillicrap, T., Riedmiller, M., and Silver, D. (2016). Learning and transfer of modulated locomotor controllers. arXiv.
- Eysenbach, B., Gupta, A., Ibarz, J., and Levine, S. (2019). Diversity Is All You Need: Learning Skills without a Reward Function. arXiv.
- Abel, D., Umbanhowar, N., Khetarpal, K., Arumugam, D., Precup, D., and Littman, M.L. (2020, January 26–28). Value Preserving State-Action Abstractions. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, Palermo, Italy.
- Fox, R., Krishnan, S., Stoica, I., and Goldberg, K. (2017). Multi-level discovery of deep options. arXiv.
- Ng, A.Y., and Russell, S.J. (July, January 29). Algorithms for Inverse Reinforcement Learning. Proceedings of the International Conference on Machine Learning, Stanford, CA, USA.
- Pan, X., Ohn-Bar, E., Rhinehart, N., Xu, Y., Shen, Y., and Kitani, K.M. (2018). Human-Interactive Subgoal Supervision for Efficient Inverse Reinforcement Learning. arXiv.
- Krishnan, S., Garg, A., Liaw, R., Miller, L., Pokorny, F.T., and Goldberg, K. (2016). Hirl: Hierarchical inverse reinforcement learning for long-horizon tasks with delayed rewards. arXiv.
- Kumar, A., Swersky, K., and Hinton, G. (2017, January 9). Feudal Learning for Large Discrete Action Spaces with Recursive Substructure. Proceedings of the NIPS Workshop Hierarchical Reinforcement Learning, Long Beach, CA, USA.
- Kulkarni, T.D., Saeedi, A., Gautam, S., and Gershman, S.J. (2016). Deep successor reinforcement learning. arXiv.
- Lakshminarayanan, A.S., Krishnamurthy, R., Kumar, P., and Ravindran, B. (2016). Option discovery in hierarchical reinforcement learning using spatio-temporal clustering. arXiv.
- Da Silva, B.C., Konidaris, G., and Barto, A.G. (July, January 26). Learning Parameterized Skills. Proceedings of the International Conference on Machine Learning, Edinburgh, Scotland.
- Wang, J.X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J.Z., Munos, R., Blundell, C., Kumaran, D., and Botvinick, M. (2017). Learning to Reinforcement Learn. arXiv.
- Duan, Y., Schulman, J., Chen, X., Bartlett, P.L., Sutskever, I., and Abbeel, P. (2016). RL$2^$: Fast Reinforcement Learning via Slow Reinforcement Learning. arXiv.
- Li, A.C., Florensa, C., Clavera, I., and Abbeel, P. (2020). Sub-Policy Adaptation for Hierarchical Reinforcement Learning. arXiv.
- Sohn, S., Woo, H., Choi, J., and Lee, H. (2020). Meta Reinforcement Learning with Autonomous Inference of Subtask Dependencies. arXiv.
- Bengio, E., Thomas, V., Pineau, J., Precup, D., and Bengio, Y. (2017). Independently Controllable Features. arXiv.
- Van Seijen, H., Fatemi, M., Romoff, J., Laroche, R., Barnes, T., and Tsang, J. (2017). Hybrid reward architecture for reinforcement learning. arXiv.
- Sukhbaatar, S., Denton, E., Szlam, A., and Fergus, R. (2018). Learning Goal Embeddings via Self-Play for Hierarchical Reinforcement Learning. arXiv.
- Gregor, K., Rezende, D.J., and Wierstra, D. (2016). Variational Intrinsic Control. arXiv.
- Achiam, J., Edwards, H., Amodei, D., and Abbeel, P. (2018). Variational option discovery algorithms. arXiv.
- Beattie, C., Leibo, J.Z., Teplyashin, D., Ward, T., Wainwright, M., Küttler, H., Lefrancq, A., Green, S., Valdés, V., and Sadik, A. (2016). Deepmind lab. arXiv.
- Vinyals, O., Ewalds, T., Bartunov, S., Georgiev, P., Vezhnevets, A.S., Yeo, M., Makhzani, A., Küttler, H., Agapiou, J., and Schrittwieser, J. (2017). Starcraft ii: A new challenge for reinforcement learning. arXiv.
- Osband, I., Doron, Y., Hessel, M., Aslanides, J., Sezener, E., Saraiva, A., McKinney, K., Lattimore, T., Szepezvari, C., and Singh, S. (2019). Behaviour Suite for Reinforcement Learning. arXiv.
- Shang, W., Trott, A., Zheng, S., Xiong, C., and Socher, R. (2019). Learning World Graphs to Accelerate Hierarchical Reinforcement Learning. arXiv.
Cited by
Al-Hamadani, M. N. A., Fadhel, M. A., Alzubaidi, L., & Harangi, B. (2024) · Sensors · doi:10.3390/s24082461 · Cited by 123
A sensory–motor theory of the neocortex
Rao, R. P. N. (2024) · Nature Neuroscience · doi:10.1038/s41593-024-01673-9 · Cited by 103
Reinforcement learning for predictive maintenance: a systematic technical review
Siraskar, R., Kumar, S., Patil, S., Bongale, A. M., & Kotecha, K. V. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10468-6 · Cited by 92
Deep reinforcement learning challenges and opportunities for urban water systems
Negm, A. S., Ma, X., & Aggidis, G. (2024) · Water Research · doi:10.1016/j.watres.2024.121145 · Cited by 70
Reinforcement learning for vehicle-to-grid: A review
Xie, H., Song, G., Shi, Z., Zhang, J., Lin, Z., Yu, Q., Fu, H., Song, X., & Zhang, H. (2025) · Advances in Applied Energy · doi:10.1016/j.adapen.2025.100214 · Cited by 81
Solving combinatorial optimization problems over graphs with BERT-Based Deep Reinforcement Learning
Wang, Q., Lai, K. H., & Tang, C. (2022) · Information Sciences · doi:10.1016/j.ins.2022.11.073 · Cited by 49
Toward QoS Monitoring in IoT Edge Devices Driven Healthcare—A Systematic Literature Review
Younas, M. I., Iqbal, M. J., Aziz, A., & Sodhro, A. H. (2023) · Sensors · doi:10.3390/s23218885 · Cited by 45
Rao, R. P. N., Gklezakos, D. C., & Sathish, V. (2023) · Neural Computation · doi:10.1162/neco_a_01627 · Cited by 53
Sun, Q., Zhang, L., Yu, H., Zhang, W., Mei, Y., & Xiong, H. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599839 · Cited by 48
Recent Advances in Reinforcement Learning for Chemical Process Control
Devarakonda, V. S., Sun, W., Tang, X., & Tian, Y. (2025) · Processes · doi:10.3390/pr13061791 · Cited by 43
Gradient Routing: Masking Gradients to Localize Computation in Neural Networks
Cloud, A., Goldman-Wetzler, J., Wybitul, E., Miller, J., & Turner, A. M. (2024) · arXiv · doi:10.48550/arxiv.2410.04332
Sun, R., Xi, Y., Stefanidis, A., Jiang, Z., & Su, J. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-01884-y · Cited by 30
Ramezani, M., & Atashgah, M. A. A. (2024) · Drones · doi:10.3390/drones8070283 · Cited by 35
Li, L., Tan, R., Fang, J., Xue, J., & Lv, C. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128736 · Cited by 32
Hierarchical generative modelling for autonomous robots
Yuan, K., Sajid, N., Friston, K., & Li, Z. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00752-z · Cited by 23
“To overcome the aforementioned limitations, learning approaches—such as hierarchical reinforcement learning 13 —is an alternative approach to accomplish tasks that require solving a discrete sequence of sub-tasks in a close-loop fashion, such as path-following for quadruped locomotion 14 ,…”
Hierarchical Coordination Multi-Agent Reinforcement Learning With Spatio-Temporal Abstraction
Ma, T., Peng, K., Rong, H., Qian, Y., & Al-Nabhan, N. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2023.3309738 · Cited by 26
Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation
Schmalstieg, F., Honerkamp, D., Welschehold, T., & Valada, A. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3329619 · Cited by 21
Jeng, S.-L., & Chiang, C. (2023) · Sensors · doi:10.3390/s23208651 · Cited by 25
Rizinski, M., & Trajanov, D. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.069678 · Cited by 36
Hierarchical control over foraging behavior by anterior cingulate cortex
Alejandro, R. J., & Holroyd, C. B. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105623 · Cited by 18
Zhou, H., Elsayed, M., Bavand, M., Gaigalas, R., Furr, S., & Erol‐Kantarci, M. (2024) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2024.3435850 · Cited by 20
Adamu, A. I., Donta, P. K., Ali, D. M., Sarang, S., Stojanović, G. M., & Sarnin, S. S. (2025) · IEEE Access · doi:10.1109/access.2025.3631230 · Cited by 25
Item-Difficulty-Aware Learning Path Recommendation: From a Real Walking Perspective
Zhang, H., Shen, S., Xu, B., Huang, Z., Wu, J., Sha, J. J., & Wang, S. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671947 · Cited by 14
Hierarchical Reinforcement Learning for Power Network Topology ControlPreprint
Manczak, B., Viebahn, J., & van Hoof, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.02129 · Cited by 19
Ye, M., Zhao, C., Wen, P. P., Wang, Y., Wang, X., & Qiu, H. (2024) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2024.3402275 · Cited by 13
Deineko, E., Jungnickel, P., & Kehrt, C. (2024) · Applied Sciences · doi:10.3390/app14198642 · Cited by 14
Zhou, X., Yuan, Y., Yang, S., & Hao, J. (2025) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2025.3529902 · Cited by 15
Li, J., Xie, H., Zhang, J., Li, L., Song, G., Fu, H., Chen, P., Liu, C., Zhang, L., Shi, Z., et al. (2025) · Advances in Applied Energy · doi:10.1016/j.adapen.2025.100239 · Cited by 15
AI-driven Orchestration for 6G Networking: the Hexa-X vision
Pérez-Valero, J., Virdis, A., Sánchez, A. G., Ntogkas, C., Serrano, P., Landi, G., Kukliński, S., Morin, C., Pavón, I. L., & Sayadi, B. (2022) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/gcwkshps56602.2022.10008726 · Cited by 12
Resilience-based explainable reinforcement learning in chemical process safety
Szatmári, K., Horváth, G., Németh, S., Bai, W., & Kummer, A. (2024) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2024.108849 · Cited by 13
Babar, A., Halabi, T., & Zulkernine, M. (2025) · ACM Transactions on Cyber-Physical Systems · doi:10.1145/3765622 · Cited by 16
AI practical wisdom and compassion
Graves, M. (2025) · AI and Ethics · doi:10.1007/s43681-025-00877-4 · Cited by 9
Lawson, D., & Qureshi, A. H. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341628 · Cited by 15
Wang, J., Lu, Z., Hong, X., Wu, Z., & Li, W. (2025) · Journal of Marine Science and Engineering · doi:10.3390/jmse13050843 · Cited by 10
Enhanced hierarchical reinforcement learning for co-optimization of HVAC system operations
Zhou, X., Li, J., Mo, H., Yan, J., Liang, L., & Pan, D. (2025) · Journal of Building Engineering · doi:10.1016/j.jobe.2025.112663 · Cited by 8
Hieros: Hierarchical Imagination on Structured State Space Sequence World ModelsPreprint
Mattes, P., Schlößer, R., & Herbrich, R. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.05167 · Cited by 10
Chen, X., Yang, Y., Xu, S., Fu, S., & Yang, D. (2024) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc62350.2024.10587952 · Cited by 7
A Hierarchical Reinforcement Learning Framework for UAV Path Planning in Tactical Environments
Alpdemir, M. N. (2023) · Turkish Journal of Science and Technology · doi:10.55525/tjst.1219845 · Cited by 7
Liu, C., Cong, J., Chen, B., Jin, Y., & Zhu, E. (2025) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2025.3631239 · Cited by 11
AI-driven intrusion detection for UAV in Smart Urban ecosystems: A comprehensive survey
Khanfor, A., Hamadi, R., Lasla, N., & Ghazzai, H. (2026) · Computer Communications · doi:10.1016/j.comcom.2026.108435 · Cited by 7
Four-Legged Gait Control via the Fusion of Computer Vision and Reinforcement Learning
Dassori, I., Adams, M., & Vasquez, J. E. (2024) · International Conference on Information Fusion (FUSION) · doi:10.23919/fusion59988.2024.10706406 · Cited by 6
Carvalho, D. S., Santos, P. A., & Melo, F. S. (2025) · Artificial Intelligence · doi:10.1016/j.artint.2025.104382 · Cited by 8
Reinforcement Learning-Based Collision Avoidance of Kinova Gen3 Robot for Ball Balancing Task
Kadam, A. C., & Patil, S. M. (2024) · International Research Journal of Multidisciplinary Scope · doi:10.47857/irjms.2024.v05i01.0213 · Cited by 5
Optimizing Information Freshness in Uplink Multiuser MIMO Networks With Partial Observations
Liu, J., Wang, Q., & Chen, H. H. (2024) · IEEE Transactions on Communications · doi:10.1109/tcomm.2024.3487799 · Cited by 5
Evaluating machine learning-based routing algorithms on various wireless network topologies
Turlykozhayeva, D., Akhtanov, S., Zhexebay, D., Ussipov, N., Baigaliyeva, A., Wójcik, W., & Boranbayeva, N. (2024) · Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments (WILGA) · doi:10.1117/12.3058676 · Cited by 5
HCPI-HRL: Human Causal Perception and Inference-driven Hierarchical Reinforcement Learning
Chen, B., Cao, Z., Mayer, W., Stumptner, M., & Kowalczyk, R. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107318 · Cited by 6
Hierarchical Learning for Robotic Assembly Tasks Leveraging Learning from Demonstration
Singh, S., Chang, Q., & Tian, Y. (2025) · Advanced Robotics Research · doi:10.1002/adrr.202400024 · Cited by 6
Hierarchical optimization of reservoir development strategy based on reinforcement learning
Wang, H., Zhang, K., Chen, N., Wang, Z., Xin, G., Liu, C., Zhang, L., Zhou, W., & Liu, C. (2023) · Geoenergy Science and Engineering · doi:10.1016/j.geoen.2023.211678 · Cited by 5
Exploiting Multiple Abstractions in Episodic RL via Reward Shaping
Cipollone, R., Giacomo, G. D., Favorito, M., Iocchi, L., & Patrizi, F. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i6.25881 · Cited by 5
Ferreira, R. P., Schubert, E., & Scotti, A. (2025) · Journal of Manufacturing and Materials Processing · doi:10.3390/jmmp9040107 · Cited by 4
Context-Aware Hierarchical Learning for Mobile Relay Control in mmWave 6G-IoT Networks
Batool, I., Fouda, M. M., & Fadlullah, Z. M. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3672032 · Cited by 4
A Brief Review of Recent Hierarchical Reinforcement Learning for Robotic Manipulation
Liu, S., Tang, T., Zhang, W., Cui, J., & Xu, X. (2022) · International Conference on Information Systems and Computing Technology (ISCTech) · doi:10.1109/isctech58360.2022.00119 · Cited by 6
Personalised Multi-modal Interactive Recommendation with Hierarchical State Representations
Wu, Y., Macdonald, C., & Ounis, I. (2024) · ACM Transactions on Recommender Systems · doi:10.1145/3651169 · Cited by 6
He, S., Jin, B., Tian, S., Liu, J., Deng, Z., & Shi, C. L. (2024) · Electronics · doi:10.3390/electronics13245028 · Cited by 4
Continuous-time hierarchical reinforcement learning for satellite pursuit decision
Wei, L.-S., Ning, X., Lian, X.-B., Wang, F., Zhang, G.-P., & Lin, M. (2025) · Chinese Journal of Aeronautics · doi:10.1016/j.cja.2025.103662 · Cited by 2
Parnada, A., Qu, M., Castellani, M., Chang, H. J., & Wang, Y. (2025) · Proceedings of the Institution of Mechanical Engineers Part I Journal of Systems and Control Engineering · doi:10.1177/09596518251350353 · Cited by 8
Xia, C., Shi, X., Xie, H. L., Lu, Y., Li, P., & Shang, M. (2025) · ACM Transactions on the Web · doi:10.1145/3769471 · Cited by 4
Xiao, W., Luo, Q., Yu, T., Wu, Y., Huang, Z., & Pan, Z. (2026) · Applied Energy · doi:10.1016/j.apenergy.2026.128001 · Cited by 3
Du, Y., Prébot, B., Malloy, T., & González, C. (2025) · ACM Transactions on Social Computing · doi:10.1145/3712672 · Cited by 3
ROTR: Role-Transformable Multi-Agent Resource Allocation for Nonstationary Vehicular Communications
Li, Y., Yuan, Q., Fu, X., Luo, G., Kang, J., Li, J., & Niyato, D. T. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3567652 · Cited by 3
Uncertainty-Driven Distributional Reinforcement Learning for Flight Control
Homola, M., Li, Y., & Kampen, E.-J. V. (2025) · AIAA SciTech Forum · doi:10.2514/6.2025-2793 · Cited by 4
Pan, W., Chen, M., Lin, B., Wang, Y., Zhao, X., He, X., & Ye, J. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3609173 · Cited by 2
Hu, J., Wang, T., Wang, H., & Cao, J. (2025) · Drones · doi:10.3390/drones9120813 · Cited by 2
Wang, N., Lin, Y., Miu, S., Chen, W., Yang, Y., & Huang, H. (2025) · Industrial Robot the international journal of robotics research and application · doi:10.1108/ir-03-2025-0105 · Cited by 2
Calzolari, G., Sumathy, V., Kanellakis, C., & Nikolakopoulos, G. (2026) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57385.2026.11696319 · Cited by 2
Kim, B.-S., An, J., & Sim, M. K. (2023) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2023.101177 · Cited by 2
Hierarchical Reinforcement Learning: Structuring Decision-Making for Complex AI Tasks
Kosaraju, D. (2024) · International Journal of Science and Healthcare Research · doi:10.52403/ijshr.20220468 · Cited by 2
Leveraging Symbolic Models in Reinforcement Learning for Multi-skill Chaining*
Lu, W., Diehl, M., Sjoberg, J. S., & Ramírez-Amaro, K. (2025) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii59315.2025.10870988 · Cited by 3
Real-Time Power Dispatch in Grid-Connected Microgrids using Deep Q-Learning
Cambambi, C. A. C., Yang, R. L., Milani, I. P., Rangel, C. A. S., Canha, L. N., & Wolter, M. (2024) · International Universities Power Engineering Conference (UPEC) · doi:10.1109/upec61344.2024.10892513 · Cited by 2
Ryu, H., Sohn, C.-B., & Kim, D. (2025) · Applied Sciences · doi:10.3390/app15168956 · Cited by 2
Leveraging More of Biology in Evolutionary Reinforcement Learning
Gašperov, B., Ðurasević, M., & Jakobović, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56855-8_6 · Cited by 3
Learning to Escape: Multi-mode Policy Learning for the Traveling Salesmen Problem
Ha, M. H., Chi, S., & Lee, S. W. (2024) · IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS) · doi:10.1109/eais58494.2024.10569999 · Cited by 1
Sample Efficient Hierarchical Reinforcement Learning for the Game of Othello
Chang, T., Neshatian, K., & Atlas, J. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-97-5035-1_32 · Cited by 1
Elguea-Aguinaco, Í., Aguirre-Ortuzar, A., Izagirre, U., Inziarte-Hidalgo, I., Bøgh, S., & Arana-Arexolaleiba, N. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112870 · Cited by 1
Task-oriented adaptive learning of robot manipulation skills
Jin, K., Tian, G., Huang, B., Cui, Y., & Zheng, X. (2025) · Artificial Life and Robotics · doi:10.1007/s10015-025-01036-y · Cited by 1
Leveraging Sequentiality in Reinforcement Learning from a Single Demonstration
Chenu, A., Serris, O., Sigaud, O., & Perrin, N. (2025) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids65713.2025.11203179 · Cited by 2
A Hierarchical Reinforcement Learning Method in Multi-UAV Target-Attacker-Defender Games
Li, X., Zhou, X., Yang, Y., Yang, X., & Li, Z. (2025) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon58223.2025.11221157 · Cited by 1
Progressive Policy Learning: A Hierarchical Framework for Dexterous Bimanual Manipulation
Lee, K.-W., Lee, J. M., Kim, S.-Y., & Lim, S. (2025) · Mathematics · doi:10.3390/math13223585 · Cited by 1
Cell Configuration for QoE-Aware Volumetric Video Streaming via Hierarchical Reinforcement Learning
Wu, J., Zhang, J., Wang, X., Wang, D., & Jiang, J. B. (2025) · ACM Transactions on Sensor Networks · doi:10.1145/3786787 · Cited by 1
Orhadahwe, T. A., Adeleke, A. A., Boyede, I. D., Oladunni, G. O., & Oladunni, A. A. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-40447-4.00013-x · Cited by 1
Three-stage medical few-shot classification based on adaptive regularization with HMCE loss
Chen, Y., Mao, S., & Yang, Y. (2026) · Applied Intelligence · doi:10.1007/s10489-026-07125-0 · Cited by 1
A Comprehensive Review of Reinforcement Learning
Ghimire, R., & Karna, D. K. L. (2025) · Mid-West University Journal of Engineering & Innovation · doi:10.3126/mujoei.v1i1.91094 · Cited by 1
Context-Aware Artificial Intelligence Systems for Long-Horizon Decision Making
Joshi, V. C., Pillai, S., Gupta, D., Khumora, S., Gulumovna, Y. G., & Ashirova, A. I. (2025) · IEEE International Conference on Emerging Trends in Computing and Communication (ETCOM) · doi:10.1109/etcom66606.2025.11437335 · Cited by 1
Structural entropy guided hierarchical symmetric multi-agent reinforcement learning
Tian, Y., Yu, X., Qi, Y., Wang, L., Feng, P., Wu, W., Shi, R., & Luo, J. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132289 · Cited by 1
Hu, S., & Horii, T. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3742285 · Cited by 1
Zhou, Z., Shang, J., & Li, Y. (2023) · Electronics · doi:10.3390/electronics12173700 · Cited by 1
A Survey on the Development of Human-Centered Artificial Agents for Fighting Games
Tavares, M., de Carvalho, J. M., & Pinho, A. J. (2025) · IEEE Transactions on Games · doi:10.1109/tg.2025.3577552 · Cited by 1
GPI-tree search: algorithms for decision-time planning with the general policy improvement theorem
Bagot, L., D’eer, L., Latré, S., Schepper, T. D., & Mets, K. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11304-4 · Cited by 1
Self-citationAdaptive Successor Features Composition for Transfer Reinforcement Learning
Garcés, K., Xuan, J., & Zuo, H. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3610392 · Cited by 1
Dynamic Legged Ball Manipulation on Rugged Terrains With Hierarchical Reinforcement Learning
Zhu, D., Yang, Z., Li, X., Xu, W., Liu, Q., & Li, X. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3682979 · Cited by 2
An artificially intelligent space-filling trajectory planning for wire arc additive manufacturing
Ferreira, R. (2023) · doi:10.14393/ufu.te.2023.431 · Cited by 1
Leveraging Sequentiality in Reinforcement Learning from a Single DemonstrationPreprint
Chenu, A., Serris, O., Sigaud, O., & Perrin, N. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4627349 · Cited by 1
Online Self-Training for Co-Adaptation in Hierarchical Diffusion PoliciesPreprint
Grislain, C., Kappel, M., Sigaud, O., & Chétouani, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.05291 · Cited by 1
Neuro-symbolic deep reinforcement learning for safe urban driving using low-cost sensors.
Albilani, M. (2024) · theses.fr (ABES) · doi:10.70675/ea6f5d11z7d78z4484zb809z38299a88535e · Cited by 3
Leveraging Sequentiality in Reinforcement Learning from a Single Demonstration
Chenu, A., Serris, O., Sigaud, O., & Perrin-Gilbert, N. (2022) · arXiv · doi:10.48550/arxiv.2211.04786
“S seq : At step t , an extended state is described by a ( s t , i t ) pair where i t is the index of the current goal.”
The Explainable Model to Multi-Objective Reinforcement Learning Toward an Autonomous Smart System
Yamaguchi, T. (2023) · Advances in systems analysis, software engineering, and high performance computing book series · doi:10.4018/978-1-6684-7684-0.ch002
Learning Hierarchical Planning-Based Policies from Offline Data
Wöhlke, J., Schmitt, F., & van Hoof, H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43421-1_29
Past Data-Driven Adaptation in Hierarchical Reinforcement Learning
Sijie, Z., Chen, A., Zhou, X., & Wang, T. (2024) · International Conference on Machine Learning and Computing (ICMLC) · doi:10.1145/3651671.3651714
Offline Hierarchical Reinforcement Learning: Enable Large-Scale Training in HRL
Wu, Y., Zhang, H., & Wang, Y.-X. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-3336-1_34
Using experience classification for training non-Markovian tasks
Miao, R., Lu, X., Tian, C., Yu, B., Cui, J. R., & Duan, Z. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124649 · Cited by 1
Metadata
- Authors
- Hutsebaut-Buysse, M., Mets, K., & Latré, S.
- Affiliations
- University of Antwerp (BE)
- Corresponding author
- Hutsebaut-Buysse, Matthias
- Venue
- Machine Learning and Knowledge Extraction
- Publisher
- MDPI AG
- Published
- 2022-02-17
- Volume
- 4
- Issue
- 1
- Pages
- 172-221
- ISSN
- 2504-4990 (ISSN-L)
- OpenAlex
- W4212996499
- Cited by
- 129
- References
- 208
- Citations by year
- 2026: 30 · 2025: 45 · 2024: 29 · 2023: 19 · 2022: 6
- FWCI
- 8.15
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Machine Learning and Knowledge Extraction (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- University of Antwerp (x)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Keywords
- hierarchical reinforcement learning, abstraction, temporal abstraction, state abstraction, options framework, goal-conditioned policies, sample efficiency, exploration, interpretability, high-dimensional environments, sequential decision making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.