Cited by 320Reinforcement Learning in RoboticsAI-based Problem Solving and PlanningArtificial Intelligence in Games

The DOI matches its Crossref record; 63 of the 64 references it cites were found in a registry or an index.

Recent Advances in Hierarchical Reinforcement Learning

Barto, Andrew G., Mahadevan, Sridhar·2003·Discrete Event Dynamic Systems

DOI 10.1023/a:1025696116075

Open this paper in CoChat — free account

Integrity signals

Registry

Crossref · 10.1023/a:1025696116075

Cites

63 of 63 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Statecharts: a visual formalism for complex systems
Harel, D. (1987) · Science of Computer Programming · doi:10.1016/0167-6423(87)90035-9 · Cited by 6,771
Markov Decision Processes
Puterman, M. L. (1994) · Wiley series in probability and statistics · doi:10.1002/9780470316887 · Cited by 6,057
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149
Multiagent Systems : A Modern Approach to Distributed Artificial Intelligence
Weiß, G. (2000) · Medical Entomology and Zoology · Cited by 3,458
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119
Singular Perturbation Methods in Control: Analysis and Design
Kokotović, P. V., Khalil, H. K., & O’Reilly, J. (1999) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611971118 · Cited by 2,504
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition
Dietterich, T. G. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.639 · Cited by 1,526
Toward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450
Self-citation
Dynamic Programming: Deterministic and Stochastic Models
Bertsekas, D. P. (1987) · Medical Entomology and Zoology · Cited by 1,348

Show all 64 references

Research works printed but not linked: 27

Bertsekas, D. P., and Tsitsiklis, J. N. 1996. Neuro-Dynamic Programming. Belmont, MA: Athena Scientific.
Bradtke, S. J., and Duff. M. O. 1995. Reinforcement learning methods for continuous-time markov decision problems. In G. Tesauro. D. S. Touretzky and T. Leen (eds.). Advances in Neural Information Processing Systems: Proceedings of the 1994 Conference. Cambridge. MA: MIT Press. pp. 393-400.
Crites. R. H. 1996. Large-Scale Dvnaniic Optimization Using Teams of Reinforcement Learning Agents. Ph.D. thesis. Amherst. MA: University of Massachusetts.
Digney, B. 1998. Learning hierarchical control structure from multiple tasks and changing environments. In From Animals to Animals 5: The Fifth Conference on Simulation of Adaptive Behavior. Carnbndge, MA: MIT Press.
Grudic. G. Z., and Ungar. L. H. 2000. Localizing search in reinforcement learning. In Proceedings of the 18th National Conference on Artificial Intelligence, (AAAI-00), pp. 590-595.
Hernandez, N., and Mahadevan. S. 2001. Hierarchical memory-based reinforcement learning. In Advances in Neural information Processing Systems: Proceedings of the 2000 conference. Cambridge, MA: MIT Press, pp. 1047-1053.
Howard, R. A. 1971. Dynamic Probabilistic Systems: Semi-Markar and Decision Processes. New York: Wiley.
Jonsson. A., and Barto. A. G. 2001. Automated state abstraction for options using the li-tree algorithm. In Advances in Neural Information Processing Systems: Proceedings of the 2000 Conference. Cambridge, MA: MIT Press, pp. 1054-1060.
Littman. M. 1994. Markov games as a framework for multi-agent reinforcement learning. In Proceedings of the Eleventh International Conference on Machine Learning. San Francisco, CA: Morgan Kaufmann, pp. 157-163.
McCalIum, A. K. 1996. Reinforcement Learning with Selective Perceptioin and Hidden State. Ph.D. thesis, University of Rochester.
Nourbakhsh, I., Powers, R., and Birchfield, S. 1995. Dervish: An office-navigation robot. Al Magazine 16(2): 53-60.
Precup, D. 2000. Temporal Abstraction in Reinforcentent Learning. Ph.D. thesis, Amherst, MA: University of Massachusetts.
Precup. D., and Sutton, R. S. 1998. Multi-time models for temporally abstract planning. In Advances in Neural information Processing Systems: Proceedings of the 1997 Conference. Cambridge MA: MIT Press, pp. 1050-1056.
Ross. S. 1983. Introduction to Stochastic Dynamic Programming. New York: Academic Press.
Rummery, G. A., and Niranjan, M. 1994. On-line q-learrnng using connectionist systems. Technical Report CUED/F-INFENG/'FR 166, Cambridge University Engineering Department.
Samuel. A. L. 1963. Some studies in machine learning using the game of checkers. IBM Journal on Research and Developntent 3: 211-229, 1959. Reprinted in E. A. Feigenbaurn and J. Feldman (eds.), Computers and Thought. New York: McGraw-Hill, pp. pp71-105.
Shatkay, H., and Kaelbling. L. P. 1997. Learning topological maps with weak local odometric information. In IJC4I (2). 920-929.
Singh. S., and Bertsekas, D. 1997. Reinforcement learning for dynamic channel allocation in cellular telephone systems. In Advances in Neural Information Processing Systems: Proceedings of the 1996 conference. Cambridge, MA: MIT Press, pp. 974-980.
Sutton, R. S. 1996. Generalization in reinforcement leaming: Successful examples using sparse coarse coding. In D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo (eds.). Advances in Neural Information Processing Svstenis: Proceedings of the 1995 Conference. Cambridge. MA: MIT Press. pp. 1038-1044.
Sutton. R. S., and Barto, A. G. 1998. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.
Tan, M. 1993. Multi-agent reinforcement leaming: Independent vs. cooperative agents. In Proceedings of the Tenth International Conference on Mac/tine Learning. San Francisco, CA: Morgan Kaufmann, pp. 330-337.
Theocharous, G. 2002. Hierarchical Learning and Planning in Partially Observable Markov Derision Processes. Ph.D. Thesis, Michigan State University.
Theocharous, G., and Mahadevan. S. 2002. Approximate planning with hierarchical partially observable markov decision processs for robot navigation. In Proceedings of the IEEE International Conference on Robotics and Autontation (ICRA).
Theocharous, G., Rohanimanesh. K., and Mahadevan, S. 2001. Learning hierarchical partially observable markov decision processs for robot navigation. In Proceedings of the IEEE International Conference o,i Robotics and Automation (ICRA).
Watkins, C. J. C. H. 1989. Learning front Delayed Rewards. Ph.D. thesis, Cambridge, England: Cambridge University.
Watkins, C. J. C. H., and Dayan, P. 1992. Q-learning. Machine Learning 8: 279-292.
Werbos, P. J. 1977. Advanced forecasting methods for global crisis warning and models of intelligence. General Systems Yearbook 22: 25-38.

Sorted by citations

Items 101–200 of 320

AUTOMATIC COMPLEXITY REDUCTION IN REINFORCEMENT LEARNING
Chiu, C., & Soo, V. (2010) · Computational Intelligence · doi:10.1111/j.1467-8640.2009.00350.x · Cited by 23
Intrinsic motivation and mental replay enable efficient online adaptation in stochastic recurrent networks
Tanneberg, D., Peters, J., & Rueckert, E. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.10.005 · Cited by 25
Anticipations, Brains, Individual and Social Behavior: An Introduction to Anticipatory Systems
Butz, M. V., Sigaud, O., Pezzulo, G., & Baldassarre, G. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-74262-3_1 · Cited by 24
Robot Learning
Peters, J., Lee, D. D., Kober, J., Nguyen-Tuong, D., Bagnell, J. A., & Schaal, S. (2016) · Springer handbooks · doi:10.1007/978-3-319-32552-1_15 · Cited by 27
Multi-UAV air combat cooperative game based on virtual opponent and value attention decomposition policy gradient
Xu, X., Wang, Y., Guo, X., Huang, K., & Zhang, X. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126069 · Cited by 22
PANOLA: A Personal Assistant for Supporting Users in Preserving Privacy
Ulusoy, O., & Yolum, P. (2021) · ACM Transactions on Internet Technology · doi:10.1145/3471187 · Cited by 22
Incremental Skill Acquisition for Self-motivated Learning Animats
Bonarini, A., Lazaric, A., & Restelli, M. (2006) · Lecture notes in computer science · doi:10.1007/11840541_30 · Cited by 22
Extending BDI plan selection to incorporate learning from experience
Singh, D., Sardiña, S., & Padgham, L. (2010) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2010.05.008 · Cited by 22
Towards Learning of Generic Skills for Robotic Manipulation
Metzen, J. H., Fabisch, A., Senger, L., Fernández, J. D. G., & Kirchner, E. A. (2013) · Künstliche Intell. · doi:10.1007/s13218-013-0280-1 · Cited by 21
Intrinsic motivation mechanisms for competence acquisition
Santucci, V. G., Baldassarre, G., & Mirolli, M. (2012) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2012.6400835 · Cited by 21
Improving reinforcement learning by using sequence trees
Girgin, S., Polat, F., & Alhajj, R. (2010) · Machine Learning · doi:10.1007/s10994-010-5182-y · Cited by 21
Abstraction from demonstration for efficient reinforcement learning in high-dimensional domains
Cobo, L. C., Subramanian, K., Isbell, C. L., Lanterman, A. D., & Thomaz, A. L. (2014) · Artificial Intelligence · doi:10.1016/j.artint.2014.07.003 · Cited by 19
Systems view on spatial planning and perception based on invariants in agent-environment dynamics
Mettler, B., Kong, Z., Li, B., & Andersh, J. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00439 · Cited by 22
Deep reinforcement learning algorithms for steering an underactuated ship
Le, T. P., Layek, M. A., Vien, N. A., & Chung, T. (2017) · IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI) · doi:10.1109/mfi.2017.8170388 · Cited by 19
Hierarchical Behaviours: Getting the Most Bang for Your Bit
van Dijk, S. G., Polani, D., & Nehaniv, C. L. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21314-4_43 · Cited by 19
Aggregating E-commerce Search Results from Heterogeneous Sources via Hierarchical Reinforcement Learning
Takanobu, R., Zhuang, T., Huang, M., Feng, J., Tang, H., & Zheng, B. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313455 · Cited by 17
Dynamic Real-time Multimodal Routing with Hierarchical Hybrid Planning
Choudhury, S., Knickerbocker, J. P., & Kochenderfer, M. J. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8814018 · Cited by 18
Behavioral Hierarchy: Exploration and Representation
Barto, A. G., Konidaris, G., & Vigorito, C. M. (2013) · Springer eBooks · doi:10.1007/978-3-642-39875-9_2 · Cited by 18
Self-citation
Hierarchical control over foraging behavior by anterior cingulate cortex
Alejandro, R. J., & Holroyd, C. B. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105623 · Cited by 18
Offline reinforcement learning with task hierarchies
Schwab, D., & Ray, S. (2017) · Machine Learning · doi:10.1007/s10994-017-5650-8 · Cited by 18
Learning and Generalization of Dynamic Movement Primitives by Hierarchical Deep Reinforcement Learning from Demonstration
Kim, W., Lee, C., & Kim, H. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594476 · Cited by 18
Psychological and Neuroscientific Connections with Reinforcement Learning
Shah, A. (2012) · Adaptation, learning, and optimization · doi:10.1007/978-3-642-27645-3_16 · Cited by 17
Automatic Skill Acquisition in Reinforcement Learning Agents Using Connection Bridge Centrality
Moradi, P., Shiri, M. E., & Entezari, N. (2010) · Communications in computer and information science · doi:10.1007/978-3-642-17604-3_6 · Cited by 17
Computational Properties of the Hippocampus Increase the Efficiency of Goal-Directed Foraging through Hierarchical Reinforcement Learning
Chalmers, E., Luczak, A., & Gruber, A. J. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00128 · Cited by 18
Lessons for Robotics From the Control Architecture of the Octopus
Sivitilli, D. M., Smith, J. R., & Gire, D. H. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.862391 · Cited by 13
Autonomous Identification and Goal-Directed Invocation of Event-Predictive Behavioral Primitives
Gumbsch, C., Butz, M. V., & Martius, G. (2019) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2019.2925890 · Cited by 15
Learning arbitrary stimulus-reward associations for naturalistic stimuli involves transition from learning about features to learning about objects
Farashahi, S. G., Xu, J. Y., Wu, S., & Soltani, A. (2020) · Cognition · doi:10.1016/j.cognition.2020.104425 · Cited by 16
A layered approach to learning coordination knowledge in multiagent environments
Erus, G., & Polat, F. (2007) · Applied Intelligence · doi:10.1007/s10489-006-0034-y · Cited by 16
Interplay of Rhythmic and Discrete Manipulation Movements During Development: A Policy-Search Reinforcement-Learning Robot Model
Meola, V. C., Caligiore, D., Sperati, V., Zollo, L., Ciancio, A. L., Taffoni, F., Guglielmelli, E., & Baldassarre, G. (2015) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tamd.2015.2494460 · Cited by 15
Strategies for Scaleable Communication and Coordination in Multi-Agent (UAV) Systems
Ponniah, J., & Dantsker, O. D. (2022) · Aerospace · doi:10.3390/aerospace9090488 · Cited by 17
Exploration from Generalization Mediated by Multiple Controllers
Dayan, P. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_4 · Cited by 15
Hierarchical reinforcement learning and central pattern generators for modeling the development of rhythmic manipulation skills
Ciancio, A. L., Zollo, L., Guglielmelli, E., Caligiore, D., & Baldassarre, G. (2011) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2011.6037370 · Cited by 15
Exploiting Generalization in the Subspaces for Faster Model-Based Reinforcement Learning
Hashemzadeh, M., Hosseini, R., & Ahmadabadi, M. N. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2869978 · Cited by 15
A linear-complexity reparameterisation strategy for the hierarchical bootstrapping of capabilities within perception–action architectures
Shevchenko, M., Windridge, D., & Kittler, J. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.12.002 · Cited by 15
Learning to Interrupt: A Hierarchical Deep Reinforcement Learning Framework for Efficient Exploration
Li, T., Pan, J., Zhu, D., & Meng, M. Q. ‐. (2018) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio.2018.8665177 · Cited by 15
Optimizing energy management of smart grid using reinforcement learning aided by surrogate models built using physics-informed neural networks
Cestero, J., Femine, C. D., Muro, K. S., Quartulli, M., & Restelli, M. (2025) · Applied Energy · doi:10.1016/j.apenergy.2025.126750 · Cited by 14
Optimal Trade Execution Based on Deep Deterministic Policy Gradient
Ye, Z., Deng, W., Zhou, S., Xu, Y., & Guan, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59410-7_42 · Cited by 14
Arithmetic value representation for hierarchical behavior composition
Makino, H. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01211-5 · Cited by 14
Deciding Which Skill to Learn When: Temporal-Difference Competence-Based Intrinsic Motivation (TD-CB-IM)
Baldassarre, G., & Mirolli, M. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_11 · Cited by 14
Hierarchical control using networks trained with higher-level forward models
Wayne, G., & Abbott, L. F. (2014) · Neural Computation · doi:10.1162/neco_a_00639 · Cited by 14
Connect-Based Subgoal Discovery for Options in Hierarchical Reinforcement Learning
Chen, F., Gao, Y., Chen, S., & Ma, Z. (2007) · International Conference on Natural Computation (ICNC) · doi:10.1109/icnc.2007.312 · Cited by 14
Femtocaching in video content delivery: Assignment of video clips to serve dynamic mobile users
Yue, J., Yang, B., Chen, C., Guan, X., & Zhang, W. (2014) · Computer Communications · doi:10.1016/j.comcom.2014.05.010 · Cited by 13
Deep reinforcement learning boosted by external knowledge
Bougie, N., & Ichise, R. (2018) · ACM Symposium on Applied Computing · doi:10.1145/3167132.3167165 · Cited by 13
Positive Impact of State Similarity on Reinforcement Learning Performance
Girgin, S., Polat, F., & Alhajj, R. (2007) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2007.899419 · Cited by 13
Combining deep reinforcement learning with prior knowledge and reasoning
Bougie, N., Cheng, L. K., & Ichise, R. (2018) · ACM SIGAPP Applied Computing Review · doi:10.1145/3243064.3243067 · Cited by 14
A Model of Reaching that Integrates Reinforcement Learning and Population Encoding of Postures
Ognibene, D., Rega, A., & Baldassarre, G. (2006) · Lecture notes in computer science · doi:10.1007/11840541_32 · Cited by 12
A hierarchical maze navigation algorithm with Reinforcement Learning and mapping
Mannucci, T., & Kampen, E.-J. V. (2016) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2016.7849365 · Cited by 12
Learning-agent-based simulation for queue network systems
Fuller, D. B., Arruda, E. F., & Filho, V. J. M. F. (2019) · Journal of the Operational Research Society · doi:10.1080/01605682.2019.1633232 · Cited by 12
Toward Generalization of Automated Temporal Abstraction to Partially Observable Reinforcement Learning
Çіlden, E., & Polat, F. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2352038 · Cited by 12
Learning Contextual Reward Expectations for Value Adaptation
Rigoli, F., Chew, B., Dayan, P., & Dolan, R. J. (2017) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01191 · Cited by 12
Shape grammar parsing : application to image-based modeling
Teboul, O. (2011) · doi:10.70675/36465d8bz8269z49e7z8f85zedde7e6d6011 · Cited by 12
Layered direct policy search for learning hierarchical skills
End, F., Akrour, R., Peters, J., & Neumann, G. M. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989761 · Cited by 11
Automatic Discovery and Transfer of Task Hierarchies in Reinforcement Learning
Mehta, N., Ray, S., Tadepalli, P. V., & Dietterich, T. G. (2011) · AI Magazine · doi:10.1609/aimag.v32i1.2342 · Cited by 11
Hierarchical Reinforcement Learning Approach for the Road Intersection Task
Shikunov, M., & Panov, A. I. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-25719-4_64 · Cited by 11
Model-Based Reinforcement Learning for Partially Observable Games with Sampling-Based State Estimation
Fujita, H., & Ishii, S. (2007) · Neural Computation · doi:10.1162/neco.2007.19.11.3051 · Cited by 11
Learning Graph-Based Representations for Continuous Reinforcement Learning Domains
Metzen, J. H. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40988-2_6 · Cited by 11
Hierarchical Reinforcement Learning, Sequential Behavior, and the Dorsal Frontostriatal System
Janssen, M., LeWarne, C., Burk, D. C., & Averbeck, B. B. (2022) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01869 · Cited by 10
Automatic Construction of Temporally Extended Actions for MDPs Using Bisimulation Metrics
Castro, P. S., & Precup, D. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-29946-9_16 · Cited by 10
GPR modelling and inversion to quantify the debris content within ice
Santin, I., Roncoroni, G., Forte, E., Gutgesell, P., & Pipan, M. (2023) · Near Surface Geophysics · doi:10.1002/nsg.12274 · Cited by 9
Needs, Pains, and Motivations in Autonomous Agents
Starzyk, J. A., Graham, J., & Puzio, L. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2596787 · Cited by 9
Actor-Critic Learning for Platform-Independent Robot Navigation
Muse, D., & Wermter, S. (2009) · Cognitive Computation · doi:10.1007/s12559-009-9021-z · Cited by 9
A Dual Process Account of Coarticulation in Motor Skill Acquisition
Shah, A., Barto, A. G., & Fagg, A. H. (2013) · Journal of Motor Behavior · doi:10.1080/00222895.2013.837423 · Cited by 9
Self-citation
Approximate planning for bayesian hierarchical reinforcement learning
Vien, N. A., Ngo, H., Lee, S., & Chung, T. (2014) · Applied Intelligence · doi:10.1007/s10489-014-0565-6 · Cited by 9
Multilevel Constrained Bandits: A Hierarchical Upper Confidence Bound Approach with Safety Guarantees
Baheri, A. (2025) · Mathematics · doi:10.3390/math13010149 · Cited by 10
HPRS: hierarchical potential-based reward shaping from task specifications
Berducci, L., Aguilar, E. A., Ničković, D., & Grosu, R. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1444188 · Cited by 18
Dynamic scheduling for cloud manufacturing with uncertain events by hierarchical reinforcement learning and attention mechanism
Zhang, J., Jiang, Y., Guo, B., Liu, T., Hu, D., Zhang, J., Deng, Y., Wang, H., Yang, J., & Ding, X. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113335 · Cited by 8
An Adaptive Solution for Web Service Composition
Wang, H., & Guo, X. (2010) · IEEE World Congress on Services (SERVICES) · doi:10.1109/services.2010.20 · Cited by 8
A generic adaptive simulation algorithm for component-based simulation systems
Helms, T., Ewald, R., Rybacki, S., & Uhrmacher, A. M. (2013) · ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS) · doi:10.1145/2486092.2486095 · Cited by 8
Deep reinforcement learning‐based ordering mechanism for performance optimization in multi‐echelon supply chains
Kurian, D. S., Pillai, V. M., Raut, A., & Gautham, J. (2022) · Applied Stochastic Models in Business and Industry · doi:10.1002/asmb.2723 · Cited by 8
Strength Adjustment and Assessment for MCTS-Based Programs [Research Frontier]
Liu, A.-J., Wu, T.-R., Wu, I., Guei, H., & Wei, T.-H. (2020) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2020.2998315 · Cited by 8
Object-Oriented Representation and Hierarchical Reinforcement Learning in Infinite Mario
Joshi, M., Khobragade, R., Sarda, S., Deshpande, U. A., & Mohan, S. (2012) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai.2012.152 · Cited by 8
Adopting animal concepts in hierarchical reinforcement learning and control of intelligent agents
Kadleček, D., & Nahodil, P. (2008) · IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) · doi:10.1109/biorob.2008.4762882 · Cited by 8
Extracting bottlenecks for reinforcement learning agent by holonic concept clustering and attentional functions
Ghazanfari, B., & Mozayani, N. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.01.030 · Cited by 8
Sequential Association Rule Mining for Autonomously Extracting Hierarchical Task Structures in Reinforcement Learning
Ghazanfari, B., Afghah, F., & Taylor, M. E. (2020) · IEEE Access · doi:10.1109/access.2020.2965930 · Cited by 8
Imitation learning with hierarchical actions
Friesen, A. L., & Rao, R. P. N. (2010) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2010.5578832 · Cited by 8
Memristive Bellman solver for decision-making
Feng, Z., Wu, Z., Zou, J., Cheng, L., Zhao, X., Zhang, X., Lü, J., Wang, C., Wang, Y., Wang, H., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-60085-w · Cited by 9
Real-Time QoS Control for Service Orchestration
Bosman, J., van den Berg, H., & van der Mei, R. (2015) · International Teletraffic Congress · doi:10.1109/itc.2015.25 · Cited by 7
A mobile robot controller using reinforcement learning under scLTL specifications with uncertainties
Mi, J., Kuze, N., & Ushio, T. (2021) · Asian Journal of Control · doi:10.1002/asjc.2712 · Cited by 7
Hierarchical DDPG for Manipulator Motion Planning in Dynamic Environments
Um, D., Nethala, P., & Shin, H. (2022) · AI · doi:10.3390/ai3030037 · Cited by 7
REAL-X—Robot Open-Ended Autonomous Learning Architecture: Building Truly End-to-End Sensorimotor Autonomous Learning Systems
Cartoni, E., Montella, D., Triesch, J., & Baldassarre, G. (2023) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2023.3270081 · Cited by 7
Computational models of motivated frontal function
O’Reilly, R. C., Russin, J., & Herd, S. A. (2019) · Handbook of clinical neurology · doi:10.1016/b978-0-12-804281-6.00017-3 · Cited by 7
The explicit form of the rate function for semi-Markov processes and its contractions
Sughiyama, Y., & Kobayashi, T. J. (2018) · Journal of Physics A Mathematical and Theoretical · doi:10.1088/1751-8121/aaab6d · Cited by 7
Lifelong transfer learning with an option hierarchy
Hawasly, M., & Ramamoorthy, S. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696523 · Cited by 7
Reinforcement learning for balancer embedded humanoid locomotion
Yamaguchi, A., Hyon, S.-H., & Ogasawara, T. (2010) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/ichr.2010.5686296 · Cited by 6
Automatic construction and evaluation of macro-actions in reinforcement learning
Farahani, M. D., & Mozayani, N. (2019) · Applied Soft Computing · doi:10.1016/j.asoc.2019.105574 · Cited by 6
A Local Graph Clustering Algorithm for Discovering Subgoals in Reinforcement Learning
Entezari, N., Shiri, M. E., & Moradi, P. (2010) · Communications in computer and information science · doi:10.1007/978-3-642-17604-3_5 · Cited by 6
Reinforcement learning algorithms that assimilate and accommodate skills with multiple tasks
Tommasino, P., Caligiore, D., Mirolli, M., & Baldassarre, G. (2012) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2012.6400871 · Cited by 6
Selecting simultaneous actions of different durations to optimally manage an ecological network
Péron, M., Jansen, C. C., Mantyka‐Pringle, C., Nicol, S., Schellhorn, N. A., Becker, K. H., & Chadès, I. (2017) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12744 · Cited by 6
The detour problem in a stochastic environment: Tolman revisited
Fakhari, P., Khodadadi, A., & Busemeyer, J. R. (2017) · Cognitive Psychology · doi:10.1016/j.cogpsych.2017.12.002 · Cited by 6
Hierarchical Reinforcement Learning for Playing a Dynamic Dungeon Crawler Game
Niel, R., & Wiering, M. (2018) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2018.8628914 · Cited by 6
Agents for Preserving Privacy: Learning and Decision Making Collaboratively
Ulusoy, O., & Yolum, P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-66412-1_8 · Cited by 6
Dynamic file grouping for load balancing in streaming media clustered server systems
Jiang, Q., Xi, H., & Yin, B. (2009) · International Journal of Control Automation and Systems · doi:10.1007/s12555-009-0413-3 · Cited by 6
LLM-Based Multi-agent Systems: Frameworks, Evaluation, Open Challenges, and Research Frontiers
Shaikh, S. H. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-15632-7_9 · Cited by 6
A taxonomy of spatial navigation in mammals: Insights from computational modeling
Vijayabaskaran, S., Zeng, X., Ghazinouri, B., Wiskott, L., & Cheng, S. (2025) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2025.106282 · Cited by 6
Phoenix: A Motion-Based Self-Reflection Framework for Fine-Grained Robotic Action Correction
Xia, W., Feng, R., Wang, D., & Hu, D. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00655 · Cited by 5
Humanoid Tactile Gesture Production using a Hierarchical SOM-based Encoding
Pierris, G., & Dahl, T. S. (2014) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2014.2313615 · Cited by 5
Toward Self-Referential Autonomous Learning of Object and Situation Models
Damerow, F., Knoblauch, A., Körner, U., Eggert, J. P., & Körner, E. (2016) · Cognitive Computation · doi:10.1007/s12559-016-9407-7 · Cited by 5
Extending the “Open-Closed Principle” to Automated Algorithm Configuration
Swan, J., Adriænsen, S., Barwell, A. D., Hammond, K., & White, D. R. (2018) · Evolutionary Computation · doi:10.1162/evco_a_00245 · Cited by 5
A Hierarchical Reinforcement Learning Framework for UAV Path Planning in Tactical Environments
Alpdemir, M. N. (2023) · Turkish Journal of Science and Technology · doi:10.55525/tjst.1219845 · Cited by 7
A robot to study the development of artwork appreciation through social interactions
Karaouzene, A., Gaussier, P., & Vidal, D. (2013) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2013.6652554 · Cited by 5

Authors: Barto, A. G., & Mahadevan, S.
Affiliations: University of Massachusetts Amherst (US)
ORCID: Barto, Andrew G.
Mahadevan, Sridhar
Venue: Discrete Event Dynamic Systems
Publisher: Springer Science and Business Media LLC
Published: 2003-10
Volume: 13
Issue: 4
Pages: 341-379
ISSN: 0924-6703 (ISSN-L)
1573-7594
DOI: 10.1023/a:1025696116075
OpenAlex: W4249441547
Cited by: 317
References: 64
Citations by year: 2026: 29 · 2025: 44 · 2024: 8 · 2023: 4 · 2022: 10 · 2021: 14 · 2020: 13 · 2019: 20 · 2018: 21 · 2017: 17 · 2016: 11 · 2015: 12 · 2014: 18 · 2013: 20 · 2012: 23
FWCI: 2.49
Type: journal article
Language: en
Open access: closed
Licence: www.springernature.com/gp/researchers/text-and-data-mining (tdm)
www.springernature.com/gp/researchers/text-and-data-mining (vor)
Locations: Discrete Event Dynamic Systems (publishedVersion)
Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
Keywords: hierarchical reinforcement learning, temporal abstraction, semi-Markov decision process, curse of dimensionality, hierarchical control architecture, concurrent actions, multi-agent coordination, hierarchical memory, partial observability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Recent Advances in Hierarchical Reinforcement Learning

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks