The DOI matches its Crossref record; 63 of the 64 references it cites were found in a registry or an index.
Recent Advances in Hierarchical Reinforcement Learning
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Hierarchical control has been suggested as a general framework for the control of complex systems (Barto and Mahadevan, 2003), and the octopus is an exemplar of this form of control (Zullo et al., 2009; Wayne and Abbott, 2014).”
From Lessons for Robotics From the Control Architecture of the Octopus (2022) · cites this paper as Barto and Mahadevan, 2003
Citations
Cites
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Statecharts: a visual formalism for complex systems
Harel, D. (1987) · Science of Computer Programming · doi:10.1016/0167-6423(87)90035-9 · Cited by 6,771
Puterman, M. L. (1994) · Wiley series in probability and statistics · doi:10.1002/9780470316887 · Cited by 6,057
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149
Multiagent Systems : A Modern Approach to Distributed Artificial Intelligence
Weiß, G. (2000) · Medical Entomology and Zoology · Cited by 3,458
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119
Singular Perturbation Methods in Control: Analysis and Design
Kokotović, P. V., Khalil, H. K., & O’Reilly, J. (1999) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611971118 · Cited by 2,504
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition
Dietterich, T. G. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.639 · Cited by 1,526
Toward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450
Self-citationDynamic Programming: Deterministic and Stochastic Models
Bertsekas, D. P. (1987) · Medical Entomology and Zoology · Cited by 1,348
Research works printed but not linked: 27
- Bertsekas, D. P., and Tsitsiklis, J. N. 1996. Neuro-Dynamic Programming. Belmont, MA: Athena Scientific.
- Bradtke, S. J., and Duff. M. O. 1995. Reinforcement learning methods for continuous-time markov decision problems. In G. Tesauro. D. S. Touretzky and T. Leen (eds.). Advances in Neural Information Processing Systems: Proceedings of the 1994 Conference. Cambridge. MA: MIT Press. pp. 393-400.
- Crites. R. H. 1996. Large-Scale Dvnaniic Optimization Using Teams of Reinforcement Learning Agents. Ph.D. thesis. Amherst. MA: University of Massachusetts.
- Digney, B. 1998. Learning hierarchical control structure from multiple tasks and changing environments. In From Animals to Animals 5: The Fifth Conference on Simulation of Adaptive Behavior. Carnbndge, MA: MIT Press.
- Grudic. G. Z., and Ungar. L. H. 2000. Localizing search in reinforcement learning. In Proceedings of the 18th National Conference on Artificial Intelligence, (AAAI-00), pp. 590-595.
- Hernandez, N., and Mahadevan. S. 2001. Hierarchical memory-based reinforcement learning. In Advances in Neural information Processing Systems: Proceedings of the 2000 conference. Cambridge, MA: MIT Press, pp. 1047-1053.
- Howard, R. A. 1971. Dynamic Probabilistic Systems: Semi-Markar and Decision Processes. New York: Wiley.
- Jonsson. A., and Barto. A. G. 2001. Automated state abstraction for options using the li-tree algorithm. In Advances in Neural Information Processing Systems: Proceedings of the 2000 Conference. Cambridge, MA: MIT Press, pp. 1054-1060.
- Littman. M. 1994. Markov games as a framework for multi-agent reinforcement learning. In Proceedings of the Eleventh International Conference on Machine Learning. San Francisco, CA: Morgan Kaufmann, pp. 157-163.
- McCalIum, A. K. 1996. Reinforcement Learning with Selective Perceptioin and Hidden State. Ph.D. thesis, University of Rochester.
- Nourbakhsh, I., Powers, R., and Birchfield, S. 1995. Dervish: An office-navigation robot. Al Magazine 16(2): 53-60.
- Precup, D. 2000. Temporal Abstraction in Reinforcentent Learning. Ph.D. thesis, Amherst, MA: University of Massachusetts.
- Precup. D., and Sutton, R. S. 1998. Multi-time models for temporally abstract planning. In Advances in Neural information Processing Systems: Proceedings of the 1997 Conference. Cambridge MA: MIT Press, pp. 1050-1056.
- Ross. S. 1983. Introduction to Stochastic Dynamic Programming. New York: Academic Press.
- Rummery, G. A., and Niranjan, M. 1994. On-line q-learrnng using connectionist systems. Technical Report CUED/F-INFENG/'FR 166, Cambridge University Engineering Department.
- Samuel. A. L. 1963. Some studies in machine learning using the game of checkers. IBM Journal on Research and Developntent 3: 211-229, 1959. Reprinted in E. A. Feigenbaurn and J. Feldman (eds.), Computers and Thought. New York: McGraw-Hill, pp. pp71-105.
- Shatkay, H., and Kaelbling. L. P. 1997. Learning topological maps with weak local odometric information. In IJC4I (2). 920-929.
- Singh. S., and Bertsekas, D. 1997. Reinforcement learning for dynamic channel allocation in cellular telephone systems. In Advances in Neural Information Processing Systems: Proceedings of the 1996 conference. Cambridge, MA: MIT Press, pp. 974-980.
- Sutton, R. S. 1996. Generalization in reinforcement leaming: Successful examples using sparse coarse coding. In D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo (eds.). Advances in Neural Information Processing Svstenis: Proceedings of the 1995 Conference. Cambridge. MA: MIT Press. pp. 1038-1044.
- Sutton. R. S., and Barto, A. G. 1998. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.
- Tan, M. 1993. Multi-agent reinforcement leaming: Independent vs. cooperative agents. In Proceedings of the Tenth International Conference on Mac/tine Learning. San Francisco, CA: Morgan Kaufmann, pp. 330-337.
- Theocharous, G. 2002. Hierarchical Learning and Planning in Partially Observable Markov Derision Processes. Ph.D. Thesis, Michigan State University.
- Theocharous, G., and Mahadevan. S. 2002. Approximate planning with hierarchical partially observable markov decision processs for robot navigation. In Proceedings of the IEEE International Conference on Robotics and Autontation (ICRA).
- Theocharous, G., Rohanimanesh. K., and Mahadevan, S. 2001. Learning hierarchical partially observable markov decision processs for robot navigation. In Proceedings of the IEEE International Conference o,i Robotics and Automation (ICRA).
- Watkins, C. J. C. H. 1989. Learning front Delayed Rewards. Ph.D. thesis, Cambridge, England: Cambridge University.
- Watkins, C. J. C. H., and Dayan, P. 1992. Q-learning. Machine Learning 8: 279-292.
- Werbos, P. J. 1977. Advanced forecasting methods for global crisis warning and models of intelligence. General Systems Yearbook 22: 25-38.
Cited by
Sorted by citations
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160
Deep Reinforcement Learning: An Overview
Mousavi, S. S., Schukat, M., & Howley, E. (2017) · Lecture notes in networks and systems · doi:10.1007/978-3-319-56991-8_32 · Cited by 1,045
Reinforcement learning: The Good, The Bad and The Ugly
Dayan, P., & Niv, Y. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.08.003 · Cited by 733
Hierarchical models of behavior and prefrontal function
Botvinick, M. M. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.02.009 · Cited by 532
The Geometry of Abstraction in the Hippocampus and Prefrontal Cortex
Bernardi, S., Benna, M. K., Rigotti, M., Munuera, J., Fusi, S., & Salzman, C. D. (2020) · Cell · doi:10.1016/j.cell.2020.09.031 · Cited by 528
Cushman, F. A. (2013) · Personality and Social Psychology Review · doi:10.1177/1088868313495594 · Cited by 507
Hierarchical Reinforcement Learning
Pateria, S., Subagdja, B., Tan, A., & Quek, C. H. (2021) · ACM Computing Surveys · doi:10.1145/3453160 · Cited by 565
Reinforcement learning for control: Performance, stability, and deep approximators
Buşoniu, L., de Bruin, T., Tolić, D., Kober, J., & Palunko, I. (2018) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2018.09.005 · Cited by 490
Intrinsically Motivated Reinforcement Learning: An Evolutionary Perspective
Singh, S. P., Lewis, R. L., Barto, A. G., & Sorg, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2051031 · Cited by 444
Self-citationMultiobjective Reinforcement Learning: A Comprehensive Overview
Liu, C., Xu, X., & Hu, D. (2015) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2014.2358639 · Cited by 361
Traffic light control using deep policy‐gradient and value‐function‐based reinforcement learning
Mousavi, S. S., Schukat, M., & Howley, E. (2017) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2017.0153 · Cited by 370
Speed/accuracy trade-off between the habitual and the goal-directed processes
Keramati, M., Dezfouli, A., & Piray, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002055 · Cited by 346
Inner Monologue: Embodied Reasoning through Planning with Language Models
Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.05608 · Cited by 287
Deep Learning for Video Game Playing
Justesen, N., Bontrager, P., Togelius, J., & Risi, S. (2019) · IEEE Transactions on Games · doi:10.1109/tg.2019.2896986 · Cited by 304
Hierarchical reinforcement learning and decision making
Botvinick, M. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.05.008 · Cited by 230
Flexible shaping: How learning in small steps helps
Krueger, K. A., & Dayan, P. (2009) · Cognition · doi:10.1016/j.cognition.2008.11.014 · Cited by 233
Intrinsic Motivation and Reinforcement Learning
Barto, A. G. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_2 · Cited by 243
Self-citationDynamics systems vs. optimal control — a unifying view
Schaal, S., Mohajerian, P., & Ijspeert, A. (2007) · Progress in Brain Research · doi:10.1016/s0079-6123(06)65027-9 · Cited by 260
Reinforcement learning for iot security: A comprehensive survey
Uprety, A., & Rawat, D. B. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3040957 · Cited by 214
A Neural Signature of Hierarchical Reinforcement Learning
Fernandes, J. J. F. R., Solway, A., Diuk, C., McGuire, J. T., Barto, A. G., Niv, Y., & Botvinick, M. (2011) · Neuron · doi:10.1016/j.neuron.2011.05.042 · Cited by 207
Self-citationTheory and Simulation in Neuroscience
Wulfram Gerstner, B., Sprekeler, H., & Deco, G. (2012) · Science · doi:10.1126/science.1227356 · Cited by 185
Ethology as a physical science
Brown, A. E., & de Bivort, B. L. (2018) · Nature Physics · doi:10.1038/s41567-018-0093-0 · Cited by 171
Reinforcement learning to adjust parametrized motor primitives to new situations
Kober, J., Wilhelm, A., Öztop, E., & Peters, J. (2012) · Autonomous Robots · doi:10.1007/s10514-012-9290-3 · Cited by 163
Cooperative Deep Q-Learning With Q-Value Transfer for Multi-Intersection Signal Control
Ge, H., Song, Y., Wu, C., Ren, J., & Tan, G. (2019) · IEEE Access · doi:10.1109/access.2019.2907618 · Cited by 160
Model-based hierarchical reinforcement learning and human action control
Botvinick, M., & Weinstein, A. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0480 · Cited by 166
Liu, Z., Yao, C., Yu, H., & Wu, T. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.02.068 · Cited by 153
Reinforcement Learning in Robotics: A Survey
Kober, J., & Peters, J. (2012) · Adaptation, learning, and optimization · doi:10.1007/978-3-642-27645-3_18 · Cited by 150
Solway, A., Diuk, C., Córdova, N. I., Yee, D. M., Barto, A. G., Niv, Y., & Botvinick, M. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003779 · Cited by 154
Self-citationAlexander, W. H., & Brown, J. W. (2015) · Neural Computation · doi:10.1162/neco_a_00779 · Cited by 119
Reinforcement learning and human behavior
Shteingart, H., & Loewenstein, Y. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2013.12.004 · Cited by 107
System Design Perspective for Human-Level Agents Using Deep Reinforcement Learning: A Survey
Nguyen, N. D., Nguyen, T., & Nahavandi, S. (2017) · IEEE Access · doi:10.1109/access.2017.2777827 · Cited by 100
Learning to Navigate Through Complex Dynamic Environment With Modular Deep Reinforcement Learning
Wang, Y., He, H., & Sun, C. (2018) · IEEE Transactions on Games · doi:10.1109/tg.2018.2849942 · Cited by 100
A theoretical account of cognitive effects in delay discounting
Kurth-Nelson, Z. L., Bickel, W. K., & Redish, A. D. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08058.x · Cited by 101
The structure of reinforcement-learning mechanisms in the human brain
O’Doherty, J. P., Lee, S. W., & McNamee, D. (2014) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2014.10.004 · Cited by 106
Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learning
Dilokthanakul, N., Kaplanis, C., Pawlowski, N., & Shanahan, M. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2891792 · Cited by 100
GRAIL: A Goal-Discovering Robotic Architecture for Intrinsically-Motivated Learning
Santucci, V. G., Baldassarre, G., & Mirolli, M. (2016) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2016.2538961 · Cited by 99
Symbol Emergence in Cognitive Developmental Systems: A Survey
Taniguchi, T., Piater, J., Worgotter, F., Ugur, E., Hoffmann, M., Jamone, L., Nagai, T., Rosman, B., Matsuka, T., Iwahashi, N., et al. (2019) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2018.2867772 · Cited by 92
From motor control to team play in simulated humanoid football
Liu, S., Lever, G., Wang, Z., Merel, J., Eslami, S. M. A., Hennes, D., Czarnecki, W. M., Tassa, Y., Omidshafiei, S., Abdolmaleki, A., et al. (2022) · Science Robotics · doi:10.1126/scirobotics.abo0235 · Cited by 88
The new robotics—towards human‐centered machines
Schaal, S. (2007) · HFSP journal · doi:10.2976/1.2748612 · Cited by 87
Parsing Facades with Shape Grammars and Reinforcement Learning
Teboul, O., Kokkinos, I., Simon, L., Koutsourakis, P., & Nikos Paragios, A. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.252 · Cited by 80
Diuk, C., Tsai, K., Wallis, J. D., Botvinick, M., & Niv, Y. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.5445-12.2013 · Cited by 79
Fast reinforcement learning with generalized policy updates
Barreto, A. D. M. S., Hou, S., Borsa, D., Silver, D., & Precup, D. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1907370117 · Cited by 94
A reinforcement learning extension to the Almgren-Chriss framework for optimal trade execution
Hendricks, D., & Wilcox, D. (2014) · IEEE Conference on Computational Intelligence for Financial Engineering & Economics · doi:10.1109/cifer.2014.6924109 · Cited by 77
RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs
Chaudhari, S., Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., Deshpande, A., & Castro da Silva, B. (2026) · ACM Computing Surveys · doi:10.1145/3743127 · Cited by 92
Dialogue Management in Conversational Systems: A Review of Approaches, Challenges, and Opportunities
Brabra, H., Báez, M., Benatallah, B., Gaaloul, W., Bouguelia, S., & Zamanirad, S. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3086565 · Cited by 75
Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulation
Beyret, B., Shafti, A., & Faisal, A. A. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968488 · Cited by 74
Feature-based learning improves adaptability without compromising precision
Farashahi, S. G., Rowe, K., Aslami, Z., Lee, D., & Soltani, A. (2017) · Nature Communications · doi:10.1038/s41467-017-01874-w · Cited by 76
Agent-based evacuation modeling with multiple exits using NeuroEvolution of Augmenting Topologies
Yüksel, M. E. (2017) · Advanced Engineering Informatics · doi:10.1016/j.aei.2017.11.003 · Cited by 69
Functions and Mechanisms of Intrinsic Motivations
Mirolli, M., & Baldassarre, G. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_3 · Cited by 70
Hamaguchi, K., & MOONEY, R. D. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.1666-12.2012 · Cited by 67
Rusu, S. I., & Pennartz, C. M. A. (2019) · Hippocampus · doi:10.1002/hipo.23167 · Cited by 63
He, Y., Wang, Y., Lin, Q., & Li, J. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3146439 · Cited by 65
Multi-agent Reinforcement Learning for Intrusion Detection
Servin, A., & Kudenko⋆, D. (2008) · Springer eBooks · doi:10.1007/978-3-540-77949-0_15 · Cited by 63
QL-BT: Enhancing behaviour tree design and implementation with Q-learning
Dey, R., & Child, C. (2013) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2013.6633623 · Cited by 62
Deep Reinforcement Learning for Event-Driven Multi-Agent Decision Processes
Menda, K., Chen, Y.-C., Grana, J., Bono, J. W., Tracey, B., Kochenderfer, M. J., & Wolpert, D. H. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2848264 · Cited by 61
Deep Hierarchical Reinforcement Learning for Autonomous Driving with Distinct Behaviors
Chen, J., Wang, Z., & Tomizuka, M. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500368 · Cited by 59
Divide et impera : subgoaling reduces the complexity of probabilistic inference and problem solving
Maisto, D., Donnarumma, F., & Pezzulo, G. (2015) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.1335 · Cited by 59
Multiagent Learning of Coordination in Loosely Coupled Multiagent Systems
Yu, C., Zhang, M., Ren, F., & Tan, G. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2387277 · Cited by 58
Evidence integration in model-based tree search
Solway, A., & Botvinick, M. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1505483112 · Cited by 57
Yu, T., Wang, Y., Ye, W., Zhou, B., & Chan, K. W. (2011) · IET Generation Transmission & Distribution · doi:10.1049/iet-gtd.2010.0600 · Cited by 56
Introduction. Modelling natural action selection
Prescott, T. J., Bryson, J. J., & Seth, A. K. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2050 · Cited by 56
Reinforcement learning, efficient coding, and the statistics of natural tasks
Botvinick, M., Weinstein, A., Solway, A., & Barto, A. G. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.08.009 · Cited by 53
Self-citationAbstraction and Generalization in Reinforcement Learning: A Summary and Framework
Ponsen, M., Taylor, M. E., & Tuyls, K. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-11814-2_1 · Cited by 50
Polvara, R., Sharma, S. K., Wan, J., Manning, A. J., & Sutton, R. (2019) · Robotica · doi:10.1017/s0263574719000316 · Cited by 47
Advanced Reinforcement Learning
Daw, N. D. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00016-4 · Cited by 46
Using dimensionality reduction to exploit constraints in reinforcement learning
Bitzer, S., Howard, M., & Vijayakumar, S. (2010) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2010.5650243 · Cited by 43
The Misbehavior of Reinforcement Learning
Mongillo, G., Shteingart, H., & Loewenstein, Y. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2307022 · Cited by 39
Learning like a baby: a survey of artificial intelligence approaches
Guérin, F. (2011) · The Knowledge Engineering Review · doi:10.1017/s0269888911000038 · Cited by 41
Hierarchical Decision Theoretic Planning for Navigation Among Movable Obstacles
Levihn, M., Scholz, J., & Stilman, M. (2013) · Springer tracts in advanced robotics · doi:10.1007/978-3-642-36279-8_2 · Cited by 41
Suryanarayana, S. M., Kotaleski, J. H., Grillner, S. E., & GURNEY, K. N. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.10.003 · Cited by 39
Cerebellar Input Configuration Toward Object Model Abstraction in Manipulation Tasks
Luque, N. R., Garrido, J. A., Carrillo, R. R., Coenen, O., & Ros, E. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2156809 · Cited by 37
Sim-to-Real Quadrotor Landing via Sequential Deep Q-Networks and Domain Randomization
Polvara, R., Patacchiola, M., Hanheide, M., & Neumann, G. M. (2020) · Robotics · doi:10.3390/robotics9010008 · Cited by 38
Ginzburg-Ganz, E., Segev, I., Balabanov, A., Segev, E., Naveh, S. K., Machlev, R., Belikov, J., Katzir, L., Keren, S., & Levron, Y. (2024) · Energies · doi:10.3390/en17215307 · Cited by 37
Competence progress intrinsic motivation
Stout, A., & Barto, A. G. (2010) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2010.5578835 · Cited by 37
Self-citationTowards a common implementation of reinforcement learning for multiple robotic tasks
Martínez-Tenor, Á., Fernández‐Madrigal, J., Cruz‐Martín, A., & González-Jiménez, J. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.11.011 · Cited by 36
Online Planning for Large Markov Decision Processes with Hierarchical Decomposition
Bai, A., Wu, F., & Chen, X. (2015) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2717316 · Cited by 35
Event‐Predictive Cognition: A Root for Conceptual Human Thought
Butz, M. V., Achimova, A., Bilkey, D. K., & Knott, A. (2020) · Topics in Cognitive Science · doi:10.1111/tops.12522 · Cited by 40
Strategy selection as rational metareasoningPreprint
Lieder, F., & Griffiths, T. L. (2016) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/6yugk · Cited by 48
Online Selective Kernel-Based Temporal Difference Learning
Chen, X., Gao, Y., & Wang, R. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2270561 · Cited by 30
Fraud detection in banking using deep reinforcement learning
Bouchti, A. E., Chakroun, A., Abbar, H., & Okar, C. (2017) · International Conference on Innovative Computing Technology · doi:10.1109/intech.2017.8102446 · Cited by 31
Evolving neural network ensembles for control problems
Pardoe, D., Ryoo, M. S., & Miikkulainen, R. P. (2005) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1068009.1068230 · Cited by 30
An Overview of Deep Reinforcement Learning
Cao, L., & Zhimin (2019) · International Conference on Automation, Control and Robotics Engineering (CACRE) · doi:10.1145/3351917.3351989 · Cited by 31
Doroodgar, B., & Nejat, G. (2010) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/coase.2010.5584599 · Cited by 29
Hierarchical correlated Q-learning for multi-layer optimal generation command dispatch
Yu, T., Zhang, X., Zhou, B., & Chan, K. W. (2015) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2015.11.057 · Cited by 29
Automated Flowsheet Synthesis Using Hierarchical Reinforcement Learning: Proof of Concept
Göttl, Q., Tönges, Y., Grimm, D. G., & Burger, J. (2021) · Chemie Ingenieur Technik · doi:10.1002/cite.202100086 · Cited by 28
A Prioritized objective actor-critic method for deep reinforcement learning
Nguyen, N. D., Nguyen, T. T., Vamplew, P. W., Dazeley, R., & Nahavandi, S. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-05795-0 · Cited by 28
Tommasino, P., Caligiore, D., Mirolli, M., & Baldassarre, G. (2016) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2016.2607018 · Cited by 27
Building a state space for song learning
Mackevicius, E. L., & Fee, M. S. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.12.001 · Cited by 28
Intrinsic Motivation and Introspection in Reinforcement Learning
Merrick, K. E. (2012) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2012.2208457 · Cited by 26
Divide and Conquer: Hierarchical Reinforcement Learning and Task Decomposition in Humans
Diuk, C., Schapiro, A. C., Córdova, N. I., Fernandes, J. J. F. R., Niv, Y., & Botvinick, M. (2013) · Springer eBooks · doi:10.1007/978-3-642-39875-9_12 · Cited by 26
Human-AI interaction in safety-critical network infrastructures
Mussi, M., Metelli, A. M., Restelli, M., Losapio, G., Bessa, R. J., Boos, D., Borst, C., Leto, G., Castagna, A., Chavarriaga, R., et al. (2025) · iScience · doi:10.1016/j.isci.2025.113400 · Cited by 33
How Organisms Gained Causal Independence and How It Might Be Quantified
Farnsworth, K. D. (2018) · Biology · doi:10.3390/biology7030038 · Cited by 26
Multi-Task Hierarchical Imitation Learning for Home Automation
Fox, R., Berenstein, R., Stoica, I. G., & Goldberg, K. Y. (2019) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/coase.2019.8843293 · Cited by 26
Sun, R., Xi, Y., Stefanidis, A., Jiang, Z., & Su, J. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-01884-y · Cited by 30
Effective and Diverse Adaptive Game AI
Szita, I., Ponsen, M., & Spronck, P. H. M. (2009) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2009.2018706 · Cited by 24
Safe option-critic: learning safety in the option-critic architecture
Jain, A., Khetarpal, K., & Precup, D. (2021) · The Knowledge Engineering Review · doi:10.1017/s0269888921000035 · Cited by 24
ADAPS: Autonomous Driving Via Principled Simulations
Li, W., Wolinski, D., & Lin, M. C. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794239 · Cited by 24
MOSAIC for Multiple-Reward Environments
Sugimoto, N., Haruno, M., Doya, K., & Kawato, M. (2011) · Neural Computation · doi:10.1162/neco_a_00246 · Cited by 24
Traffic Light Control Using Hierarchical Reinforcement Learning and Options Framework
Borges, D. F., Leite, J. P., Moreira, E. M., & Carpinteiro, O. A. S. (2021) · IEEE Access · doi:10.1109/access.2021.3096666 · Cited by 23
Metadata
- Authors
- Barto, A. G., & Mahadevan, S.
- Affiliations
- University of Massachusetts Amherst (US)
- Venue
- Discrete Event Dynamic Systems
- Published
- 2003-10
- Volume
- 13
- Issue
- 4
- Pages
- 341-379
- ISSN
- 0924-6703 (ISSN-L)
- 1573-7594
- OpenAlex
- W4249441547
- Cited by
- 317
- References
- 64
- Citations by year
- 2026: 29 · 2025: 44 · 2024: 8 · 2023: 4 · 2022: 10 · 2021: 14 · 2020: 13 · 2019: 20 · 2018: 21 · 2017: 17 · 2016: 11 · 2015: 12 · 2014: 18 · 2013: 20 · 2012: 23
- FWCI
- 2.49
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Discrete Event Dynamic Systems (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Keywords
- hierarchical reinforcement learning, temporal abstraction, semi-Markov decision process, curse of dimensionality, hierarchical control architecture, concurrent actions, multi-agent coordination, hierarchical memory, partial observability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
