The DOI matches its Crossref record; 74 of the 75 references it cites were found in a registry or an index.
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In robotics, skills are often modelled using the Option framework [78].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [78]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Axten, N., Newell, A., & Simon, H. A. (1973) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2063712 · Cited by 6,886
Newell, A. (1972) · Cited by 6,222
Dynamic Programming and Markov Processes.
Freimer, M. L., & Howard, R. A. (1961) · Journal of the American Statistical Association · doi:10.2307/2282268 · Cited by 3,388
A qualitative physics based on confluences
de Kleer, J., & Brown, J. S. (1984) · Artificial Intelligence · doi:10.1016/0004-3702(84)90037-7 · Cited by 1,510
Planning in a hierarchy of abstraction spaces
Sacerdoti, E. D. (1974) · Artificial Intelligence · doi:10.1016/0004-3702(74)90026-5 · Cited by 1,185
Learning to act using real-time dynamic programming
Barto, A. G., Bradtke, S. J., & Singh, S. P. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(94)00011-o · Cited by 1,129
Learning and executing generalized robot plans
Fikes, R. E., Hart, P. E., & Nilsson, N. J. (1972) · Artificial Intelligence · doi:10.1016/0004-3702(72)90051-3 · Cited by 1,025
Reinforcement learning for robots using neural networks
Lin, L.-J. (1992) · Defense Technical Information Center (DTIC) · Cited by 841
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
Jaakkola, T., Jordan, M. I., & Singh, S. P. (1994) · Neural Computation · doi:10.1162/neco.1994.6.6.1185 · Cited by 801
Research works printed but not linked: 8
- E.G. Araujo, R.A. Grupen, Learning control composition in a complex environment, in: Proc. 4th International Conference on Simulation of Adaptive Behavior, 1996, pp. 333-342.
- S. Mahadevan, N. Marchalleck, T. Das, A. Gosavi, Self-improving factory simulation using continuous-time average-reward reinforcement learning, in: Proc. 14th International Conference on Machine Learning, 1997, pp. 202-210.
- D. Precup, R.S. Sutton, Multi-time models for reinforcement learning, in: Proc. ICML'97 Workshop on Modeling in Reinforcement Learning, 1997.
- D. Precup, R.S. Sutton, S.P. Singh, Planning with closed-loop macro actions, in: Working Notes 1997 AAAI Fall Symposium on Model-directed Autonomous Systems, 1997, pp. 70-76.
- S. Sastry, Algorithms for design of hybrid systems, in: Proc. International Conference of Information Sciences, 1997.
- S.P. Singh, Scaling reinforcement learning by learning variable temporal resolution models, in: Proc. 9th International Conference on Machine Learning, Morgan Kaufmann, San Mateo, CA, 1992, pp. 406-415.
- R.S. Sutton, B. Pinette, The learning of world models by connectionist networks, in: Proc. 7th Annual Conference of the Cognitive Science Society, 1985, pp. 54-64.
- C.J.C.H. Watkins, Learning with delayed rewards, Ph.D. Thesis, Cambridge University, 1989.
Cited by
Sorted by citations
Items 301–400 of 1,012
Efficient skill learning using abstraction selection
Konidaris, G., & Barto, A. G. (2009) · Cited by 69
Which is the best intrinsic motivation signal for learning multiple skills?
Santucci, V. G., Baldassarre, G., & Mirolli, M. (2013) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2013.00022 · Cited by 75
Semi-markov decision problems and performance sensitivity analysis
Cao, X. (2003) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2003.811252 · Cited by 69
Omidshafiei, S., Agha–mohammadi, A., Amato, C., & How, J. P. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7140035 · Cited by 68
Functions and Mechanisms of Intrinsic Motivations
Mirolli, M., & Baldassarre, G. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_3 · Cited by 70
PUMA: Planning Under Uncertainty with Macro-Actions
He, R., Brunskill, E., & Roy, N. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7749 · Cited by 68
Approximate Planning in POMDPs with Macro-Actions
Theocharous, G., & Kaelbling, L. P. (2003) · Cited by 68
Online learning of shaping rewards in reinforcement learning
Grześ, M., & Kudenko⋆, D. (2010) · Neural Networks · doi:10.1016/j.neunet.2010.01.001 · Cited by 68
From Reinforcement Learning to Deep Reinforcement Learning: An Overview
Agostinelli, F., Hocquet, G., Singh, S., & Baldi, P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99492-5_13 · Cited by 73
Motor primitive and sequence self-organization in a hierarchical recurrent neural network
Paine, R. W., & Tani, J. (2004) · Neural Networks · doi:10.1016/j.neunet.2004.08.005 · Cited by 70
Multi-timescale nexting in a reinforcement learning robot
Modayil, J., White, A., & Sutton, R. S. (2014) · Adaptive Behavior · doi:10.1177/1059712313511648 · Cited by 73
Joint Inference of Reward Machines and Policies for Reinforcement Learning
Xu, Z., Gavran, I., Ahmad, Y., Majumdar, R., Neider, D., Topcu, U., & Wu, B. (2020) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v30i1.6756 · Cited by 67
Toward a classification of finite partial-monitoring games
Antos, A., Bartók, G., Pál, D., & Szepesvári, C. (2012) · Theoretical Computer Science · doi:10.1016/j.tcs.2012.10.008 · Cited by 66
Open-Ended Learning: A Conceptual Framework Based on Representational Redescription
Doncieux, S., Filliat, D., Díaz-Rodríguez, N., Hospedales, T. M., Duro, R. J., Coninx, A., Roijers, D. M., Girard, B., Perrin, N., & Sigaud, O. (2018) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2018.00059 · Cited by 66
Urban Driving with Multi-Objective Deep Reinforcement Learning
Li, C., & Czarnecki, K. (2019) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/loxq3741 · Cited by 66
He, Y., Wang, Y., Lin, Q., & Li, J. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3146439 · Cited by 65
BITS: Bi-level Imitation for Traffic Simulation
Xu, D., Chen, Y., Ivanovic, B., & Pavone, M. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161167 · Cited by 65
An algebraic approach to abstraction in reinforcement learning
Ravindran, B., & Barto, A. G. (2004) · Cited by 65
Doroodgar, B., Ficocelli, M., Mobedi, B., & Nejat, G. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509530 · Cited by 65
Hoey, J., Boutilier, C. E., Poupart, P., Olivier, P. L., Monk, A. F., & Mihailidis, A. (2012) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2395123.2395125 · Cited by 65
Autonomous Penetration Testing using Reinforcement LearningPreprint
Schwartz, J., & Kurniawati, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05965 · Cited by 128
Learning and multiagent reasoning for autonomous agents
Stone, P. (2007) · International Joint Conference on Artificial Intelligence · Cited by 62
Khadivi, M., Charter, T., Yaghoubi, M., Jalayer, M., Ahang, M., Shojaeinasab, A., & Najjaran, H. (2025) · Computers & Industrial Engineering · doi:10.1016/j.cie.2025.110856 · Cited by 81
Model-free control based on reinforcement learning for a wastewater treatment problem
Syafiie, S., Tadeo, F., Martínez, E., & Álvarez, T. (2009) · Applied Soft Computing · doi:10.1016/j.asoc.2009.10.018 · Cited by 64
Modeling and Planning with Macro-Actions in Decentralized POMDPs
Amato, C., Konidaris, G., Kaelbling, L. P., & How, J. P. (2019) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11418 · Cited by 69
Symbolic Plans as High-Level Instructions for Reinforcement Learning
Illanes, L., Yan, X., Icarte, R. T., & McIlraith, S. A. (2020) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v30i1.6750 · Cited by 64
Autonomous Skill Acquisition on a Mobile Manipulator
Konidaris, G., Kuindersma, S., Grupen, R. A., & Barto, A. G. (2011) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v25i1.7982 · Cited by 64
Stout, A., Konidaris, G., & Barto, A. G. (2005) · doi:10.21236/ada440079 · Cited by 64
Discrete Sequential Prediction of Continuous Actions for Deep RLPreprint
Metz, L., Ibarz, J., Jaitly, N., & Davidson, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.05035 · Cited by 125
Learning Parameterized SkillsPreprint
Da Silva, B., Konidaris, G., & Barto, A. G. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6398 · Cited by 64
The alignment problem from a deep learning perspective
Ngo, R., Chan, L., & Mindermann, S. (2022) · arXiv · doi:10.48550/arxiv.2209.00626 · Cited by 68
“At each timestep t , it first initializes a model of action sequences (or plans/options [Sutton et al., 1999]) over the next H timesteps: q ( a t : t + H ) .”
MethodTonic exploration governs both flexibility and lapses
Ebitz, R. B., Sleezer, B. J., Jedema, H. P., Bradberry, C. W., & Hayden, B. Y. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007475 · Cited by 58
Knowledge Graph Grounded Goal Planning for Open-Domain Conversation Generation
Xu, J., Wang, H., Niu, Z.-Y., Wu, H., & Che, W. (2020) · AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6474 · Cited by 70
Reinforcement Learning With Human Advice: A Survey
Najar, A., & Chétouani, M. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.584075 · Cited by 71
Liu, H., Wu, W., & Wang, Y. (2022) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2022.3168700 · Cited by 71
Krishnan, S., Garg, A., Liaw, R., Thananjeyan, B., Miller, L., Pokorny, F. T., & Goldberg, K. Y. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364918784350 · Cited by 63
Using Homomorphisms to transfer options across continuous reinforcement learning domains
Soni, V. D., & Singh, S. P. (2006) · Cited by 63
Multi-Level Discovery of Deep OptionsPreprint
Fox, R., Krishnan, S., Stoica, I. G., & Goldberg, K. Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.08294 · Cited by 111
Su, C., Zhang, C., Xia, D., Han, B. A., Wang, C., Chen, G., & Xie, L. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110596 · Cited by 63
Modular deep reinforcement learning from reward and punishment for robot navigation
Wang, J., Elfwing, S., & Uchibe, E. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.12.001 · Cited by 62
Deep Reinforcement Learning for Event-Driven Multi-Agent Decision Processes
Menda, K., Chen, Y.-C., Grana, J., Bono, J. W., Tracey, B., Kochenderfer, M. J., & Wolpert, D. H. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2848264 · Cited by 61
Reinforcement learning models of the dopamine system and their behavioral implications
Daw, N. D., & Touretzky, D. S. (2003) · Cited by 62
Donnarumma, F., Maisto, D., & Pezzulo, G. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004864 · Cited by 63
Hierarchical Reinforcement Learning for Quadruped Locomotion
Jain, D., Işçen, A., & Caluwaerts, K. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967913 · Cited by 66
Yin, B., Zhang, S., & Cheng, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2996562 · Cited by 60
Model-free learning control of neutralization processes using reinforcement learning
Syafiie, S., Tadeo, F., & Martinez, E. C. (2007) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2006.10.009 · Cited by 60
Learning Kinematic Feasibility for Mobile Manipulation Through Deep Reinforcement Learning
Honerkamp, D., Welschehold, T., & Valada, A. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3092685 · Cited by 59
Explainable reinforcement learning for broad-XAI: a conceptual framework and survey
Dazeley, R., Vamplew, P. W., & Cruz, F. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08423-1 · Cited by 91
Intrinsically Motivated Learning of Real-World Sensorimotor Skills with Developmental Constraints
Oudeyer, P., Baranès, A., & Kaplan, F. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_13 · Cited by 59
Poincaré-Map-Based Reinforcement Learning For Biped Walking
Morimoto, J., Nakanishi, J., Endo, G., Cheng, G., Atkeson, C. G., & Zeglin, G. J. (2006) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2005.1570469 · Cited by 59
Omidshafiei, S., Agha–mohammadi, A., Amato, C., Liu, S., How, J. P., & Vian, J. L. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917692864 · Cited by 59
Le, T. P., Vien, N. A., & Chung, T. (2018) · IEEE Access · doi:10.1109/access.2018.2854283 · Cited by 66
Deep Hierarchical Reinforcement Learning for Autonomous Driving with Distinct Behaviors
Chen, J., Wang, Z., & Tomizuka, M. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500368 · Cited by 59
Inter-module credit assignment in modular reinforcement learning
Samejima, K., Doya, K., & Kawato, M. (2003) · Neural Networks · doi:10.1016/s0893-6080(02)00235-6 · Cited by 59
Divide et impera : subgoaling reduces the complexity of probabilistic inference and problem solving
Maisto, D., Donnarumma, F., & Pezzulo, G. (2015) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.1335 · Cited by 59
Deep Reinforcement Learning for Decentralized Multi-Robot Exploration With Macro Actions
Tan, A. H., Bejarano, F. P., Zhu, Y., Ren, R., & Nejat, G. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3224667 · Cited by 58
Reinforcement learning for predictive maintenance: a systematic technical review
Siraskar, R., Kumar, S., Patil, S., Bongale, A. M., & Kotecha, K. V. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10468-6 · Cited by 73
Genetic Programming for Reward Function Search
Niekum, S., Barto, A. G., & Spector, L. C. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2051436 · Cited by 58
Henderson, P., Chang, W.-D., Bacon, P., Meger, D., Pineau, J., & Precup, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11775 · Cited by 12
A survey of reinforcement learning in relational domains
van Otterlo, M. (2005) · CTIT technical reports series · Cited by 58
Distributed Planning in Hierarchical Factored MDPsPreprint
Guestrin, C., & Gordon, G. J. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.0571 · Cited by 58
Temporal-Difference NetworksPreprint
Sutton, R. S., & Tanner, B. K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.05539 · Cited by 62
Mice learn multi-step routes by memorizing subgoal locations
Shamash, P., Olesen, S. F., Ιορδανίδου, Π., Campagner, D., Banerjee, N., & Branco, T. (2021) · Nature Neuroscience · doi:10.1038/s41593-021-00884-8 · Cited by 55
RETRACTED: 3D reconstruction based on hierarchical reinforcement learning with transferability
Li, L., He, F., Fan, R., Fan, B., & Yan, X. (2023) · Integrated Computer-Aided Engineering · doi:10.3233/ica-230710 · Cited by 57
RetractedA framework for meta-level control in multi-agent systems
Raja, A., & Lesser, V. (2007) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-006-9008-z · Cited by 58
Learning Generalizable Control Programs
Hart, S., & Grupen, R. A. (2011) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2103311 · Cited by 57
Deep Reinforcement Learning: A State-of-the-Art Walkthrough
Lazaridis, A., Fachantidis, A., & Vlahavas, I. P. (2020) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12412 · Cited by 59
Reinforcement learning and its connections with neuroscience and psychology
Subramanian, A., Chitlangia, S., & Baths, V. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.10.003 · Cited by 66
Chaudhuri, S., Ellis, K., Polozov, O., Singh, R., Solar-Lezama, A., & Yue, Y. (2021) · Foundations and Trends® in Programming Languages · doi:10.1561/2500000049 · Cited by 83
Liu, B., Han, W., Wang, E., Xiong, S., Wu, L., Wang, Q., Wang, J., & Qiao, C. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3233879 · Cited by 58
Yu, T., Wang, Y., Ye, W., Zhou, B., & Chan, K. W. (2011) · IET Generation Transmission & Distribution · doi:10.1049/iet-gtd.2010.0600 · Cited by 56
A Comparative Study of Value Systems for Self-Motivated Exploration and Learning by Robots
Merrick, K. E. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2051435 · Cited by 56
Tractable planning under uncertainty: exploiting structure
Pineau, J., Gordon, G. J., & Thrun, S. (2004) · Cited by 57
Option Discovery using Deep Skill Chaining
Bagaria, A., & Konidaris, G. (2020) · International Conference on Learning Representations · Cited by 56
Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigation
Irshad, M. Z., Ma, C., & Kira, Z. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561806 · Cited by 63
Unifying task specification in reinforcement learningPreprint
White, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.01995 · Cited by 56
Zhu, A., Ouyang, D., Liang, S., & Shao, J. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108843 · Cited by 54
Machine learning of motor skills for robotics
Schaal, S., & Peters, J. (2007) · Cited by 55
An Introduction to Intertask Transfer for Reinforcement Learning
Taylor, M. E., & STONE, P. R. (2011) · AI Magazine · doi:10.1609/aimag.v32i1.2329 · Cited by 57
Autonomous Acquisition of Behavior Trees for Robot Control
Banerjee, B. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594083 · Cited by 55
Q-learning-based simulated annealing algorithm for constrained engineering design problems
Samma, H., Mohamad–Saleh, J., Suandi, S. A., & Lahasan, B. M. (2019) · Neural Computing and Applications · doi:10.1007/s00521-019-04008-z · Cited by 55
Hierarchical Multiagent Reinforcement Learning
Ghavamzadeh, M., & Mahadevan, S. (2004) · doi:10.21236/ada440418 · Cited by 55
Memorize or generalize? Searching for a compositional RNN in a haystackPreprint
Liška, A., Kruszewski, G., & Baroni, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.06467 · Cited by 81
Hierarchical Reinforcement Learning with HindsightPreprint
Levy, A., Platt, R., & Saenko, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08180 · Cited by 55
Bee Behaviour in Multi-agent Systems
Lemmens, N., de Jong, S., Tuyls, K., & Nowé, A. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-77949-0_11 · Cited by 54
Bayesian Nonparametric Reward Learning From Demonstration
Michini, B., Walsh, T. J., Agha–mohammadi, A., & How, J. P. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2405593 · Cited by 54
Federated Reinforcement Learning in IoT: Applications, Opportunities and Open Challenges
Neto, E. C. P., Sadeghi, S., Zhang, X., & Dadkhah, S. (2023) · Applied Sciences · doi:10.3390/app13116497 · Cited by 53
Projective Simulation for Classical Learning Agents: A Comprehensive Investigation
Mautner, J., Makmal, A., Manzano, D., Tiersch, M., & Briegel, H. J. (2015) · New Generation Computing · doi:10.1007/s00354-015-0102-0 · Cited by 53
Complex Program Induction for Querying Knowledge Bases in the Absence of Gold Programs
Saha, A., Ansari, G. A., Laddha, A., Sankaranarayanan, K., & Chakrabarti, S. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00262 · Cited by 55
Memory Augmented Control NetworksPreprint
Khan, A., Zhang, C., Atanasov, N., Karydis, K., Kumar, V., & Lee, D. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.05706 · Cited by 53
DDCO: Discovery of Deep Continuous Options for Robot Learning from DemonstrationsPreprint
Krishnan, S., Fox, R., Stoica, I. G., & Goldberg, K. Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.05421 · Cited by 53
Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal GenerationPreprint
Nair, S., & Finn, C. (2019) · International Conference on Learning Representations · doi:10.48550/arxiv.1909.05829 · Cited by 163
Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?Preprint
Nachum, O., Tang, H., Lu, X., Gu, S., Lee, H., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.10618 · Cited by 53
Parrot: Data-Driven Behavioral Priors for Reinforcement LearningPreprint
Singh, A., Liu, H., Zhou, G., Yu, A., Rhinehart, N., & Levine, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.10024 · Cited by 101
Hierarchical Query Graph Generation for Complex Question Answering over Knowledge Graph
Qiu, Y., Zhang, K., Wang, Y., Jin, X., Bai, L., Guan, S., & Cheng, X. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3411888 · Cited by 52
An Information-Theoretic Perspective on Intrinsic Motivation in Reinforcement Learning: A Survey
Aubret, A., Matignon, L., & Hassas, S. (2023) · Entropy · doi:10.3390/e25020327 · Cited by 52
Incorporating Q-learning and gradient search scheme into JAYA algorithm for global optimization
Deng, L., & Liu, S. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10613-1 · Cited by 52
Continual curiosity-driven skill acquisition from high-dimensional video inputs for humanoid robots
Kompella, V. R., Stollenga, M., Luciw, M., & Schmidhuber, J. (2015) · Artificial Intelligence · doi:10.1016/j.artint.2015.02.001 · Cited by 52
Developing PFC representations using reinforcement learning
Reynolds, J. R., & O’Reilly, R. C. (2009) · Cognition · doi:10.1016/j.cognition.2009.05.015 · Cited by 53
Nonparametric Bayesian reward segmentation for skill discovery using inverse reinforcement learning
Ranchod, P., Rosman, B., & Konidaris, G. (2015) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353414 · Cited by 52
Metadata
- Authors
- Sutton, R. S., Precup, D., & Singh, S.
- Affiliations
- AT&T (United States) (US)
- University of Massachusetts Amherst (US)
- Corresponding author
- Sutton, Richard S.
- Venue
- Artificial Intelligence
- Publisher
- Elsevier BV
- Published
- 1999-08
- Volume
- 112
- Issue
- 1-2
- Pages
- 181-211
- ISSN
- 0004-3702 (ISSN-L)
- OpenAlex
- W2109910161
- Cited by
- 3,119
- References
- 75
- Citations by year
- 2026: 179 · 2025: 206 · 2024: 188 · 2023: 212 · 2022: 213 · 2021: 270 · 2020: 303 · 2019: 238 · 2018: 183 · 2017: 112 · 2016: 99 · 2015: 80 · 2014: 74 · 2013: 91 · 2012: 86
- FWCI
- 27.32
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Artificial Intelligence (publishedVersion)
- Funders
- National Science Foundation (9711753, 9511805)
- Air Force Office of Scientific Research (F49620)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- temporal abstraction, options framework, semi-Markov decision process, Markov decision process, hierarchical reinforcement learning, Q-learning, dynamic programming
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
