The DOI matches its Crossref record; 74 of the 75 references it cites were found in a registry or an index.
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In robotics, skills are often modelled using the Option framework [78].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [78]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Axten, N., Newell, A., & Simon, H. A. (1973) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2063712 · Cited by 6,886
Newell, A. (1972) · Cited by 6,222
Dynamic Programming and Markov Processes.
Freimer, M. L., & Howard, R. A. (1961) · Journal of the American Statistical Association · doi:10.2307/2282268 · Cited by 3,388
A qualitative physics based on confluences
de Kleer, J., & Brown, J. S. (1984) · Artificial Intelligence · doi:10.1016/0004-3702(84)90037-7 · Cited by 1,510
Planning in a hierarchy of abstraction spaces
Sacerdoti, E. D. (1974) · Artificial Intelligence · doi:10.1016/0004-3702(74)90026-5 · Cited by 1,185
Learning to act using real-time dynamic programming
Barto, A. G., Bradtke, S. J., & Singh, S. P. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(94)00011-o · Cited by 1,129
Learning and executing generalized robot plans
Fikes, R. E., Hart, P. E., & Nilsson, N. J. (1972) · Artificial Intelligence · doi:10.1016/0004-3702(72)90051-3 · Cited by 1,025
Reinforcement learning for robots using neural networks
Lin, L.-J. (1992) · Defense Technical Information Center (DTIC) · Cited by 841
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
Jaakkola, T., Jordan, M. I., & Singh, S. P. (1994) · Neural Computation · doi:10.1162/neco.1994.6.6.1185 · Cited by 801
Research works printed but not linked: 8
- E.G. Araujo, R.A. Grupen, Learning control composition in a complex environment, in: Proc. 4th International Conference on Simulation of Adaptive Behavior, 1996, pp. 333-342.
- S. Mahadevan, N. Marchalleck, T. Das, A. Gosavi, Self-improving factory simulation using continuous-time average-reward reinforcement learning, in: Proc. 14th International Conference on Machine Learning, 1997, pp. 202-210.
- D. Precup, R.S. Sutton, Multi-time models for reinforcement learning, in: Proc. ICML'97 Workshop on Modeling in Reinforcement Learning, 1997.
- D. Precup, R.S. Sutton, S.P. Singh, Planning with closed-loop macro actions, in: Working Notes 1997 AAAI Fall Symposium on Model-directed Autonomous Systems, 1997, pp. 70-76.
- S. Sastry, Algorithms for design of hybrid systems, in: Proc. International Conference of Information Sciences, 1997.
- S.P. Singh, Scaling reinforcement learning by learning variable temporal resolution models, in: Proc. 9th International Conference on Machine Learning, Morgan Kaufmann, San Mateo, CA, 1992, pp. 406-415.
- R.S. Sutton, B. Pinette, The learning of world models by connectionist networks, in: Proc. 7th Annual Conference of the Cognitive Science Society, 1985, pp. 54-64.
- C.J.C.H. Watkins, Learning with delayed rewards, Ph.D. Thesis, Cambridge University, 1989.
Cited by
Sorted by citations
Items 201–300 of 1,012
SDRL: Interpretable and Data-Efficient Deep Reinforcement Learning Leveraging Symbolic Planning
Lyu, D., Yang, F., Liu, B., & Gustafson, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33012970 · Cited by 103
Yang, F., Lyu, D., Liu, B., & Gustafson, S. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/675 · Cited by 103
A Survey on Reinforcement Learning for Recommender Systems
Lin, Y., Liu, Y., Lin, F., Zou, L., Wu, P., Zeng, W., Chen, H., & Miao, C. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3280161 · Cited by 125
Bayesian Relational Memory for Semantic Visual Navigation
Wu, Y., Wu, Y., Tamar, A., Russell, S., Gkioxari, G., & Tian, Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00286 · Cited by 97
Yoo, S. B. M., Hayden, B. Y., & Pearson, J. M. (2021) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2019.0664 · Cited by 101
Intelligent problem-solving as integrated hierarchical reinforcement learning
Eppe, M., Gumbsch, C., Kerzel, M., Nguyen, P. D. H., Butz, M. V., & Wermter, S. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-021-00433-9 · Cited by 110
An intrinsic reward mechanism for efficient exploration
Şimşek, Ö., & Barto, A. G. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143949 · Cited by 97
Near-Optimal Representation Learning for Hierarchical Reinforcement LearningPreprint
Nachum, O., Gu, S., Lee, H., & Levine, S. (2018) · International Conference on Learning Representations · doi:10.48550/arxiv.1810.01257 · Cited by 239
The structure of reinforcement-learning mechanisms in the human brain
O’Doherty, J. P., Lee, S. W., & McNamee, D. (2014) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2014.10.004 · Cited by 106
Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learning
Dilokthanakul, N., Kaplanis, C., Pawlowski, N., & Shanahan, M. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2891792 · Cited by 100
Hierarchical Reinforcement Learning for Air Combat at DARPA's AlphaDogfight Trials
Pope, A. P., Ide, J. S., Mićović, D., Díaz, H., Twedt, J. C., Alcedo, K., Walker, T. T., Rosenbluth, D., Ritholtz, L., & Javorsek, D. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3222143 · Cited by 104
Teaching Multiple Tasks to an RL Agent using LTL
Icarte, R. T., Klassen, T. Q., Valenzano, R., & McIlraith, S. A. (2018) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/fknq6967 · Cited by 97
IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Tasks
Lee, Y., Hu, E. S., & Lim, J. J. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560986 · Cited by 102
Discovery of hierarchical representations for efficient planning
Tomov, M. S., Yagati, S., Kumar, A., Yang, W., & Gershman, S. J. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007594 · Cited by 100
The Naïve Utility Calculus as a unified, quantitative framework for action understanding
Jara‐Ettinger, J., Schulz, L., & Tenenbaum, J. B. (2020) · Cognitive Psychology · doi:10.1016/j.cogpsych.2020.101334 · Cited by 100
Hierarchical Imitation and Reinforcement LearningPreprint
Le, H. M., Jiang, N., Agarwal, A., Dudı́k, M., Yue, Y., & Daumé, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.00590 · Cited by 107
Hierarchical Reinforcement Learning With Universal Policies for Multistep Robotic Manipulation
Yang, X., Ji, Z., Wu, J., Lai, Y., Wei, C., Liu, G., & Setchi, R. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3059912 · Cited by 94
Amo, R., Matias, S., Yamanaka, A., Tanaka, K. F., Uchida, N., & Watabe‐Uchida, M. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01109-2 · Cited by 94
GRAIL: A Goal-Discovering Robotic Architecture for Intrinsically-Motivated Learning
Santucci, V. G., Baldassarre, G., & Mirolli, M. (2016) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2016.2538961 · Cited by 99
A Laplacian Framework for Option Discovery in Reinforcement LearningPreprint
Machado, M. C., Bellemare, M. G., & Bowling, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00956 · Cited by 93
Goal-directed control and its antipodes
Dayan, P. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.004 · Cited by 93
Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
Parker-Holder, J., Rajan, R., Song, X., Biedenkapp, A., Miao, Y., Eimer, T., Zhang, B., Nguyen, V., Calandra, R., Faust, A., et al. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13596 · Cited by 114
(2021) · Journal of Machine Learning Research · Cited by 224
Symbol Emergence in Cognitive Developmental Systems: A Survey
Taniguchi, T., Piater, J., Worgotter, F., Ugur, E., Hoffmann, M., Jamone, L., Nagai, T., Rosman, B., Matsuka, T., Iwahashi, N., et al. (2019) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2018.2867772 · Cited by 92
Theory and application of reward shaping in reinforcement learning
DEJONG, G. F., & Laud, A. (2004) · Cited by 92
PolicyBlocks: An Algorithm for Creating Useful Macro-Actions in Reinforcement Learning
Pickett, M., & Barto, A. G. (2002) · Cited by 92
Energy Efficient Multiple Target Tracking in Wireless Sensor Networks
Yeow, W.-L., Tham, C., & Wong, W. (2007) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2007.891480 · Cited by 93
Planning for decentralized control of multiple robots under uncertainty
Amato, C., Konidaris, G., Cruz, G., Maynor, C. A., How, J. P., & Kaelbling, L. P. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139350 · Cited by 91
How Hierarchical Control Self-organizes in Artificial Adaptive Systems
Paine, R. W., & Tani, J. (2005) · Adaptive Behavior · doi:10.1177/105971230501300303 · Cited by 91
Co-Reyes, J. D., Liu, Y., Gupta, A., Eysenbach, B., Abbeel, P., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02813 · Cited by 107
MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at ScalePreprint
Kalashnikov, D., Varley, J., Chebotar, Y., Swanson, B., Jonschkowski, R., Finn, C., Levine, S., & Hausman, K. (2021) · arXiv.org · doi:10.48550/arxiv.2104.08212 · Cited by 344
Autonomous Learning of High-Level States and Actions in Continuous Environments
Mugan, J., & Kuipers, B. J. (2011) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2011.2160943 · Cited by 91
GTI: Learning to Generalize across Long-Horizon Tasks from Human Demonstrations
Mandlekar, A., Xu, D., Martín-Martín, R., Savarese, S., & Fei-Fei, L. (2020) · Robotics: Science and Systems XVI · doi:10.15607/rss.2020.xvi.061 · Cited by 90
Skew-Fit: State-Covering Self-Supervised Reinforcement LearningPreprint
Pong, V. H., Dalal, M., Lin, S., Nair, A., Bahl, S., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03698 · Cited by 90
A survey on intrinsic motivation in reinforcement learningPreprint
Aubret, A., Matignon, L., & Hassas, S. (2019) · arXiv.org · doi:10.48550/arxiv.1908.06976 · Cited by 175
Probabilistic inference for determining options in reinforcement learning
Daniel, C., van Hoof, H., Peters, J., & Neumann, G. M. (2016) · Machine Learning · doi:10.1007/s10994-016-5580-x · Cited by 95
Silva, B. D. (2021) · Scholarworks (University of Massachusetts Amherst) · doi:10.7275/6439644.0 · Cited by 89
A survey on visual navigation for artificial agents with deep reinforcement learning
Zeng, F., Wang, C., & Ge, S. S. (2020) · IEEE Access · doi:10.1109/access.2020.3011438 · Cited by 98
Computational evidence for hierarchically structured reinforcement learning in humans
Eckstein, M. K., & Collins, A. G. E. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1912330117 · Cited by 92
From motor control to team play in simulated humanoid football
Liu, S., Lever, G., Wang, Z., Merel, J., Eslami, S. M. A., Hennes, D., Czarnecki, W. M., Tassa, Y., Omidshafiei, S., Abdolmaleki, A., et al. (2022) · Science Robotics · doi:10.1126/scirobotics.abo0235 · Cited by 88
Optimal control in microgrid using multi-agent reinforcement learning
Li, F.-D., Wu, M., He, Y., & Chen, X. (2012) · ISA Transactions · doi:10.1016/j.isatra.2012.06.010 · Cited by 88
Intrinsically motivated goal exploration for active motor learning in robots: A case study
Baranès, A., & Oudeyer, P. (2010) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2010.5651385 · Cited by 88
Probabilistic Policy Reuse for inter-task transfer learning
Fernández, F., García, J., & Veloso, M. (2010) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2010.03.007 · Cited by 87
UAV Path Planning using Global and Local Map Information with Deep Reinforcement Learning
Theile, M., Bayerlein, H., Nai, R., Gesbert, D., & Caccamo, M. (2021) · International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar53236.2021.9659413 · Cited by 85
Transfer Learning in Natural Language Processing
Yang, Q., Zhang, Y., Dai, W., & Pan, S. J. (2020) · Cambridge University Press eBooks · doi:10.1017/9781139061773.020 · Cited by 94
Distributional Reinforcement Learning
Bellemare, M. G., Dabney, W., & Rowland, M. (2023) · The MIT Press eBooks · doi:10.7551/mitpress/14207.001.0001 · Cited by 102
Qiu, D., Wang, Y., Zhang, T., Sun, M., & Štrbac, G. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.120826 · Cited by 84
How can we define intrinsic motivation
Oudeyer, P., & Kaplan, F. (2008) · Cited by 84
Transfer in variable-reward hierarchical reinforcement learning
Mehta, N., Natarajan, S., Tadepalli, P. V., & Fern, A. (2008) · Machine Learning · doi:10.1007/s10994-008-5061-y · Cited by 84
Efficient reinforcement learning: computational theories, neuroscience and robotics
Kawato, M., & Samejima, K. (2007) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2007.03.004 · Cited by 82
Multitime scale Markov decision processes
Chang, H. S., Fard, P. J., Marcus, S. I., & Shayman, M. A. (2003) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2003.812782 · Cited by 83
Spatial Learning and Action Planning in a Prefrontal Cortical Network Model
Martinet, L., Sheynikhovich, D., Benchenane, K., & Arleo, A. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002045 · Cited by 84
Constructing Symbolic Representations for High-Level Planning
Konidaris, G., Kaelbling, L. P., & Lozano‐Pérez, T. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v28i1.9004 · Cited by 81
Review of mission planning for autonomous marine vehicle fleets
Thompson, F., & Guihen, D. (2018) · Journal of Field Robotics · doi:10.1002/rob.21819 · Cited by 87
Ashraf, N. M., Mostafa, R. R., Sakr, R. H., & Rashad, M. Z. (2021) · PLoS ONE · doi:10.1371/journal.pone.0252754 · Cited by 87
Efficient Planning under Uncertainty with Macro-actions
He, R., Brunskill, E., & Roy, N. (2011) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3171 · Cited by 81
Constructing Skill Trees for Reinforcement Learning Agents from Demonstration Trajectories
Konidaris, G., Kuindersma, S., Grupen, R. A., & Barreto, A. S. (2010) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 81
Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning
Li, R., Jabri, A., Darrell, T., & Agrawal, P. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197468 · Cited by 79
Learning compositional models of robot skills for task and motion planning
Wang, Z., Garrett, C. R., Kaelbling, L. P., & Lozano‐Pérez, T. (2021) · The International Journal of Robotics Research · doi:10.1177/02783649211004615 · Cited by 80
SMDP homomorphisms: an algebraic approach to abstraction in semi-Markov decision processes
Ravindran, B., & Barto, A. G. (2003) · Cited by 80
A simple reinforcement learning algorithm for biped walking
Morimoto, J., Cheng, G., Atkeson, C. G., & Zeglin, G. J. (2004) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2004.1307522 · Cited by 80
Internal Models and Anticipations in Adaptive Learning Systems
Butz, M. V., Sigaud, O., & Gérard, P. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-45002-3_6 · Cited by 82
Transfer in Reinforcement Learning via Shared Features
Konidaris, G., Scheidwasser, I., & Barto, A. G. (2012) · DSpace@MIT (Massachusetts Institute of Technology) · doi:10.5555/2188385.2343689 · Cited by 79
Morimoto, J., & Atkeson, C. G. (2007) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2007.380654 · Cited by 79
Evolving internal reinforcers for an intrinsically motivated reinforcement-learning robot
Schembri, M., Mirolli, M., & Baldassarre, G. (2007) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2007.4354052 · Cited by 85
Robot weightlifting by direct policy search
Rosenstein, M. T., & Barto, A. G. (2001) · Cited by 79
Diuk, C., Tsai, K., Wallis, J. D., Botvinick, M., & Niv, Y. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.5445-12.2013 · Cited by 79
PAC-inspired Option Discovery in Lifelong Reinforcement Learning
Brunskill, E., & Li, L. (2014) · Cited by 79
Reinforcement Learning Enabled Dynamic Resource Allocation in the Internet of Vehicles
Liang, H., Zhang, X., Hong, X., Zhang, Z., Li, M., Hu, G., & Hou, F. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3019386 · Cited by 78
Gaze Control for Face Learning and Recognition by Humans and Machines
Henderson, J. M., Falk, R., Minur, S., Dyer, F. C., & Mahadevan, S. (2001) · Advances in psychology · doi:10.1016/s0166-4115(01)80034-0 · Cited by 78
Causal Graph Based Decomposition of Factored MDPs
Jönsson, A., & Barto, A. G. (2006) · Cited by 78
Fast reinforcement learning with generalized policy updates
Barreto, A. D. M. S., Hou, S., Borsa, D., Silver, D., & Precup, D. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1907370117 · Cited by 94
Autonomous discovery of temporal abstractions from interaction with an environment
Mcgovern, E. A., & Barto, A. G. (2002) · Cited by 78
Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigation
Li, J., Wang, X., Tang, S., Shi, H., Wu, F. Q., Zhuang, Y., & Wang, W. Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01214 · Cited by 77
Deep reinforcement learning for robust emotional classification in facial expression recognition
Li, H., & Xu, H. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106172 · Cited by 78
Modeling the role of working memory and episodic memory in behavioral tasks
Zilli, E. A., & Hasselmo, M. E. (2007) · Hippocampus · doi:10.1002/hipo.20382 · Cited by 81
Kidziński, Ł., Mohanty, S. P., Ong, C. F., Huang, Z., Zhou, S., Pechenko, A., Stelmaszczyk, A., Jarosik, P., Pavlov, M., Kolesnikov, S., et al. (2018) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-94042-7_7 · Cited by 77
State Abstractions for Lifelong Reinforcement Learning
Abel, D. L., Arumugam, D., Lehnert, L., & Littman, M. L. (2018) · International Conference on Machine Learning · Cited by 76
On Reinforcement Learning for Full-Length Game of StarCraft
Pang, Z.-J., Liu, R.-Z., Meng, Z.-Y., Zhang, Y., Yu, Y., & Lü, T. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014691 · Cited by 82
Automatic discovery and transfer of MAXQ hierarchies
Mehta, N., Ray, S., Tadepalli, P. V., & Dietterich, T. G. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390238 · Cited by 76
Colas, C., Karch, T., Sigaud, O., & Oudeyer, P. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13554 · Cited by 137
Investigating Contingency Awareness Using Atari 2600 Games
Bellemare, M. G., Veness, J., & Bowling, M. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v26i1.8321 · Cited by 75
Instrumental vigour in punishment and reward
Dayan, P. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08026.x · Cited by 73
Solving Multi-Agent Routing Problems Using Deep Attention Mechanisms
Bono, G., Dibangoye, J., Simonin, O., Matignon, L., & Pereyron, F. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3009289 · Cited by 74
Plan-based reward shaping for reinforcement learning
Grześ, M., & Kudenko⋆, D. (2008) · IEEE International Conference on Intelligent Systems · doi:10.1109/is.2008.4670492 · Cited by 74
Model Minimization in Hierarchical Reinforcement Learning
Ravindran, B., & Barto, A. G. (2002) · Lecture notes in computer science · doi:10.1007/3-540-45622-8_15 · Cited by 74
Batch reinforcement learning in a complex domain
Kalyanakrishnan, S., & Stone, P. (2007) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/1329125.1329241 · Cited by 74
AlgaeDICE: Policy Gradient from Arbitrary ExperiencePreprint
Nachum, O., Dai, B., Kostrikov, I., Chow, Y., Li, L., & Schuurmans, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02074 · Cited by 98
A time aggregation approach to Markov decision processes
Cao, X., Ren, Z., Bhatnagar, S., Fu, M. C., & Marcus, S. I. (2002) · Automatica · doi:10.1016/s0005-1098(01)00282-5 · Cited by 73
Neural Approaches to Conversational AI
Gao, J., Galley, M., & Li, L. (2018) · Proceedings of ACL 2018, Tutorial Abstracts · doi:10.18653/v1/p18-5002 · Cited by 73
Hierarchical visuomotor control of humanoidsPreprint
Merel, J., Ahuja, A., Pham, V., Tunyasuvunakool, S., Liu, S., Tirumala, D., Heess, N., & Wayne, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.09656 · Cited by 95
Eigenoption Discovery through the Deep Successor RepresentationPreprint
Machado, M. C., Rosenbaum, C., Guo, X., Liu, M., Tesauro, G. J., & Campbell, M. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.11089 · Cited by 102
Belief Reward Shaping in Reinforcement Learning
Marom, O., & Rosman, B. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11741 · Cited by 70
Jin, S., Wang, X., & Meng, Q. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111358 · Cited by 74
Deep Reinforcement Learning: A Chronological Overview and Methods
Terven, J. R. (2025) · AI · doi:10.3390/ai6030046 · Cited by 98
Hybrid Control for Robot Navigation - A Hierarchical Q-Learning Algorithm
Chen, C., Li, H., & Dong, D. (2008) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2008.921541 · Cited by 70
Design Principles of the Hippocampal Cognitive Map
Stachenfeld, K., Botvinick, M., & Gershman, S. J. (2014) · Cited by 74
On The Necessity of Abstraction
Konidaris, G. (2018) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2018.11.005 · Cited by 114
Doya, K., & Uchibe, E. (2005) · Adaptive Behavior · doi:10.1177/105971230501300206 · Cited by 69
Leveraging Human Guidance for Deep Reinforcement Learning Tasks
Zhang, R., Torabi, F., Guan, L., Ballard, D. H., & Stone, P. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/884 · Cited by 93
Metadata
- Authors
- Sutton, R. S., Precup, D., & Singh, S.
- Affiliations
- AT&T (United States) (US)
- University of Massachusetts Amherst (US)
- Corresponding author
- Sutton, Richard S.
- Venue
- Artificial Intelligence
- Publisher
- Elsevier BV
- Published
- 1999-08
- Volume
- 112
- Issue
- 1-2
- Pages
- 181-211
- ISSN
- 0004-3702 (ISSN-L)
- OpenAlex
- W2109910161
- Cited by
- 3,119
- References
- 75
- Citations by year
- 2026: 179 · 2025: 206 · 2024: 188 · 2023: 212 · 2022: 213 · 2021: 270 · 2020: 303 · 2019: 238 · 2018: 183 · 2017: 112 · 2016: 99 · 2015: 80 · 2014: 74 · 2013: 91 · 2012: 86
- FWCI
- 27.32
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Artificial Intelligence (publishedVersion)
- Funders
- National Science Foundation (9711753, 9511805)
- Air Force Office of Scientific Research (F49620)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- temporal abstraction, options framework, semi-Markov decision process, Markov decision process, hierarchical reinforcement learning, Q-learning, dynamic programming
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
