The DOI matches its Crossref record; 74 of the 75 references it cites were found in a registry or an index.
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In robotics, skills are often modelled using the Option framework [78].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [78]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Axten, N., Newell, A., & Simon, H. A. (1973) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2063712 · Cited by 6,886
Newell, A. (1972) · Cited by 6,222
Dynamic Programming and Markov Processes.
Freimer, M. L., & Howard, R. A. (1961) · Journal of the American Statistical Association · doi:10.2307/2282268 · Cited by 3,388
A qualitative physics based on confluences
de Kleer, J., & Brown, J. S. (1984) · Artificial Intelligence · doi:10.1016/0004-3702(84)90037-7 · Cited by 1,510
Planning in a hierarchy of abstraction spaces
Sacerdoti, E. D. (1974) · Artificial Intelligence · doi:10.1016/0004-3702(74)90026-5 · Cited by 1,185
Learning to act using real-time dynamic programming
Barto, A. G., Bradtke, S. J., & Singh, S. P. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(94)00011-o · Cited by 1,129
Learning and executing generalized robot plans
Fikes, R. E., Hart, P. E., & Nilsson, N. J. (1972) · Artificial Intelligence · doi:10.1016/0004-3702(72)90051-3 · Cited by 1,025
Reinforcement learning for robots using neural networks
Lin, L.-J. (1992) · Defense Technical Information Center (DTIC) · Cited by 841
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
Jaakkola, T., Jordan, M. I., & Singh, S. P. (1994) · Neural Computation · doi:10.1162/neco.1994.6.6.1185 · Cited by 801
Research works printed but not linked: 8
- E.G. Araujo, R.A. Grupen, Learning control composition in a complex environment, in: Proc. 4th International Conference on Simulation of Adaptive Behavior, 1996, pp. 333-342.
- S. Mahadevan, N. Marchalleck, T. Das, A. Gosavi, Self-improving factory simulation using continuous-time average-reward reinforcement learning, in: Proc. 14th International Conference on Machine Learning, 1997, pp. 202-210.
- D. Precup, R.S. Sutton, Multi-time models for reinforcement learning, in: Proc. ICML'97 Workshop on Modeling in Reinforcement Learning, 1997.
- D. Precup, R.S. Sutton, S.P. Singh, Planning with closed-loop macro actions, in: Working Notes 1997 AAAI Fall Symposium on Model-directed Autonomous Systems, 1997, pp. 70-76.
- S. Sastry, Algorithms for design of hybrid systems, in: Proc. International Conference of Information Sciences, 1997.
- S.P. Singh, Scaling reinforcement learning by learning variable temporal resolution models, in: Proc. 9th International Conference on Machine Learning, Morgan Kaufmann, San Mateo, CA, 1992, pp. 406-415.
- R.S. Sutton, B. Pinette, The learning of world models by connectionist networks, in: Proc. 7th Annual Conference of the Cognitive Science Society, 1985, pp. 54-64.
- C.J.C.H. Watkins, Learning with delayed rewards, Ph.D. Thesis, Cambridge University, 1989.
Cited by
Sorted by citations
Items 101–200 of 1,012
Hierarchical Deep Reinforcement Learning for Continuous Action Control
Yang, Z., Merrick, K. E., Jin, L., & Abbass, H. A. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2805379 · Cited by 195
Hierarchical Reinforcement Learning for Course Recommendation in MOOCs
Zhang, J., Hao, B., Chen, B., Li, C., Chen, H., & Sun, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301435 · Cited by 196
Hybrid Reward Architecture for Reinforcement LearningPreprint
van Seijen, H., Fatemi, M., Romoff, J., Laroche, R., Barnes, T., & Tsang, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.04208 · Cited by 196
Tan, B., Song, Y., Zhong, E., & Yang, Q. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783295 · Cited by 200
A Neural Signature of Hierarchical Reinforcement Learning
Fernandes, J. J. F. R., Solway, A., Diuk, C., McGuire, J. T., Barto, A. G., Niv, Y., & Botvinick, M. (2011) · Neuron · doi:10.1016/j.neuron.2011.05.042 · Cited by 207
Stochastic Neural Networks for Hierarchical Reinforcement LearningPreprint
Florensa, C., Duan, Y., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.03012 · Cited by 266
Relational reinforcement learning
Džeroski, S., Raedt, L. D., & Blockeel, H. (1998) · Lecture notes in computer science · doi:10.1007/bfb0027307 · Cited by 196
Dynamic abstraction in reinforcement learning via clustering
Mannor, S., Menache, I., Hoze, A., & Klein, U. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015355 · Cited by 196
A Deep Value-network Based Approach for Multi-Driver Order Dispatching
Tang, X., Qin, Z. T., Zhang, F., Wang, Z., Xu, Z., Ma, Y., Hongtu Zhu, B., & Ye, J. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330724 · Cited by 191
Partially Observable Markov Decision Processes in Robotics: A Survey
Lauri, M., Hsu, D., & Pajarinen, J. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3200138 · Cited by 227
The 2014 General Video Game Playing Competition
Pérez-Liébana, D., Samothrakis, S., Togelius, J., Schaul, T., Lucas, S. M., Couëtoux, A., Lee, J., Lim, C.-U., & Thompson, T. (2015) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2015.2402393 · Cited by 193
Reward Machines: Exploiting Reward Function Structure in Reinforcement Learning
Icarte, R. T., Klassen, T. Q., Valenzano, R., & McIlraith, S. A. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12440 · Cited by 239
An Introduction to Reinforcement Learning
Morales, E. F., & Zaragoza, J. H. (2011) · IGI Global eBooks · doi:10.4018/978-1-60960-165-2.ch004 · Cited by 189
Learning an Embedding Space for Transferable Robot Skills
Hausman, K., Springenberg, J. T., Wang, Z., Heess, N., & Riedmiller, M. (2018) · International Conference on Learning Representations · Cited by 189
Duan, J., Li, S. E., Guan, Y., Sun, Q., & Cheng, B. (2020) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2019.0317 · Cited by 198
Reinforcement learning, conditioning, and the brain: Successes and challenges
Maia, T. V. (2009) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.9.4.343 · Cited by 191
Linear-time inference in Hierarchical HMMs
Murphy, K. P., & Paskin, M. A. (2001) · Cited by 188
Sentience and the Origins of Consciousness: From Cartesian Duality to Markovian Monism
Friston, K. J., Wiese, W., & Hobson, J. A. (2020) · Entropy · doi:10.3390/e22050516 · Cited by 186
Robot Learning from Human Teachers
Chernova, S., & Thomaz, A. L. (2014) · Synthesis lectures on artificial intelligence and machine learning · doi:10.1007/978-3-031-01570-0 · Cited by 240
Behavior Trees in Robotics and AI: An IntroductionPreprint
Colledanchise, M., & Ögren, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.00084 · Cited by 183
Reinforcement learning algorithms with function approximation: Recent advances and applications
Xu, X., Zuo, L., & Huang, Z. (2013) · Information Sciences · doi:10.1016/j.ins.2013.08.037 · Cited by 186
Graying the black box: Understanding DQNsPreprint
Zahavy, T., Ben-Zrihem, N., & Mannor, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02658 · Cited by 200
Using relative novelty to identify useful temporal abstractions in reinforcement learning
Şimşek, Ö., & Barto, A. G. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015353 · Cited by 179
Learning agile soccer skills for a bipedal robot with deep reinforcement learning
Haarnoja, T., Moran, B., Lever, G., Huang, S. H., Tirumala, D., Humplik, J., Wulfmeier, M., Tunyasuvunakool, S., Siegel, N., Hafner, R., et al. (2024) · Science Robotics · doi:10.1126/scirobotics.adi8022 · Cited by 253
The algorithmic anatomy of model-based evaluation
Daw, N. D., & Dayan, P. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0478 · Cited by 195
A New Q-Learning Algorithm Based on the Metropolis Criterion
Guo, M., Liu, Y., & Malec, J. (2004) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2004.832154 · Cited by 178
A Berkeley View of Systems Challenges for AI
Stoica, I., Song, D., Popa, R. A., Patterson, D., Mahoney, M. W., Katz, R., Joseph, A. D., Jordan, M., Hellerstein, J. M., Gonzalez, J. E., et al. (2017) · arXiv · doi:10.48550/arxiv.1712.05855 · Cited by 171
Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning
Peng, B., Li, X., Li, L., Gao, J., Çelikyılmaz, A., Lee, S., & Wong, K. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1237 · Cited by 172
Effective Charging Planning Based on Deep Reinforcement Learning for Electric Vehicles
Zhang, C., Liu, Y., Wu, F., Tang, B., & Fan, W. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3002271 · Cited by 168
Zero-Shot Task Generalization with Multi-Task Deep Reinforcement LearningPreprint
Oh, J., Singh, S. P., Lee, H., & Kohli, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05064 · Cited by 182
Dynamics-Aware Unsupervised Discovery of SkillsPreprint
Sharma, A., Gu, S., Levine, S., Kumar, V., & Hausman, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01657 · Cited by 301
A new Reinforcement Learning-based Memetic Particle Swarm Optimizer
Samma, H., Lim, C. P., & Mohamad–Saleh, J. (2016) · Applied Soft Computing · doi:10.1016/j.asoc.2016.01.006 · Cited by 168
Scaling Reinforcement Learning toward RoboCup Soccer
Stone, P., & Sutton, R. S. (2001) · Cited by 166
Habitual control of goal selection in humans
Cushman, F. A., & Morris, A. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1506367112 · Cited by 172
Neural Task Programming: Learning to Generalize Across Hierarchical Tasks
Xu, D., Nair, S., Zhu, Y., Gao, J., Garg, A., Fei-Fei, L., & Savarese, S. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460689 · Cited by 172
Towards Sample Efficient Reinforcement Learning
Yu, Y. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/820 · Cited by 169
Liang, Y., Ding, Z., Ding, T., & Lee, W. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.3025082 · Cited by 166
Hierarchical Relative Entropy Policy Search
Daniel, C., Neumann, G. M., Kroemer, O., & Peters, J. (2014) · Lincoln Repository (University of Lincoln) · Cited by 166
A survey on model-based reinforcement learning
Luo, F.-M., Xu, T., Lai, H., Chen, X., Zhang, W., & Yu, Y. (2024) · Science China Information Sciences · doi:10.1007/s11432-022-3696-5 · Cited by 205
Dealing with non-stationary environments using context detection
da Silva, B. C., Basso, E. W., Bazzan, A. L. C., & Engel, P. M. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143872 · Cited by 154
Hierarchical multi-agent reinforcement learning
Makar, R., Mahadevan, S., & Ghavamzadeh, M. (2001) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/375735.376302 · Cited by 155
Model-based hierarchical reinforcement learning and human action control
Botvinick, M., & Weinstein, A. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0480 · Cited by 166
Hsiao, K., Kaelbling, L. P., & Lozano‐Pérez, T. (2007) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2007.364201 · Cited by 152
Projective simulation for artificial intelligence
Briegel, H. J., & Cuevas, G. D. L. (2012) · Scientific Reports · doi:10.1038/srep00400 · Cited by 158
Solway, A., Diuk, C., Córdova, N. I., Yee, D. M., Barto, A. G., Niv, Y., & Botvinick, M. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003779 · Cited by 154
Incremental Learning of Tasks From User Demonstrations, Past Experiences, and Vocal Comments
Pardowitz, M., Knoop, S., Dillmann, R., & Zöllner, R. (2007) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2006.886951 · Cited by 143
Soar-RL: integrating reinforcement learning with Soar
Nason, S., & Laird, J. E. (2004) · Cognitive Systems Research · doi:10.1016/j.cogsys.2004.09.006 · Cited by 140
A scalable approach to control diverse behaviors for physically simulated characters
Won, J., Gopinath, D., & Hodgins, J. K. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392381 · Cited by 141
An Emphatic Approach to the Problem of Off-policy Temporal-Difference LearningPreprint
Sutton, R. S., Mahmood, A. R., & White, M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.04269 · Cited by 160
Hierarchical Deep Reinforcement Learning for Backscattering Data Collection With Multiple UAVs
Zhang, Y., Mou, Z., Gao, F., Xing, L., Jiang, J., & Han, Z. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3024666 · Cited by 134
Ride-Hailing Order Dispatching at DiDi via Reinforcement Learning
Qin, Z. T., Tang, X., Jiao, Y., Zhang, F., Xu, Z., Hongtu Zhu, B., & Ye, J. (2020) · INFORMS Journal on Applied Analytics · doi:10.1287/inte.2020.1047 · Cited by 137
Ito, M., & Doya, K. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.04.001 · Cited by 141
Hierarchical Reinforcement Learning for Air-to-Air Combat
Pope, A. P., Ide, J. S., Mićović, D., Díaz, H., Rosenbluth, D., Ritholtz, L., Twedt, J. C., Walker, T. T., Alcedo, K., & Javorsek, D. (2021) · International Conference on Unmanned Aircraft Systems (ICUAS) · doi:10.1109/icuas51884.2021.9476700 · Cited by 135
Hierarchical multi-agent reinforcement learning
Ghavamzadeh, M., Mahadevan, S., & Makar, R. (2006) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-006-7035-4 · Cited by 140
Hierarchical Reinforcement Learning: A Survey and Open Research Challenges
Hutsebaut-Buysse, M., Mets, K., & Latré, S. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4010009 · Cited by 129
End-to-End Driving Via Conditional Imitation Learning
Codevilla, F., Muller, M., Lopez, A., Koltun, V., & Dosovitskiy, A. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460487 · Cited by 129
Rational metareasoning and the plasticity of cognitive control
Lieder, F., Shenhav, A., Musslick, S., & Griffiths, T. L. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006043 · Cited by 136
Robot Learning From Randomized Simulations: A Review
Muratore, F., Ramos, F., Turk, G., Yu, W., Gienger, M., & Peters, J. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.799893 · Cited by 128
Skill Characterization Based on Betweenness
Şimşek, Ö., & Barreto, A. S. (2008) · neural information processing systems · Cited by 127
Variational End-to-End Navigation and Localization
Amini, A., Rosman, G., Karaman, S., & Rus, D. L. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793579 · Cited by 124
Sutton, R. S., Maei, H. R., & Szepesvári, C. (2008) · Cited by 273
A Survey of POMDP Solution Techniques
Murphy, K. P. (2007) · Cited by 125
Latent Space Policies for Hierarchical Reinforcement LearningPreprint
Haarnoja, T., Hartikainen, K., Abbeel, P., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.02808 · Cited by 141
Hierarchical Apprenticeship Learning with Application to Quadruped Locomotion
Kolter, J. Z., Abbeel, P., & Ng, A. Y. (2007) · Cited by 122
Learning from observation using primitives
Bentivegna, D. C., & Atkeson, C. G. (2002) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2001.932899 · Cited by 121
Deep Successor Reinforcement LearningPreprint
Kulkarni, T. D., Saeedi, A., Gautam, S., & Gershman, S. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.02396 · Cited by 202
Robot Learning from Human Teachers
Chernova, S., & Thomaz, A. L. (2014) · Synthesis lectures on artificial intelligence and machine learning · doi:10.2200/s00568ed1v01y201402aim028 · Cited by 119
Multi-Agent Patrolling with Reinforcement Learning
Santana, H., Ramalho, G. L., Corruble, V., & Ratitch, B. (2004) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/mphg3605 · Cited by 119
Value Prediction NetworkPreprint
Oh, J., Singh, S. P., & Lee, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.03497 · Cited by 142
Search on the Replay Buffer: Bridging Planning and Reinforcement Learning
Eysenbach, B., Salakhutdinov, R., & Levine, S. (2019) · arXiv · doi:10.48550/arxiv.1906.05253 · Cited by 123
The successor representation and temporal context
Gershman, S. J., Moore, C. D., Todd, M. T., Norman, K. A., & Sederberg, P. B. (2012) · Neural Computation · doi:10.1162/neco_a_00282 · Cited by 117
Multi-robot task allocation through vacancy chain scheduling
Dahl, T. S., Matarić, M. J., & Sukhatme, G. S. (2008) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.12.001 · Cited by 117
Near Optimal Behavior via Approximate State AbstractionPreprint
Abel, D. L., Hershkowitz, D. E., & Littman, M. L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.04113 · Cited by 117
Unsupervised Learning of Object Keypoints for Perception and ControlPreprint
Kulkarni, T. D., Gupta, A., Ionescu, C., Borgeaud, S., Reynolds, M., Zisserman, A., & Mnih, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.11883 · Cited by 128
Deep reinforcement learning in transportation research: A review
Farazi, N. P., Zou, B., Ahamed, T., & Barua, L. (2021) · Transportation Research Interdisciplinary Perspectives · doi:10.1016/j.trip.2021.100425 · Cited by 139
Wang, Y., Qiu, D., Teng, F., & Štrbac, G. (2023) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2023.3240479 · Cited by 119
Learning and Transfer of Modulated Locomotor ControllersPreprint
Heess, N., Wayne, G., Tassa, Y., Lillicrap, T., Riedmiller, M., & Silver, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.05182 · Cited by 113
Hierarchical Reinforcement Learning Based on Subgoal Discovery and Subpolicy Specialization
Bakker, B., & Schmidhuber, J. (2004) · Cited by 111
SemiACO: A semi-supervised feature selection based on ant colony optimization
Karimi, F., Dowlatshahi, M. B., & Hashemi, A. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.119130 · Cited by 110
Multi-Objective Molecule Generation using Interpretable SubstructuresPreprint
Jin, W., Barzilay, R., & Jaakkola, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.03244 · Cited by 134
A survey on interpretable reinforcement learning
Glanois, C., Weng, P., Zimmer, M., Li, D., Yang, T., Hao, J., & Liu, W. (2024) · Machine Learning · doi:10.1007/s10994-024-06543-w · Cited by 176
Neural Approaches to Conversational AI
Gao, J., Galley, M., & Li, L. (2019) · now publishers, Inc. eBooks · doi:10.1561/9781680835533 · Cited by 125
Mattar, M. G., & Lengyel, M. (2022) · Neuron · doi:10.1016/j.neuron.2021.12.018 · Cited by 120
Planning with Markov Decision Processes
Mausam, M., & Kolobov, A. (2012) · Synthesis lectures on artificial intelligence and machine learning · doi:10.2200/s00426ed1v01y201206aim017 · Cited by 173
Context-Aware Symptom Checking for Disease Diagnosis Using Hierarchical Reinforcement Learning
Kao, H.-C., Tang, K.-F., & Chang, E. Y. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11902 · Cited by 107
Learning Plannable Representations with Causal InfoGANPreprint
Kurutach, T., Tamar, A., Yang, G., Russell, S. J., & Abbeel, P. (2018) · Neural Information Processing Systems · doi:10.48550/arxiv.1807.09341 · Cited by 202
Programmable Reinforcement Learning Agents
André, D., & Russell, S. (2000) · Cited by 106
Gradient Temporal-Difference Learning Algorithms
Maei, H. R. (2011) · ERA: Education and Research Archive (University of Alberta) · doi:10.7939/r30q50 · Cited by 119
Intrinsically Motivated Hierarchical Skill Learning in Structured Environments
Vigorito, C. M., & Barto, A. G. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2050205 · Cited by 107
State abstraction discovery from irrelevant state variables
Jong, N. K., & Stone, P. (2005) · Cited by 105
Automatic shaping and decomposition of reward functions
Marthi, B. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273572 · Cited by 105
Real-world ride-hailing vehicle repositioning using deep reinforcement learning
Jiao, Y., Tang, X., Qin, Z. T., Li, S., Zhang, F., Hongtu Zhu, B., & Ye, J. (2021) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2021.103289 · Cited by 104
Han, C., Huo, L., Tong, X., Wang, H., & Liu, X. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2982672 · Cited by 105
DreamCoder: bootstrapping inductive program synthesis with wake-sleep library learning
Ellis, K., Wong, C., Nye, M., Sablé-Meyer, M., Morales, L., Hewitt, L., Cary, L., Solar-Lezama, A., & Tenenbaum, J. B. (2021) · Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation · doi:10.1145/3453483.3454080 · Cited by 103
RODE: Learning Roles to Decompose Multi-Agent TasksPreprint
Wang, T., Gupta, T., Mahajan, A., Peng, B., Whiteson, S., & Zhang, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01523 · Cited by 170
Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Clutter
Danielczuk, M., Kurenkov, A., Balakrishna, A., Matl, M., Wang, D., Martín-Martín, R., Garg, A., Savarese, S., & Goldberg, K. Y. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794143 · Cited by 102
System Design Perspective for Human-Level Agents Using Deep Reinforcement Learning: A Survey
Nguyen, N. D., Nguyen, T., & Nahavandi, S. (2017) · IEEE Access · doi:10.1109/access.2017.2777827 · Cited by 100
Learning to Navigate Through Complex Dynamic Environment With Modular Deep Reinforcement Learning
Wang, Y., He, H., & Sun, C. (2018) · IEEE Transactions on Games · doi:10.1109/tg.2018.2849942 · Cited by 100
How to set the switches on this thing
Dayan, P. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.05.011 · Cited by 107
Autonomously constructing hierarchical task networks for planning and human-robot collaboration
Hayes, B., & Scassellati, B. M. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487760 · Cited by 100
Metadata
- Authors
- Sutton, R. S., Precup, D., & Singh, S.
- Affiliations
- AT&T (United States) (US)
- University of Massachusetts Amherst (US)
- Corresponding author
- Sutton, Richard S.
- Venue
- Artificial Intelligence
- Publisher
- Elsevier BV
- Published
- 1999-08
- Volume
- 112
- Issue
- 1-2
- Pages
- 181-211
- ISSN
- 0004-3702 (ISSN-L)
- OpenAlex
- W2109910161
- Cited by
- 3,119
- References
- 75
- Citations by year
- 2026: 179 · 2025: 206 · 2024: 188 · 2023: 212 · 2022: 213 · 2021: 270 · 2020: 303 · 2019: 238 · 2018: 183 · 2017: 112 · 2016: 99 · 2015: 80 · 2014: 74 · 2013: 91 · 2012: 86
- FWCI
- 27.32
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Artificial Intelligence (publishedVersion)
- Funders
- National Science Foundation (9711753, 9511805)
- Air Force Office of Scientific Research (F49620)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- temporal abstraction, options framework, semi-Markov decision process, Markov decision process, hierarchical reinforcement learning, Q-learning, dynamic programming
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
