The DOI matches its Crossref record; 74 of the 75 references it cites were found in a registry or an index.
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In robotics, skills are often modelled using the Option framework [78].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [78]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Axten, N., Newell, A., & Simon, H. A. (1973) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2063712 · Cited by 6,886
Newell, A. (1972) · Cited by 6,222
Dynamic Programming and Markov Processes.
Freimer, M. L., & Howard, R. A. (1961) · Journal of the American Statistical Association · doi:10.2307/2282268 · Cited by 3,388
A qualitative physics based on confluences
de Kleer, J., & Brown, J. S. (1984) · Artificial Intelligence · doi:10.1016/0004-3702(84)90037-7 · Cited by 1,510
Planning in a hierarchy of abstraction spaces
Sacerdoti, E. D. (1974) · Artificial Intelligence · doi:10.1016/0004-3702(74)90026-5 · Cited by 1,185
Learning to act using real-time dynamic programming
Barto, A. G., Bradtke, S. J., & Singh, S. P. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(94)00011-o · Cited by 1,129
Learning and executing generalized robot plans
Fikes, R. E., Hart, P. E., & Nilsson, N. J. (1972) · Artificial Intelligence · doi:10.1016/0004-3702(72)90051-3 · Cited by 1,025
Reinforcement learning for robots using neural networks
Lin, L.-J. (1992) · Defense Technical Information Center (DTIC) · Cited by 841
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
Jaakkola, T., Jordan, M. I., & Singh, S. P. (1994) · Neural Computation · doi:10.1162/neco.1994.6.6.1185 · Cited by 801
Research works printed but not linked: 8
- E.G. Araujo, R.A. Grupen, Learning control composition in a complex environment, in: Proc. 4th International Conference on Simulation of Adaptive Behavior, 1996, pp. 333-342.
- S. Mahadevan, N. Marchalleck, T. Das, A. Gosavi, Self-improving factory simulation using continuous-time average-reward reinforcement learning, in: Proc. 14th International Conference on Machine Learning, 1997, pp. 202-210.
- D. Precup, R.S. Sutton, Multi-time models for reinforcement learning, in: Proc. ICML'97 Workshop on Modeling in Reinforcement Learning, 1997.
- D. Precup, R.S. Sutton, S.P. Singh, Planning with closed-loop macro actions, in: Working Notes 1997 AAAI Fall Symposium on Model-directed Autonomous Systems, 1997, pp. 70-76.
- S. Sastry, Algorithms for design of hybrid systems, in: Proc. International Conference of Information Sciences, 1997.
- S.P. Singh, Scaling reinforcement learning by learning variable temporal resolution models, in: Proc. 9th International Conference on Machine Learning, Morgan Kaufmann, San Mateo, CA, 1992, pp. 406-415.
- R.S. Sutton, B. Pinette, The learning of world models by connectionist networks, in: Proc. 7th Annual Conference of the Cognitive Science Society, 1985, pp. 54-64.
- C.J.C.H. Watkins, Learning with delayed rewards, Ph.D. Thesis, Cambridge University, 1989.
Cited by
Sorted by citations
Items 401–500 of 1,012
Rationality and Intelligence: A Brief Update
Russell, S. (2016) · Synthese Library/Synthese library · doi:10.1007/978-3-319-26485-1_2 · Cited by 61
Hierarchical control of goal-directed action in the cortical–basal ganglia network
Balleine, B. W., Dezfouli, A., Ito, M., & Doya, K. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.06.001 · Cited by 51
Modeling task allocation using a decision theoretic model
Abdallah, S., & Lesser, V. (2005) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/1082473.1082583 · Cited by 52
InfoBot: Transfer and Exploration via the Information BottleneckPreprint
Goyal, A., Islam, R., Strouse, D., Ahmed, Z., Botvinick, M., Larochelle, H., Bengio, Y., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.10902 · Cited by 98
Human-Robot Teaming: Grand Challenges
Natarajan, M., Seraj, E., Altundas, B., Paleja, R., Ye, S., Chen, L., Jensen, R., Chang, K. C., & Gombolay, M. (2023) · Current Robotics Reports · doi:10.1007/s43154-023-00103-1 · Cited by 58
Goal-seeking compresses neural codes for space in the human hippocampus and orbitofrontal cortex
Muhle-Karbe, P. S., Sheahan, H., Pezzulo, G., Spiers, H. J., Chien, S., Schuck, N. W., & Summerfield, C. (2023) · Neuron · doi:10.1016/j.neuron.2023.08.021 · Cited by 54
Li, Y., Zhang, Q., Yao, H., Gao, R., Xin, X., & Yu, F. R. (2024) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2024.3365878 · Cited by 51
Path planning of stratospheric airship in dynamic wind field based on deep reinforcement learning
Zheng, B., Zhu, M., Guo, X., Ou, J., & Yuan, J. (2024) · Aerospace Science and Technology · doi:10.1016/j.ast.2024.109173 · Cited by 55
Schemas, reinforcement learning and the medial prefrontal cortex
Bein, O., & Niv, Y. (2025) · Nature reviews. Neuroscience · doi:10.1038/s41583-024-00893-z · Cited by 77
A neural model of hierarchical reinforcement learning
Rasmussen, D., Voelker, A., & Eliasmith, C. (2017) · PLoS ONE · doi:10.1371/journal.pone.0180234 · Cited by 51
Toward a Unified Sub-symbolic Computational Theory of Cognition
Butz, M. V. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00925 · Cited by 58
Strategic Attentive Writer for Learning Macro-ActionsPreprint
Vezhnevets, A. S., Vezhnevets, Mnih, V., Agapiou, J., Osindero, S., Graves, A., Vinyals, O., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04695 · Cited by 111
Learning Representations in Model-Free Hierarchical Reinforcement Learning
Rafati, J., & Noelle, D. C. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.330110009 · Cited by 12
Abed-alguni, B. H., Chalup, S. K., Henskens, F. A., & Paúl, D. (2015) · Vietnam Journal of Computer Science · doi:10.1007/s40595-015-0045-x · Cited by 51
Qiu, D., Wang, Y., Sun, M., & Štrbac, G. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.118790 · Cited by 53
Bottom-Up Skill Discovery From Unsegmented Demonstrations for Long-Horizon Robot Manipulation
Zhu, Y., Stone, P., & Zhu, Y. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3146589 · Cited by 50
Abstraction and Generalization in Reinforcement Learning: A Summary and Framework
Ponsen, M., Taylor, M. E., & Tuyls, K. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-11814-2_1 · Cited by 50
Policy search for multi-robot coordination under uncertainty
Amato, C., Konidaris, G., Anders, A. S., Cruz, G., How, J. P., & Kaelbling, L. P. (2016) · The International Journal of Robotics Research · doi:10.1177/0278364916679611 · Cited by 52
Sub-domain Modelling for Dialogue Management with Hierarchical Reinforcement Learning
Budzianowski, P., Ultes, S., Su, P.-H., Mrkšić, N., Wen, T.-H., Casanueva, I., Rojas-Barahona, L. M., & Gašić, M. (2017) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w17-5512 · Cited by 50
Decentralized Cooperative Planning for Automated Vehicles with Hierarchical Monte Carlo Tree Search
Kurzer, K., Zhou, C., & Zöllner, J. M. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500712 · Cited by 52
Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discovery
Yang, J., Borovikov, I., & Hongyuan Zha, B. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/fmju3241 · Cited by 72
A Roadmap to the Last Decade of Learning Classifier System Research
Lanzi, P. L., & Riolo, R. (2000) · Virtual Community of Pathological Anatomy (University of Castilla La Mancha) · Cited by 50
Anticipations Control Behavior: Animal Behavior in an Anticipatory Learning Classifier System
Butz, M. V., & Hoffmann, J. (2002) · Adaptive Behavior · doi:10.1177/1059712302010002001 · Cited by 50
Schmidhuber, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.09249 · Cited by 120
A Benchmarking Environment for Reinforcement Learning Based Task Oriented Dialogue ManagementPreprint
Casanueva, I., Budzianowski, P., Su, P.-H., Mrkšić, N., Wen, T.-H., Ultes, S., Rojas-Barahona, L. M., Young, S. J., & Gašić, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.11023 · Cited by 57
TACO: Learning Task Decomposition via Temporal Alignment for ControlPreprint
Shiarlis, K., Wulfmeier, M., Salter, S., Whiteson, S., & Posner, I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.01840 · Cited by 50
From Machine Learning to Robotics: Challenges and Opportunities for Embodied IntelligencePreprint
Roy, N., Posner, I., Barfoot, T., Beaudoin, P., Bengio, Y., Bohg, J., Brock, O., Depatie, I., Fox, D., Koditschek, D. E., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.15245 · Cited by 116
Hierarchical and Stable Multiagent Reinforcement Learning for Cooperative Navigation Control
Jin, Y., Wei, S., Yuan, J., & Zhang, X. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3089834 · Cited by 49
Evolution and Learning in an Intrinsically Motivated Reinforcement Learning Robot
Schembri, M., Mirolli, M., & Baldassarre, G. (2007) · Springer eBooks · doi:10.1007/978-3-540-74913-4_30 · Cited by 49
Subgoal Discovery for Hierarchical Reinforcement Learning Using Learned Policies
Goel, S., & Huber, M. (2003) · The Florida AI Research Society · Cited by 49
Agha–mohammadi, A., Chakravorty, S., & Amato, N. M. (2011) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2011.6095010 · Cited by 49
A Hierarchical Reinforced Sequence Operation Method for Unsupervised Text Style Transfer
Wu, C., Ren, X., Luo, F., & Sun, X. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1482 · Cited by 49
Q-learning embedded sine cosine algorithm (QLESCA)
Hamad, Q. S., Samma, H., Suandi, S. A., & Mohamad–Saleh, J. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116417 · Cited by 48
Algorithms and applications of intelligent swarm cooperative control: A comprehensive survey
Xu, X., Yan, X., Yang, W.-Y., An, K., Huang, W., & Wang, Y. (2022) · Progress in Aerospace Sciences · doi:10.1016/j.paerosci.2022.100869 · Cited by 52
Discovering task constraints through observation and active learning
Hayes, B., & Scassellati, B. M. (2014) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2014.6943191 · Cited by 48
Constrained reinforcement learning from intrinsic and extrinsic rewards
Uchibe, E., & Doya, K. (2007) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2007.4354030 · Cited by 48
Planning with Abstract Markov Decision Processes
Gopalan, N., desJardins, M., Littman, M. L., MacGlashan, J., Squire, S., Tellex, S., Winder, J., & Wong, L. L. S. (2017) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v27i1.13867 · Cited by 48
Human Behavior Modeling with Maximum Entropy Inverse Optimal Control
Ziebart, B. D., Maas, A. L., Bagnell, J. A., & Dey, A. K. (2009) · Cited by 48
Decoupling Dynamics and Reward for Transfer LearningPreprint
Zhang, A., Satija, H., & Pineau, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.10689 · Cited by 73
Interactive activity recognition and prompting to assist people with cognitive disabilities
Chu, Y., Chol Song, Y., Levinson, R., & Kautz, H. (2012) · Journal of Ambient Intelligence and Smart Environments · doi:10.3233/ais-2012-0168 · Cited by 47
Hierarchical multi‐agent reinforcement learning for multi‐aircraft close‐range air combat
Kong, W., Zhou, D., Du, Y., Zhou, Y., & Zhao, Y. (2022) · IET Control Theory and Applications · doi:10.1049/cth2.12413 · Cited by 47
Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors
Wang, L., Liu, J., Shao, H., Wang, W., Chen, R., Liu, Y., & Waslander, S. L. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.102 · Cited by 72
Relativized options: choosing the right transformation
Ravindran, B., & Barto, A. G. (2003) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 47
Application of reinforcement learning in robot soccer
Duan, Y. G., Liu, Q., & Xu, X. (2007) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2007.01.003 · Cited by 47
Bayesian Nonparametric Inverse Reinforcement Learning
Michini, B., & How, J. P. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33486-3_10 · Cited by 47
Generating Multi-Agent Trajectories using Programmatic Weak SupervisionPreprint
Zhan, E., Zheng, S., Yue, Y., Sha, L., & Lucey, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.07612 · Cited by 72
Robust Reinforcement Learning for Continuous Control with Model MisspecificationPreprint
Mankowitz, D. J., Levine, N., Jeong, R., Shi, Y., Kay, J., Abdolmaleki, A., Springenberg, J. T., Mann, T., Hester, T., & Riedmiller, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.07516 · Cited by 47
Rupprecht, T., & Wang, Y. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.05.013 · Cited by 48
Vision-Based Autonomous Driving: A Hierarchical Reinforcement Learning Approach
Wang, J., Sun, H., & Zhu, C. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3266940 · Cited by 49
A Comprehensive Review of Humanoid Robots
Sheng, Q., Zhou, Z., Li, J., Mi, X., Xiang, P., Chen, Z., Xu, H., Jia, S., Wu, X., Cui, Y., et al. (2025) · SmartBot · doi:10.1002/smb2.12008 · Cited by 57
Concurrent Reinforcement Learning from Customer Interactions
Silver, D., Newnham, L., Barker, D., Weller, S., & McFall, J. (2013) · Cited by 46
Accurately and Efficiently Interpreting Human-Robot Instructions of Varying Granularities
Arumugam, D., Karamcheti, S., Gopalan, N., Wong, L. L. S., & Tellex, S. (2017) · Robotics: Science and Systems · doi:10.15607/rss.2017.xiii.056 · Cited by 46
Compositional clustering in task structure learning
Franklin, N., & Frank, M. J. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006116 · Cited by 50
Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset
Zhang, R., Walshe, C., Liu, Z., Guan, L., Muller, K., Whritner, J. A., Zhang, L., Hayhoe, M., & Ballard, D. H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6161 · Cited by 67
Theocharous, G., & Mahadevan, S. (2003) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2002.1014730 · Cited by 49
Synthesis of hierarchical finite-state controllers for POMDPs
Hansen, E. A., & Zhou, R. (2003) · Cited by 46
Sub-policy Adaptation for Hierarchical Reinforcement LearningPreprint
Li, A. C., Florensa, C., Clavera, I., & Abbeel, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05862 · Cited by 51
LASER: Learning a Latent Action Space for Efficient Reinforcement Learning
Allshire, A., Martín-Martín, R., Lin, C., Manuel, S., Savarese, S., & Garg, A. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561232 · Cited by 45
Skill-Based Hierarchical Reinforcement Learning for Target Visual Navigation
Wang, S., Wu, Z., Hu, X., Lin, Y., & Lv, K. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3243618 · Cited by 48
Humans decompose tasks by trading off utility and computational cost
Correa, C. G., Ho, M. K., Callaway, F., Daw, N. D., & Griffiths, T. L. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011087 · Cited by 54
Projective simulation with generalization
Melnikov, A. A., Makmal, A., Dunjko, V., & Briegel, H. J. (2017) · Scientific Reports · doi:10.1038/s41598-017-14740-y · Cited by 48
Using Abstract Models of Behaviours to Automatically Generate Reinforcement Learning Hierarchies
Ryan, M. (2002) · Cited by 45
Advanced Reinforcement Learning
Daw, N. D. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00016-4 · Cited by 46
Policy and Value Transfer in Lifelong Reinforcement Learning
Abel, D. L., Jinnai, Y., Guo, Y., Konidaris, G., & Littman, M. L. (2018) · International Conference on Machine Learning · Cited by 45
Aspect Sentiment Triplet Extraction Using Reinforcement Learning
Yu, S., Nayak, T. K., Majumder, N., & Poria, S. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482058 · Cited by 48
Learning to Communicate and Act Using Hierarchical Reinforcement Learning
Ghavamzadeh, M., & Mahadevan, S. (2004) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · doi:10.1109/aamas.2004.160 · Cited by 45
Reinforcement Learning ApplicationsPreprint
Li, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.06973 · Cited by 77
Reinforcement Learning of CPG-Regulated Locomotion Controller for a Soft Snake Robot
Liu, X., Onal, C. D., & Fu, J. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3286046 · Cited by 52
Hierarchical policy gradient algorithms
Ghavamzadeh, M., & Mahadevan, S. (2003) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 44
Scaling Up Approximate Value Iteration with Options: Better Policies with Fewer Iterations
Mann, T., & Mannor, S. (2014) · Cited by 44
Learning to combine primitive skills: A step towards versatile robotic manipulation §
Strudel, R., Pashevich, A., Kalevatykh, I., Laptev, I., Šivic, J., & Schmid, C. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196619 · Cited by 51
Learning Generalizable Locomotion Skills with Hierarchical Reinforcement Learning
Li, T., Lambert, N., Calandra, R., Meier, F., & Rai, A. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196642 · Cited by 48
A Survey of Ad Hoc Teamwork Research
Mirsky, R., Carlucho, I., Rahman, A., Fosong, E., Macke, W., Sridharan, M., Stone, P., & Albrecht, S. V. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20614-6_16 · Cited by 44
Learning to Take Concurrent Actions
Rohanimanesh, K., & Mahadevan, S. (2002) · Cited by 44
Playing Doom with SLAM-Augmented Deep Reinforcement LearningPreprint
Bhatti, S., Desmaison, A., Mikšík, O., Nardelli, N., Siddharth, N., & Torr, P. H. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00380 · Cited by 48
Deep Imitation Learning for Bimanual Robotic ManipulationPreprint
Xie, F., Chowdhury, A., Kaluza, M. C. D. P., Zhao, L., Wong, L. L. S., & Yu, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.05134 · Cited by 44
Subgoal Discovery for Hierarchical Dialogue Policy Learning
Tang, D. Q., Li, X., Gao, J., Wang, C., Li, L., & Jebara, T. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1253 · Cited by 49
Wang, P., Yang, H., Han, G., Yu, R., Yang, L., Sun, G., Qi, H., Wei, X., & Zhang, Q. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3439696 · Cited by 44
A Solution to Time-Varying Markov Decision Processes
Liu, L., & Sukhatme, G. S. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2801479 · Cited by 43
Automated State Abstraction for Options using the U-Tree Algorithm
Jönsson, A., & Barto, A. G. (2000) · Cited by 43
Dynamic Web Service Composition within a Service-Oriented Architecture
Jureta, I., Faulkner, S., Achbany, Y., & Saerens, M. (2007) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws.2007.79 · Cited by 43
Singh, S. S., Tadić, V. B., & Arnaud Doucet, B. (2006) · European Journal of Operational Research · doi:10.1016/j.ejor.2006.02.023 · Cited by 43
A Roadmap to the Last Decade of Learning Classifier System Research (From 1989 to 1999)
Lanzi, P. L., & Riolo, R. (2000) · Lecture notes in computer science · doi:10.1007/3-540-45027-0_2 · Cited by 65
On the Sample Complexity of End-to-end Training vs. Semantic Abstraction TrainingPreprint
Shalev‐Shwartz, S., & Shashua, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.06915 · Cited by 56
Learning Abstract OptionsPreprint
Riemer, M., Liu, M., & Tesauro, G. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11583 · Cited by 43
Butz, M. V. (2021) · KI - Künstliche Intelligenz · doi:10.1007/s13218-021-00705-x · Cited by 42
Adaptive Zone-aware Hierarchical Planner for Vision-Language Navigation
Gao, C., Peng, X., Yan, M., Wang, H., Yang, L., Ren, H., Li, H., & Liu, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01432 · Cited by 53
Learning Relational Options for Inductive Transfer in Relational Reinforcement Learning
Croonenborghs, T., Driessens, K., & Bruynooghe, M. (2008) · Springer eBooks · doi:10.1007/978-3-540-78469-2_12 · Cited by 42
Stochastic Abstract Policies: Generalizing Knowledge to Improve Reinforcement Learning
Koga, M. L., Freire, V., & Costa, A. H. R. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2319733 · Cited by 42
Generalized Sampling-Based Motion Planners
Chakravorty, S., & Kumar, S. (2011) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2010.2098438 · Cited by 42
Discovering Multimodal Behavior in Ms. Pac-Man Through Evolution of Modular Neural Networks
Schrum, J., & Miikkulainen, R. P. (2015) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2015.2390615 · Cited by 42
Artificial Intelligence for Prosthetics: Challenge Solutions
Kidziński, Ł., Ong, C. F., Mohanty, S. P., Hicks, J. L., Carroll, S. F., Zhou, B., Zeng, H., Wang, F., Lian, R., Tian, H., et al. (2019) · The Springer series on challenges in machine learning · doi:10.1007/978-3-030-29135-8_4 · Cited by 45
Certified reinforcement learning with logic guidance
Hasanbeig, H., Kroening, D., & Abate, A. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103949 · Cited by 67
Learning movement primitives for force interaction tasks
Kober, J., Gienger, M., & Steil, J. J. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139639 · Cited by 42
Effective control knowledge transfer through learning skill and representation hierarchies
Asadi, M., & Huber, M. (2007) · Cited by 42
Adaptive Skills, Adaptive Partitions (ASAP)Preprint
Mankowitz, D. J., Mann, T., & Mannor, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.03351 · Cited by 42
Neural Approaches to Conversational AI
Gao, J., Galley, M., & Li, L. (2019) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000074 · Cited by 424
Brown, V. M., Price, R. B., & Dombrovski, A. Y. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01080-w · Cited by 51
Learning like a baby: a survey of artificial intelligence approaches
Guérin, F. (2011) · The Knowledge Engineering Review · doi:10.1017/s0269888911000038 · Cited by 41
Symbol acquisition for probabilistic high-level planning
Konidaris, G., Kaelbling, L. P., & Lozano‐Pérez, T. (2015) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 40
Metadata
- Authors
- Sutton, R. S., Precup, D., & Singh, S.
- Affiliations
- AT&T (United States) (US)
- University of Massachusetts Amherst (US)
- Corresponding author
- Sutton, Richard S.
- Venue
- Artificial Intelligence
- Publisher
- Elsevier BV
- Published
- 1999-08
- Volume
- 112
- Issue
- 1-2
- Pages
- 181-211
- ISSN
- 0004-3702 (ISSN-L)
- OpenAlex
- W2109910161
- Cited by
- 3,119
- References
- 75
- Citations by year
- 2026: 179 · 2025: 206 · 2024: 188 · 2023: 212 · 2022: 213 · 2021: 270 · 2020: 303 · 2019: 238 · 2018: 183 · 2017: 112 · 2016: 99 · 2015: 80 · 2014: 74 · 2013: 91 · 2012: 86
- FWCI
- 27.32
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Artificial Intelligence (publishedVersion)
- Funders
- National Science Foundation (9711753, 9511805)
- Air Force Office of Scientific Research (F49620)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- temporal abstraction, options framework, semi-Markov decision process, Markov decision process, hierarchical reinforcement learning, Q-learning, dynamic programming
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
