The DOI matches its Crossref record; 74 of the 75 references it cites were found in a registry or an index.
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In robotics, skills are often modelled using the Option framework [78].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [78]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Axten, N., Newell, A., & Simon, H. A. (1973) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2063712 · Cited by 6,886
Newell, A. (1972) · Cited by 6,222
Dynamic Programming and Markov Processes.
Freimer, M. L., & Howard, R. A. (1961) · Journal of the American Statistical Association · doi:10.2307/2282268 · Cited by 3,388
A qualitative physics based on confluences
de Kleer, J., & Brown, J. S. (1984) · Artificial Intelligence · doi:10.1016/0004-3702(84)90037-7 · Cited by 1,510
Planning in a hierarchy of abstraction spaces
Sacerdoti, E. D. (1974) · Artificial Intelligence · doi:10.1016/0004-3702(74)90026-5 · Cited by 1,185
Learning to act using real-time dynamic programming
Barto, A. G., Bradtke, S. J., & Singh, S. P. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(94)00011-o · Cited by 1,129
Learning and executing generalized robot plans
Fikes, R. E., Hart, P. E., & Nilsson, N. J. (1972) · Artificial Intelligence · doi:10.1016/0004-3702(72)90051-3 · Cited by 1,025
Reinforcement learning for robots using neural networks
Lin, L.-J. (1992) · Defense Technical Information Center (DTIC) · Cited by 841
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
Jaakkola, T., Jordan, M. I., & Singh, S. P. (1994) · Neural Computation · doi:10.1162/neco.1994.6.6.1185 · Cited by 801
Research works printed but not linked: 8
- E.G. Araujo, R.A. Grupen, Learning control composition in a complex environment, in: Proc. 4th International Conference on Simulation of Adaptive Behavior, 1996, pp. 333-342.
- S. Mahadevan, N. Marchalleck, T. Das, A. Gosavi, Self-improving factory simulation using continuous-time average-reward reinforcement learning, in: Proc. 14th International Conference on Machine Learning, 1997, pp. 202-210.
- D. Precup, R.S. Sutton, Multi-time models for reinforcement learning, in: Proc. ICML'97 Workshop on Modeling in Reinforcement Learning, 1997.
- D. Precup, R.S. Sutton, S.P. Singh, Planning with closed-loop macro actions, in: Working Notes 1997 AAAI Fall Symposium on Model-directed Autonomous Systems, 1997, pp. 70-76.
- S. Sastry, Algorithms for design of hybrid systems, in: Proc. International Conference of Information Sciences, 1997.
- S.P. Singh, Scaling reinforcement learning by learning variable temporal resolution models, in: Proc. 9th International Conference on Machine Learning, Morgan Kaufmann, San Mateo, CA, 1992, pp. 406-415.
- R.S. Sutton, B. Pinette, The learning of world models by connectionist networks, in: Proc. 7th Annual Conference of the Cognitive Science Society, 1985, pp. 54-64.
- C.J.C.H. Watkins, Learning with delayed rewards, Ph.D. Thesis, Cambridge University, 1989.
Cited by
Sorted by citations
Items 501–600 of 1,012
Pac-Man Conquers Academia: Two Decades of Research Using a Classic Arcade Game
Rohlfshagen, P., Liu, J., Pérez-Liébana, D., & Lucas, S. M. (2017) · IEEE Transactions on Games · doi:10.1109/tg.2017.2737145 · Cited by 41
Animal-in-the-Loop: Using Interactive Robotic Conspecifics to Study Social Behavior in Animal Groups
Landgraf, T., Gebhardt, G. H. W., Bierbach, D., Romańczuk, P., Musiolek, L. M., Hafner, V. V., & Krause, J. (2020) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-061920-103228 · Cited by 42
End-to-End Hierarchical Reinforcement Learning With Integrated Subgoal Discovery
Pateria, S., Subagdja, B., Tan, A., & Quek, C. H. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3087733 · Cited by 41
Hierarchical Decision Theoretic Planning for Navigation Among Movable Obstacles
Levihn, M., Scholz, J., & Stilman, M. (2013) · Springer tracts in advanced robotics · doi:10.1007/978-3-642-36279-8_2 · Cited by 41
Robot learning simultaneously a task and how to interpret human instructions
Grizou, J., Lopes, M. C., & Oudeyer, P. (2013) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2013.6652523 · Cited by 41
A causal approach to hierarchical decomposition of factored MDPs
Jönsson, A., & Barto, A. G. (2005) · International Conference on Machine Learning · doi:10.1145/1102351.1102402 · Cited by 41
Hierarchical reinforcement learning for spoken dialogue systems
Cuayáhuitl, H. (2009) · Lincoln Repository (University of Lincoln) · doi:10.24385/lincoln.24325480 · Cited by 41
Hindsight policy gradientsPreprint
Rauber, P., Ummadisingu, A., Mutz, F. W., & Schmidhuber, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.06006 · Cited by 41
DreamCoder: growing generalizable, interpretable knowledge with wake–sleep Bayesian program learning
Ellis, K., Wong, L., Nye, M., Sablé-Meyer, M., Cary, L., Anaya Pozo, L., Hewitt, L., Solar-Lezama, A., & Tenenbaum, J. B. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0050 · Cited by 125
Evolutionary Development of Hierarchical Learning Structures
Elfwing, S., Uchibe, E., Doya, K., & Christensen, H. I. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2006.890270 · Cited by 40
Hierarchical Approximate Policy Iteration With Binary-Tree State Space Decomposition
Xu, X., Liu, C., Yang, S. X., & Hu, D. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2168422 · Cited by 40
The utility of temporal abstraction in reinforcement learning
Jong, N. K., Hester, T., & Stone, P. (2008) · doi:10.5555/1402383.1402429 · Cited by 40
Dialogue Systems for Intelligent Human Computer Interactions
Merdivan, E., Singh, D., Hanke, S., & Holzinger, A. (2019) · Electronic Notes in Theoretical Computer Science · doi:10.1016/j.entcs.2019.04.010 · Cited by 39
Humans can navigate complex graph structures acquired during latent learning
Rmus, M., Ritz, H., Hunter, L. E., Bornstein, A. M., & Shenhav, A. (2019) · Cognition · doi:10.1016/j.cognition.2022.105103 · Cited by 42
Review of Deep Reinforcement Learning
Yu, K., Jin, K., & Deng, X. (2022) · IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC) · doi:10.1109/imcec55388.2022.10020015 · Cited by 40
Compositional Planning Using Optimal Option ModelsPreprint
Silver, D., & Ciosek, K. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6473 · Cited by 40
Hierarchical Reinforcement Learning By Discovering Intrinsic OptionsPreprint
Zhang, J. M., Yu, H., & Xu, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.06521 · Cited by 40
Skill Induction and Planning with Latent Language
Sharma, P., Torralba, A., & Andreas, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.120 · Cited by 39
Kwan, W.-C., Wang, H., Wang, H., & Wong, K. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1347-y · Cited by 48
Najafi, S., & Lee, C.-G. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2023.109179 · Cited by 39
Monte Carlo Value Iteration with Macro-Actions
Lim, Z. W., Sun, L. M., & Hsu, D. (2011) · National University of Singapore · Cited by 39
Learning Highway Ramp Merging Via Reinforcement Learning with Temporally-Extended Actions
Triest, S., Villaflor, A., & Dolan, J. M. (2020) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv47402.2020.9304841 · Cited by 39
Cao, X. (2007) · Springer eBooks · doi:10.1007/978-0-387-69082-7_6 · Cited by 39
Hierarchical Reinforcement Learning
(2012) · SpringerReference · doi:10.1007/springerreference_179075 · Cited by 39
Machine learning meets advanced robotic manipulation
Nahavandi, S., Alizadehsani, R., Nahavandi, D., Lim, C. P., Kelly, K., & Bello, F. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102221 · Cited by 46
Learning sequential motor tasks
Daniel, C., Neumann, G. M., Kroemer, O., & Peters, J. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6630937 · Cited by 38
Teaching a Machine to Read Maps With Deep Reinforcement Learning
Brunner, G., Richter, O., Wang, Y., & Wattenhofer, R. P. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11645 · Cited by 38
Realigning Models of Habitual and Goal-Directed Decision-Making
Miller, K. J., Ludvig, E. A., Pezzulo, G., & Shenhav, A. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-812098-9.00018-8 · Cited by 46
Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog
Zhang, J., Zhao, T., & Zhou, Y. (2018) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w18-5015 · Cited by 38
Learning to Coordinate Manipulation Skills via Skill Behavior Diversification
Lee, Y., Yang, J., & Lim, J. J. (2020) · International Conference on Learning Representations · Cited by 39
Gaskett, C. (2002) · PubMed Central · doi:10.25911/5d7a2a09a7dfd · Cited by 38
Prediction and Control with Temporal Segment ModelsPreprint
Mishra, N., Abbeel, P., & Mordatch, I. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04070 · Cited by 38
Independently Controllable FeaturesPreprint
Bengio, E., Thomas, V., Pineau, J., Precup, D., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07718 · Cited by 38
Composable Planning with AttributesPreprint
Zhang, A., Lerer, A. K., Sukhbaatar, S., Fergus, R., & Szlam, A. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.00512 · Cited by 44
Logically-Constrained Reinforcement LearningPreprint
Hasanbeig, M., Abate, A., & Kroening, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.08099 · Cited by 116
Singh, C. H., Mishra, V., Jain, K. K., & Shukla, A. K. (2022) · Drones · doi:10.3390/drones6120406 · Cited by 38
Competence progress intrinsic motivation
Stout, A., & Barto, A. G. (2010) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2010.5578835 · Cited by 37
Learning complex motions by sequencing simpler motion templates
Neumann, G. M., Maass, W., & Peters, J. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553471 · Cited by 37
Graph based skill acquisition and transfer Learning for continuous reinforcement learning domains
Shoeleh, F., & Asadpour, M. (2016) · Pattern Recognition Letters · doi:10.1016/j.patrec.2016.08.009 · Cited by 37
Markovian state and action abstractions for MDPS via hierarchical MCTS
Bai, A., Srivastava, S., & Russell, S. (2016) · International Joint Conference on Artificial Intelligence · Cited by 37
Hierarchical Reinforcement Learning for Open-Domain Dialog
Saleh, A., Jaques, N., Ghandeharioun, A., Shen, J. H., & Picard, R. W. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6400 · Cited by 16
Efficient Deep Reinforcement Learning via Adaptive Policy Transfer
Yang, T., Hao, J., Meng, Z., Zhang, Z., Hu, Y., Chen, Y., Fan, C., Wang, W., Liu, W., Wang, Z., et al. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/428 · Cited by 37
Learning to Locomote with Artificial Neural-Network and CPG-based Control in a Soft Snake Robot
Liu, X., Gasoto, R., Jiang, Z., Onal, C. D., & Fu, J. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340763 · Cited by 37
Learning domain structure through probabilistic policy reuse in reinforcement learning
Fernández, F., & Veloso, M. (2012) · Progress in Artificial Intelligence · doi:10.1007/s13748-012-0026-6 · Cited by 37
Such stuff as habits are made on: A reply to Cooper and Shallice (2006).
Botvinick, M., & Plaut, D. C. (2006) · Psychological Review · doi:10.1037/0033-295x.113.4.917 · Cited by 38
A Formal Valuation Framework for Emotions and Their Control
Huys, Q. J. M., & Renz, D. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.07.003 · Cited by 39
Towards a Life-Long Learning Soccer Agent
Kleiner, A., Dietl, M., & Nebel, B. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-45135-8_10 · Cited by 37
Time-Agnostic Prediction: Predicting Predictable Video FramesPreprint
Jayaraman, D., Ebert, F., Efros, A. A., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07784 · Cited by 37
An Open-Source Framework for Adaptive Traffic Signal ControlPreprint
Genders, W., & Razavi, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.00395 · Cited by 52
Decision-Theoretic Control of Planetary Rovers
Shlomo Zilberstein, B., Washington, R. M., Bernstein, D. S., & Mouaddib, A. (2002) · Lecture notes in computer science · doi:10.1007/3-540-37724-7_16 · Cited by 36
Mirolli, M., Santucci, V. G., & Baldassarre, G. (2013) · Neural Networks · doi:10.1016/j.neunet.2012.12.012 · Cited by 40
How can we define intrinsic motivation
Oudeyer, P., & Kaplan, F. (2008) · HAL · Cited by 36
Lu, H., Lu, C., Yu, Y., Xiong, G., & Gong, J. (2022) · Automotive Innovation · doi:10.1007/s42154-022-00177-1 · Cited by 36
Learning concurrent motor skills in versatile solution spaces
Daniel, C., Neumann, G., & Peters, J. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386047 · Cited by 44
Why we should talk about option generation in decision-making research
Kalis, A., Kaiser, S., & Mojzisch, A. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00555 · Cited by 38
Optimizing Expectations: From Deep Reinforcement Learning to Stochastic Computation Graphs
Schulman, J. (2016) · eScholarship (California Digital Library) · Cited by 36
CompILE: Compositional Imitation Learning and ExecutionPreprint
Kipf, T., Li, Y., Dai, H., Zambaldi, V., Sánchez‐González, Á., Grefenstette, E., Kohli, P., & Battaglia, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.01483 · Cited by 46
Structured World Models from Human Videos
Mendonca, R., Bahl, S., & Pathak, D. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.012 · Cited by 107
Elastic step DQN: A novel multi-step algorithm to alleviate overestimation in Deep Q-Networks
Ly, A., Dazeley, R., Vamplew, P. W., Cruz, F., & Aryal, S. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127170 · Cited by 35
Online Planning for Large Markov Decision Processes with Hierarchical Decomposition
Bai, A., Wu, F., & Chen, X. (2015) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2717316 · Cited by 35
Transfer Learning for User Adaptation in Spoken Dialogue Systems
Genevay, A., & Laroche, R. (2016) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/2936924.2937067 · Cited by 35
MacAlpine, P., & Stone, P. (2017) · Artificial Intelligence · doi:10.1016/j.artint.2017.09.001 · Cited by 35
State Abstraction as Compression in Apprenticeship Learning
Abel, D., Arumugam, D., Asadi, K., Jinnai, Y., Littman, M. L., & Wong, L. L. S. (2019) · AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013134 · Cited by 66
The role of information structures in game-theoretic multi-agent learning
Li, T., Zhao, Y., & Zhu, Q. (2022) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2022.03.003 · Cited by 40
CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement LearningPreprint
Yang, J., Nakhaei, A., Isele, D., Fujimura, K., & Hongyuan Zha, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.05188 · Cited by 54
Language and culture internalization for human-like autotelic AI
Colas, C., Karch, T., Moulin-Frier, C., & Oudeyer, P.-Y. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00591-4 · Cited by 34
Motivational engine with autonomous sub-goal identification for the Multilevel Darwinist Brain
Salgado, R., Prieto, A., Bellas, F., Calvo-Varela, L., & Duro, R. J. (2016) · Biologically Inspired Cognitive Architectures · doi:10.1016/j.bica.2016.07.003 · Cited by 34
Neural circuits and symbolic processing
Do, Q. H., & Hasselmo, M. E. (2021) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2021.107552 · Cited by 38
Tiwari, A., Saha, S., & Bhattacharyya, P. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108292 · Cited by 34
Yue, L., Yang, R., Zuo, J., Zhang, Y., Li, Q., & Zhang, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3202938 · Cited by 35
Lu, C., Lu, H., Chen, D., Wang, H., Li, P., & Gong, J. (2023) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2023.104328 · Cited by 36
Zhu, L., Cheng, J., Zhang, H., Zhang, W., & Liu, Y. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3333409 · Cited by 34
Eppe, M., Nguyen, P. D. H., & Wermter, S. (2019) · Frontiers in Robotics and AI · doi:10.3389/frobt.2019.00123 · Cited by 37
Sensorimotor Contingencies as a Key Drive of Development: From Babies to Robots
Jacquey, L., Baldassarre, G., Santucci, V. G., & O'Regan, J. K. (2019) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2019.00098 · Cited by 33
Temporal and state abstractions for efficient learning, transfer, and composition in humans.
Xia, L., & Collins, A. G. E. (2021) · Psychological Review · doi:10.1037/rev0000295 · Cited by 33
Autonomously Learning an Action Hierarchy Using a Learned Qualitative State Representation
Mugan, J., & Kuipers, B. J. (2018) · Figshare · doi:10.1184/r1/6621239 · Cited by 34
Customizing directions in an automated wayfinding system for individuals with cognitive impairment
Liu, A. L., Hile, H., Borriello, G., Brown, P. A., Harniss, M. K., Kautz, H., & Johnson, K. L. (2009) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1639642.1639649 · Cited by 34
Decision-Theoretic Planning with Concurrent Temporally Extended ActionsPreprint
Rohanimanesh, K., & Mahadevan, S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.2307 · Cited by 34
Reinforcement Learning Algorithms for MDPs
Szepesvári, C. (2011) · Wiley Encyclopedia of Operations Research and Management Science · doi:10.1002/9780470400531.eorms0714 · Cited by 34
Routing Networks and the Challenges of Modular and Compositional ComputationPreprint
Rosenbaum, C., Cases, I., Riemer, M., & Klinger, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.12774 · Cited by 90
Learning Multi-Agent Coordination for Enhancing Target Coverage in Directional Sensor NetworksPreprint
Xu, J., Zhong, F., & Wang, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13110 · Cited by 37
Enhancing multi-UAV air combat decision making via hierarchical reinforcement learning
Wang, H., & Wang, J. (2024) · Scientific Reports · doi:10.1038/s41598-024-54938-5 · Cited by 33
Approximate Value Iteration with Temporally Extended Actions
Mann, T., Mannor, S., & Precup, D. (2015) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4676 · Cited by 33
Interactive Semantic Parsing for If-Then Recipes via Hierarchical Reinforcement Learning
Yao, Z., Li, X., Gao, J., Sadler, B. M., & Sun, H. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33012547 · Cited by 35
Zhao, D., Zhang, L., Zhang, B., Zheng, L., Bao, Y., & Yan, W. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401170 · Cited by 32
A Survey of Planning and Learning in Games
Duarte, F. F., Lau, N., Pereira, A., & Reis, L. P. (2020) · Applied Sciences · doi:10.3390/app10134529 · Cited by 35
Event‐Predictive Cognition: A Root for Conceptual Human Thought
Butz, M. V., Achimova, A., Bilkey, D. K., & Knott, A. (2020) · Topics in Cognitive Science · doi:10.1111/tops.12522 · Cited by 40
Cheng, J., Wu, Y., Lin, Y., Yuepeng, E., Tang, F., & Ge, J. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3120297 · Cited by 33
Adjacency Constraint for Efficient Hierarchical Reinforcement Learning
Zhang, T., Guo, S., Tan, T., Hu, X., & Chen, F. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3192418 · Cited by 33
AI Reasoning Methods for Robotics
Beetz, M., Chatila, R., Hertzberg, J., & Pecora, F. (2016) · Springer handbooks · doi:10.1007/978-3-319-32552-1_14 · Cited by 33
Ray Interference: a Source of Plateaus in Deep Reinforcement LearningPreprint
Schaul, T., Borsa, D., Modayil, J., & Pascanu, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.11455 · Cited by 33
Hierarchical Deep Multiagent Reinforcement Learning with Temporal AbstractionPreprint
Tang, H., Hao, J., Lv, T., Chen, Y., Zhang, Z., Jia, H., Ren, C., Zheng, Y., Meng, Z., Fan, C., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.09332 · Cited by 52
HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile ManipulatorsPreprint
Li, C., Xia, F., Martín-Martín, R., & Savarese, S. (2019) · Conference on Robot Learning · doi:10.48550/arxiv.1910.11432 · Cited by 130
Strategy selection as rational metareasoningPreprint
Lieder, F., & Griffiths, T. L. (2016) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/6yugk · Cited by 48
Cooperative Learning for Smart Charging of Shared Autonomous Vehicle Fleets
Ahadi, R., Ketter, W., Collins, J. E., & Daina, N. (2022) · Transportation Science · doi:10.1287/trsc.2022.1187 · Cited by 32
Li, Y., Pan, B., Chen, Z., & Lu, X. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3287269 · Cited by 32
Policy Search for Multi-Robot Coordination under Uncertainty
Amato, C., Konidaris, G., Anders, A. S., Cruz, G., How, J. P., & Kaelbling, L. P. (2015) · Robotics: Science and Systems · doi:10.15607/rss.2015.xi.007 · Cited by 32
ECA: An enactivist cognitive architecture based on sensorimotor modeling
Georgeon, O. L., Marshall, J. B., & Manzotti, R. (2013) · Biologically Inspired Cognitive Architectures · doi:10.1016/j.bica.2013.05.006 · Cited by 32
Playing FPS Games With Environment-Aware Hierarchical Reinforcement Learning
Song, S., Weng, J., Su, H., Yan, D., Zou, H., & Zhu, J. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/482 · Cited by 33
Hierarchical multiagent reinforcement learning schemes for air traffic management
Spatharis, C., Bastas, A., Kravaris, T., Blekas, K., Vouros, G. A., & Cordero, J. M. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-05748-7 · Cited by 32
Metadata
- Authors
- Sutton, R. S., Precup, D., & Singh, S.
- Affiliations
- AT&T (United States) (US)
- University of Massachusetts Amherst (US)
- Corresponding author
- Sutton, Richard S.
- Venue
- Artificial Intelligence
- Publisher
- Elsevier BV
- Published
- 1999-08
- Volume
- 112
- Issue
- 1-2
- Pages
- 181-211
- ISSN
- 0004-3702 (ISSN-L)
- OpenAlex
- W2109910161
- Cited by
- 3,119
- References
- 75
- Citations by year
- 2026: 179 · 2025: 206 · 2024: 188 · 2023: 212 · 2022: 213 · 2021: 270 · 2020: 303 · 2019: 238 · 2018: 183 · 2017: 112 · 2016: 99 · 2015: 80 · 2014: 74 · 2013: 91 · 2012: 86
- FWCI
- 27.32
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Artificial Intelligence (publishedVersion)
- Funders
- National Science Foundation (9711753, 9511805)
- Air Force Office of Scientific Research (F49620)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- temporal abstraction, options framework, semi-Markov decision process, Markov decision process, hierarchical reinforcement learning, Q-learning, dynamic programming
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
