The DOI matches its Crossref record; 74 of the 75 references it cites were found in a registry or an index.
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In robotics, skills are often modelled using the Option framework [78].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [78]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Axten, N., Newell, A., & Simon, H. A. (1973) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2063712 · Cited by 6,886
Newell, A. (1972) · Cited by 6,222
Dynamic Programming and Markov Processes.
Freimer, M. L., & Howard, R. A. (1961) · Journal of the American Statistical Association · doi:10.2307/2282268 · Cited by 3,388
A qualitative physics based on confluences
de Kleer, J., & Brown, J. S. (1984) · Artificial Intelligence · doi:10.1016/0004-3702(84)90037-7 · Cited by 1,510
Planning in a hierarchy of abstraction spaces
Sacerdoti, E. D. (1974) · Artificial Intelligence · doi:10.1016/0004-3702(74)90026-5 · Cited by 1,185
Learning to act using real-time dynamic programming
Barto, A. G., Bradtke, S. J., & Singh, S. P. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(94)00011-o · Cited by 1,129
Learning and executing generalized robot plans
Fikes, R. E., Hart, P. E., & Nilsson, N. J. (1972) · Artificial Intelligence · doi:10.1016/0004-3702(72)90051-3 · Cited by 1,025
Reinforcement learning for robots using neural networks
Lin, L.-J. (1992) · Defense Technical Information Center (DTIC) · Cited by 841
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
Jaakkola, T., Jordan, M. I., & Singh, S. P. (1994) · Neural Computation · doi:10.1162/neco.1994.6.6.1185 · Cited by 801
Research works printed but not linked: 8
- E.G. Araujo, R.A. Grupen, Learning control composition in a complex environment, in: Proc. 4th International Conference on Simulation of Adaptive Behavior, 1996, pp. 333-342.
- S. Mahadevan, N. Marchalleck, T. Das, A. Gosavi, Self-improving factory simulation using continuous-time average-reward reinforcement learning, in: Proc. 14th International Conference on Machine Learning, 1997, pp. 202-210.
- D. Precup, R.S. Sutton, Multi-time models for reinforcement learning, in: Proc. ICML'97 Workshop on Modeling in Reinforcement Learning, 1997.
- D. Precup, R.S. Sutton, S.P. Singh, Planning with closed-loop macro actions, in: Working Notes 1997 AAAI Fall Symposium on Model-directed Autonomous Systems, 1997, pp. 70-76.
- S. Sastry, Algorithms for design of hybrid systems, in: Proc. International Conference of Information Sciences, 1997.
- S.P. Singh, Scaling reinforcement learning by learning variable temporal resolution models, in: Proc. 9th International Conference on Machine Learning, Morgan Kaufmann, San Mateo, CA, 1992, pp. 406-415.
- R.S. Sutton, B. Pinette, The learning of world models by connectionist networks, in: Proc. 7th Annual Conference of the Cognitive Science Society, 1985, pp. 54-64.
- C.J.C.H. Watkins, Learning with delayed rewards, Ph.D. Thesis, Cambridge University, 1989.
Cited by
Sorted by citations
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
Dynamic bayesian networks: representation, inference and learning
Murphy, K. P., & Russell, S. (2002) · Cited by 2,579
Deep reinforcement learning for autonomous driving: A survey
Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A. A. A., Yogamani, S., & Perez, P. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3054625 · Cited by 2,342
Brochu, E., Cora, V. M., & de Freitas, N. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.2599 · Cited by 2,568
Transfer Learning for Reinforcement Learning Domains: A Survey
Taylor, M. E., & Stone, P. (2009) · doi:10.5555/1577069.1755839 · Cited by 1,524
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · RePEc: Research Papers in Economics · Cited by 1,310
Age of Information: An Introduction and Survey
Yates, R. D., Sun, Y. J., Brown, D. R., Kaul, S. K., Modiano, E., & Ulukuş, Ş. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3065072 · Cited by 1,282
Intrinsic Motivation Systems for Autonomous Mental Development
Oudeyer, P.-Y., Kaplan, F., & Hafner, V. V. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2006.890271 · Cited by 1,126
Benchmarking Deep Reinforcement Learning for Continuous Control
Duan, Y., Chen, X., Houthooft, R., Schulman, J., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1604.06778 · Cited by 1,114
Algorithms for Reinforcement Learning
Szepesvári, C. (2010) · Synthesis lectures on artificial intelligence and machine learning · doi:10.1007/978-3-031-01551-9 · Cited by 1,334
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Discrete Event Dynamic Systems · doi:10.1023/a:1022140919877 · Cited by 1,059
Deep Reinforcement Learning: A Survey
Wang, X., Wang, S., Liang, X., Zhao, D., Huang, J., Xu, X., Dai, B., & Miao, Q. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3207346 · Cited by 972
Multi-agent deep reinforcement learning: a survey
Gronauer, S., & Diepold, K. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-09996-w · Cited by 957
Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic MotivationPreprint
Kulkarni, T. D., Narasimhan, K., Saeedi, A., & Tenenbaum, J. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.06057 · Cited by 909
Opponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844
Transfer Learning in Deep Reinforcement Learning: A Survey
Zhu, Z., Lin, K., Jain, A. K., & Zhou, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292075 · Cited by 861
What is intrinsic motivation? A typology of computational approaches
Oudeyer, P.-Y. (2007) · Frontiers in Neurorobotics · doi:10.3389/neuro.12.006.2007 · Cited by 811
The design and implementation of Xiaoice, an empathetic social chatbot
Zhou, L., Gao, J., Li, D., & Shum, H.-Y. (2020) · Computational Linguistics · doi:10.1162/coli_a_00368 · Cited by 777
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762
The Option-Critic Architecture
Bacon, P., Harb, J., & Precup, D. (2017) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v31i1.10916 · Cited by 779
Reinforcement learning in the brain
Niv, Y. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2008.12.005 · Cited by 775
Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective
Botvinick, M. M., Niv, Y., & Barto, A. G. (2009) · Cognition · doi:10.1016/j.cognition.2008.08.011 · Cited by 646
Universal Value Function Approximators
Schaul, T., Horgan, D., Gregor, K., & Silver, D. (2015) · Cited by 701
Reinforcement learning algorithms: A brief survey
Shakya, A. K., Pillai, G. N., & Chakrabarty, S. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120495 · Cited by 760
Human-level performance in 3D multiplayer games with population-based reinforcement learning
Jaderberg, M., Czarnecki, W. M., Dunning, I., Marris, L., Lever, G., Castañeda, A. G., Beattie, C., Rabinowitz, N. C., Morcos, A. S., Ruderman, A., et al. (2019) · Science · doi:10.1126/science.aau6249 · Cited by 662
Challenges of real-world reinforcement learning: definitions, benchmarks and analysis
Dulac-Arnold, G., Levine, N., Mankowitz, D. J., Li, J., Păduraru, C., Gowal, S., & Hester, T. (2021) · Machine Learning · doi:10.1007/s10994-021-05961-4 · Cited by 892
Intrinsically Motivated Reinforcement Learning
Singh, S. P., Barto, A. G., & Chentanez, N. (2005) · doi:10.21236/ada440280 · Cited by 614
(2004) · Cited by 610
Hierarchical models of behavior and prefrontal function
Botvinick, M. M. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.02.009 · Cited by 532
Deep learning, reinforcement learning, and world models
Matsuo, Y., LeCun, Y., Sahani, M., Precup, D., Silver, D., Sugiyama, M., Uchibe, E., & Morimoto, J. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.03.037 · Cited by 611
On a Formal Model of Safe and Scalable Self-driving CarsPreprint
Shalev‐Shwartz, S., Shammah, S., & Shashua, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.06374 · Cited by 540
Behavior Trees in Robotics and AI
Colledanchise, M., & Ögren, P. (2018) · CRC Press eBooks · doi:10.1201/9780429489105 · Cited by 534
Model-based Reinforcement Learning: A Survey
Moerland, T. M., Broekens, J., Plaat, A., & Jonker, C. M. (2023) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000086 · Cited by 871
Learning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475
Long Text Generation via Adversarial Training with Leaked Information
Guo, J., Lu, S., Cai, H., Zhang, W., Yu, Y. H., & Wang, Y.-X. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11957 · Cited by 535
Reinforcement Learning based Recommender Systems: A Survey
Afsar, M. M., Crump, T., & Far, B. H. (2022) · ACM Computing Surveys · doi:10.1145/3543846 · Cited by 637
Fast gradient-descent methods for temporal-difference learning with linear function approximation
Sutton, R. S., Maei, H. R., Precup, D., Bhatnagar, S., Silver, D., Szepesvári, C., & Wiewiora, E. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553501 · Cited by 534
Multiple model-based reinforcement learning
Doya, K., Samejima, K., Katagiri, K.-I., & Kawato, M. (2002) · Neural Computation · doi:10.1162/089976602753712972 · Cited by 476
Hierarchical Reinforcement Learning
Pateria, S., Subagdja, B., Tan, A., & Quek, C. H. (2021) · ACM Computing Surveys · doi:10.1145/3453160 · Cited by 565
Active learning of inverse models with intrinsically motivated goal exploration in robots
Baranès, A., & Oudeyer, P. (2012) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2012.05.008 · Cited by 498
Discovery of complex behaviors through contact-invariant optimization
Mordatch, I., Todorov, E. V., & Popović, Z. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185539 · Cited by 451
Eligibility Traces for Off-Policy Policy Evaluation
Precup, D., Sutton, R. S., & Singh, S. P. (2000) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 439
LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models
Song, C. H., Sadler, B. M., Wu, J., Chao, W.-L., Washington, C., & Su, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00280 · Cited by 435
Predictive Representations of State
Littman, M. L., & Sutton, R. S. (2001) · Cited by 424
Foster, D. J. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031538 · Cited by 375
Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density
McGovern, A., & Barto, A. G. (2001) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 416
Neural Approaches to Conversational AI
Gao, J., Galley, M., & Li, L. (2018) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3209978.3210183 · Cited by 409
Multi-Agent Reinforcement Learning: A Review of Challenges and Applications
Canese, L., Cardarilli, G. C., Nunzio, L. D., Fazzolari, R., Giardino, D., Re, M., & Spanò, S. (2021) · Applied Sciences · doi:10.3390/app11114948 · Cited by 454
Reinforcement Learning for RoboCup Soccer Keepaway
STONE, P. R., Sutton, R. S., & Kuhlmann, G. (2005) · Adaptive Behavior · doi:10.1177/105971230501300301 · Cited by 388
An algorithmic perspective on imitation learning
Osa, T., Pajarinen, J., Neumann, G., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · Foundations and Trends in Robotics · doi:10.1561/2300000053 · Cited by 405
Quantum Reinforcement Learning
Dong, D., Chen, C., Li, H., & Tarn, T.-J. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2008.925743 · Cited by 410
Intrinsically Motivated Learning of Hierarchical Collections of Skills
Barto, A. G., Singh, S. P., & Chentanez, N. (2004) · Cited by 376
Mobility-Aware Edge Caching and Computing in Vehicle Networks: A Deep Reinforcement Learning
Tan, L. T., & Hu, R. Q. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2867191 · Cited by 356
FOR LEAST-SQUARES POLICY ITERATION
Schwab, D. (2016) · Cited by 372
Relational Reinforcement Learning
Džeroski, S., De Raedt, L., & Driessens, K. (2001) · Machine Learning · doi:10.1023/a:1007694015589 · Cited by 371
An Algorithmic Perspective on Imitation Learning
Osa, T., Pajarinen, J., Neumann, G. M., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · now publishers, Inc. eBooks · doi:10.1561/9781680834116 · Cited by 394
Goyal, S., & Moraga‐González, J. L. (2001) · The RAND Journal of Economics · doi:10.2307/2696388 · Cited by 351
IQA: Visual Question Answering in Interactive Environments
Gordon, D., Kembhavi, A., Rastegari, M., Redmon, J., Fox, D., & Farhadi, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00430 · Cited by 377
Schaal, S., Peters, J., Nakanishi, J., & Ijspeert, A. J. (2005) · Springer tracts in advanced robotics · doi:10.1007/11008941_60 · Cited by 349
Park, J., Chun, J., Kim, S. H., Kim, Y., & Park, J. (2021) · International Journal of Production Research · doi:10.1080/00207543.2020.1870013 · Cited by 340
Reinforcement Learning: A Tutorial Survey and Recent Advances
Gosavi, A. A. (2008) · INFORMS journal on computing · doi:10.1287/ijoc.1080.0305 · Cited by 331
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Discrete Event Dynamic Systems · doi:10.1023/a:1025696116075 · Cited by 317
Deep Reinforcement Learning for Internet of Things: A Comprehensive Survey
Chen, W., Qiu, X., Cai, T., Dai, H., Zheng, Z., & Zhang, Y. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3073036 · Cited by 334
Reinforcement learning in artificial and biological systems
Neftci, E. O., & Averbeck, B. B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0025-4 · Cited by 268
Sutton, R. S., Modayil, J., Delp, M., Degris, T., Pilarski, P. M., White, A., & Precup, D. (2011) · doi:10.5555/2031678.2031726 · Cited by 304
A Deep Hierarchical Approach to Lifelong Learning in Minecraft
Tessler, C., Givony, S., Zahavy, T., Mankowitz, D. J., & Mannor, S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10744 · Cited by 300
Hierarchical motor control in mammals and machines
Merel, J., Botvinick, M., & Wayne, G. (2019) · Nature Communications · doi:10.1038/s41467-019-13239-6 · Cited by 274
Robot learning from demonstration by constructing skill trees
Konidaris, G., Kuindersma, S., Grupen, R. A., & Barto, A. G. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911428653 · Cited by 287
Towards a Unified Theory of State Abstraction for MDPs.
Li, L. C., Walsh, T. J., & Littman, M. L. (2006) · Cited by 283
Learning Options in Reinforcement Learning
Stolle, M., & Precup, D. (2002) · Lecture notes in computer science · doi:10.1007/3-540-45622-8_16 · Cited by 271
Collision avoidance for an unmanned surface vehicle using deep reinforcement learning
Woo, J., & Kim, N. (2020) · Ocean Engineering · doi:10.1016/j.oceaneng.2020.107001 · Cited by 270
Yang, Q., Zhang, Y., Dai, W., & Pan, S. J. (2020) · Cambridge University Press eBooks · doi:10.1017/9781139061773 · Cited by 286
Transfer in Reinforcement Learning: A Framework and a Survey
Lazaric, A. (2012) · Adaptation, learning, and optimization · doi:10.1007/978-3-642-27645-3_5 · Cited by 276
Cushman, F. A. (2019) · Behavioral and Brain Sciences · doi:10.1017/s0140525x19001730 · Cited by 282
A Survey on Deep Reinforcement Learning Algorithms for Robotic Manipulation
Han, D., Mulyana, B., Stankovic, V., & Cheng, S. (2023) · Sensors · doi:10.3390/s23073762 · Cited by 253
Off-Policy Temporal Difference Learning with Function Approximation
Precup, D., Sutton, R. S., & Dasgupta, S. (2001) · Cited by 243
Policy Recognition in the Abstract Hidden Markov Model
Bui, H., Venkatesh, S., & WEST, G. A. W. (2002) · Cited by 243
Li, C. (2019) · Chapman & Hall/CRC eBooks · doi:10.1201/9781351006620-6 · Cited by 257
FIRM: Sampling-based feedback motion-planning under motion uncertainty and imperfect measurements
Agha–mohammadi, A., Chakravorty, S., & Amato, N. M. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913501564 · Cited by 236
Identifying useful subgoals in reinforcement learning by local graph partitioning
Şimşek, Ö., Wolfe, A. P., & Barto, A. G. (2005) · International Conference on Machine Learning · doi:10.1145/1102351.1102454 · Cited by 238
Hierarchical reinforcement learning and decision making
Botvinick, M. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.05.008 · Cited by 230
From Skills to Symbols: Learning Symbolic Representations for Abstract High-Level Planning
Konidaris, G., Kaelbling, L. P., & Lozano‐Pérez, T. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5575 · Cited by 250
A Hierarchical Framework for Relation Extraction with Reinforcement Learning
Takanobu, R., Zhang, T., Liu, J., & Huang, M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017072 · Cited by 229
A reinforcement learning framework for optimal operation and maintenance of power grids
Rocchetta, R., Bellani, L., Compare, M., Zio, E., & Patelli, E. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.03.027 · Cited by 228
The challenges of joint attention
Kaplan, F., & Hafner, V. V. (2006) · Interaction Studies Social Behaviour and Communication in Biological and Artificial Systems · doi:10.1075/is.7.2.04kap · Cited by 227
Skill Discovery in Continuous Reinforcement Learning Domains using Skill Chaining
Konidaris, G., & Barreto, A. S. (2009) · Cited by 227
Neural Programmer-Interpreters
Reed, S., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06279 · Cited by 347
Towards Continual Reinforcement Learning: A Review and Perspectives
Khetarpal, K., Riemer, M., Rish, I., & Precup, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13673 · Cited by 224
Self-citationQ-Cut—Dynamic Discovery of Sub-goals in Reinforcement Learning
Menache, I., Mannor, S., & Shimkin, N. (2002) · Lecture notes in computer science · doi:10.1007/3-540-36755-1_25 · Cited by 228
Intrinsic Motivation and Reinforcement Learning
Barto, A. G. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_2 · Cited by 243
Building portable options: skill transfer in reinforcement learning
Konidaris, G., & Barto, A. G. (2007) · Cited by 220
Dezfouli, A., & Balleine, B. W. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003364 · Cited by 239
Meta Learning Shared HierarchiesPreprint
Frans, K., Ho, J., Chen, X., Abbeel, P., & Schulman, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.09767 · Cited by 219
Planning and navigation as active inference
Kaplan, R., & Friston, K. J. (2018) · Biological Cybernetics · doi:10.1007/s00422-018-0753-2 · Cited by 203
Interplay of approximate planning strategies
Huys, Q. J. M., Lally, N., Faulkner, P., Eshel, N., Seifritz, E., Gershman, S. J., Dayan, P., & Roiser, J. P. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1414219112 · Cited by 207
Variational Intrinsic ControlPreprint
Gregor, K., Rezende, D. J., & Wierstra, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07507 · Cited by 386
Multi-Agent Deep Reinforcement Learning for Multi-Robot Applications: A Survey
Orr, J., & Dutta, A. (2023) · Sensors · doi:10.3390/s23073625 · Cited by 244
State abstraction for programmable reinforcement learning agents
André, D., & Russell, S. (2002) · doi:10.5555/777092.777114 · Cited by 207
Hierarchical control over effortful behavior by rodent medial frontal cortex: A computational model.
Holroyd, C. B., & McClure, S. M. (2014) · Psychological Review · doi:10.1037/a0038339 · Cited by 199
Metadata
- Authors
- Sutton, R. S., Precup, D., & Singh, S.
- Affiliations
- AT&T (United States) (US)
- University of Massachusetts Amherst (US)
- Corresponding author
- Sutton, Richard S.
- Venue
- Artificial Intelligence
- Publisher
- Elsevier BV
- Published
- 1999-08
- Volume
- 112
- Issue
- 1-2
- Pages
- 181-211
- ISSN
- 0004-3702 (ISSN-L)
- OpenAlex
- W2109910161
- Cited by
- 3,119
- References
- 75
- Citations by year
- 2026: 179 · 2025: 206 · 2024: 188 · 2023: 212 · 2022: 213 · 2021: 270 · 2020: 303 · 2019: 238 · 2018: 183 · 2017: 112 · 2016: 99 · 2015: 80 · 2014: 74 · 2013: 91 · 2012: 86
- FWCI
- 27.32
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Artificial Intelligence (publishedVersion)
- Funders
- National Science Foundation (9711753, 9511805)
- Air Force Office of Scientific Research (F49620)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- temporal abstraction, options framework, semi-Markov decision process, Markov decision process, hierarchical reinforcement learning, Q-learning, dynamic programming
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
