The DOI matches its Crossref record; 74 of the 75 references it cites were found in a registry or an index.
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In robotics, skills are often modelled using the Option framework [78].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [78]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Axten, N., Newell, A., & Simon, H. A. (1973) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2063712 · Cited by 6,886
Newell, A. (1972) · Cited by 6,222
Dynamic Programming and Markov Processes.
Freimer, M. L., & Howard, R. A. (1961) · Journal of the American Statistical Association · doi:10.2307/2282268 · Cited by 3,388
A qualitative physics based on confluences
de Kleer, J., & Brown, J. S. (1984) · Artificial Intelligence · doi:10.1016/0004-3702(84)90037-7 · Cited by 1,510
Planning in a hierarchy of abstraction spaces
Sacerdoti, E. D. (1974) · Artificial Intelligence · doi:10.1016/0004-3702(74)90026-5 · Cited by 1,185
Learning to act using real-time dynamic programming
Barto, A. G., Bradtke, S. J., & Singh, S. P. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(94)00011-o · Cited by 1,129
Learning and executing generalized robot plans
Fikes, R. E., Hart, P. E., & Nilsson, N. J. (1972) · Artificial Intelligence · doi:10.1016/0004-3702(72)90051-3 · Cited by 1,025
Reinforcement learning for robots using neural networks
Lin, L.-J. (1992) · Defense Technical Information Center (DTIC) · Cited by 841
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
Jaakkola, T., Jordan, M. I., & Singh, S. P. (1994) · Neural Computation · doi:10.1162/neco.1994.6.6.1185 · Cited by 801
Research works printed but not linked: 8
- E.G. Araujo, R.A. Grupen, Learning control composition in a complex environment, in: Proc. 4th International Conference on Simulation of Adaptive Behavior, 1996, pp. 333-342.
- S. Mahadevan, N. Marchalleck, T. Das, A. Gosavi, Self-improving factory simulation using continuous-time average-reward reinforcement learning, in: Proc. 14th International Conference on Machine Learning, 1997, pp. 202-210.
- D. Precup, R.S. Sutton, Multi-time models for reinforcement learning, in: Proc. ICML'97 Workshop on Modeling in Reinforcement Learning, 1997.
- D. Precup, R.S. Sutton, S.P. Singh, Planning with closed-loop macro actions, in: Working Notes 1997 AAAI Fall Symposium on Model-directed Autonomous Systems, 1997, pp. 70-76.
- S. Sastry, Algorithms for design of hybrid systems, in: Proc. International Conference of Information Sciences, 1997.
- S.P. Singh, Scaling reinforcement learning by learning variable temporal resolution models, in: Proc. 9th International Conference on Machine Learning, Morgan Kaufmann, San Mateo, CA, 1992, pp. 406-415.
- R.S. Sutton, B. Pinette, The learning of world models by connectionist networks, in: Proc. 7th Annual Conference of the Cognitive Science Society, 1985, pp. 54-64.
- C.J.C.H. Watkins, Learning with delayed rewards, Ph.D. Thesis, Cambridge University, 1989.
Cited by
Sorted by citations
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
Dynamic bayesian networks: representation, inference and learning
Murphy, K. P., & Russell, S. (2002) · Cited by 2,579
Deep reinforcement learning for autonomous driving: A survey
Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A. A. A., Yogamani, S., & Perez, P. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3054625 · Cited by 2,342
Brochu, E., Cora, V. M., & de Freitas, N. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.2599 · Cited by 2,568
Metadata
- Authors
- Sutton, R. S., Precup, D., & Singh, S.
- Affiliations
- AT&T (United States) (US)
- University of Massachusetts Amherst (US)
- Corresponding author
- Sutton, Richard S.
- Venue
- Artificial Intelligence
- Publisher
- Elsevier BV
- Published
- 1999-08
- Volume
- 112
- Issue
- 1-2
- Pages
- 181-211
- ISSN
- 0004-3702 (ISSN-L)
- OpenAlex
- W2109910161
- Cited by
- 3,119
- References
- 75
- Citations by year
- 2026: 179 · 2025: 206 · 2024: 188 · 2023: 212 · 2022: 213 · 2021: 270 · 2020: 303 · 2019: 238 · 2018: 183 · 2017: 112 · 2016: 99 · 2015: 80 · 2014: 74 · 2013: 91 · 2012: 86
- FWCI
- 27.32
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Artificial Intelligence (publishedVersion)
- Funders
- National Science Foundation (9711753, 9511805)
- Air Force Office of Scientific Research (F49620)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- temporal abstraction, options framework, semi-Markov decision process, Markov decision process, hierarchical reinforcement learning, Q-learning, dynamic programming
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
