The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Intrinsically Motivated Reinforcement Learning
Abstract
Psychologists call behavior intrinsically motivated when it is engaged in for its own sake rather than as a step toward solving a specific problem of clear practical value. But what we learn during intrinsically motivated behavior is essential for our development as competent autonomous entities able to efficiently solve a wide range of practical problems as they arise. In this paper we present initial results from a computational study of intrinsically motivated reinforcement learning aimed at allowing artificial agents to construct and extend hierarchies of reusable skills that are needed for competent autonomy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Self-citationMotivation reconsidered: The concept of competence.
White, R. W. (1963) · Perspectives in psychology. · doi:10.1037/14156-005 · Cited by 4,792
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119
Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping
Ng, A. Y., Harada, D., & Russell, S. (1999) · Cited by 1,744
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Discrete Event Dynamic Systems · doi:10.1023/a:1022140919877 · Cited by 1,024
Self-citationReward, Motivation, and Reinforcement Learning
Dayan, P., & Balleine, B. W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00963-7 · Cited by 855
Dopamine: generalization and bonuses
Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00048-5 · Cited by 480
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Deep reinforcement learning for autonomous driving: A survey
Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A. A. A., Yogamani, S., & Perez, P. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3054625 · Cited by 2,342
Curiosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815
Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
Kulkarni, T. D., Narasimhan, K., Saeedi, A., & Tenenbaum, J. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.06057 · Cited by 770
Formal theory of creativity, fun, and intrinsic motivation (1990–2010)
Schmidhuber, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2056368 · Cited by 747
(2016) · neural information processing systems · Cited by 735
Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective
Botvinick, M. M., Niv, Y., & Barto, A. G. (2009) · Cognition · doi:10.1016/j.cognition.2008.08.011 · Cited by 646
Self-citationNoisy Networks for Exploration
Fortunato, M., Azar, M. G., Piot, B., Menick, J., Osband, I., Graves, A., Mnih, V., Munos, R., Hassabis, D., Pietquin, O., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.10295 · Cited by 543
Model-based Reinforcement Learning: A Survey
M. Moerland, T., Broekens, J., Plaat, A., & M. Jonker, C. (2023) · Foundations and Trends in Machine Learning · doi:10.1561/2200000086 · Cited by 524
Emergent Tool Use From Multi-Agent Autocurricula
Baker, B., Kanitscheider, I., Markov, T., Wu, Y., Powell, G., McGrew, B., & Mordatch, I. (2019) · arXiv · doi:10.48550/arxiv.1909.07528 · Cited by 424
“[35] Nuttapong Chentanez, Andrew G Barto, and Satinder P Singh.”
BackgroundMethodAutonomous agents modelling other agents: A comprehensive survey and open problems
Albrecht, S. V., & Stone, P. (2018) · Artificial Intelligence · doi:10.1016/j.artint.2018.01.002 · Cited by 417
“We view them as assumptions about agents to allow for the more general notion of subjective goals, such as intrinsic rewards [245].”
BackgroundIntrinsically Motivated Reinforcement Learning: An Evolutionary Perspective
Singh, S., Lewis, R. L., Barto, A. G., & Sorg, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2051031 · Cited by 417
Self-citationTapus, A., Ţăpuş, C., & Matarić, M. J. (2008) · Intelligent Service Robotics · doi:10.1007/s11370-008-0017-4 · Cited by 402
Large-Scale Study of Curiosity-Driven Learning
Burda, Y., Edwards, H., Pathak, D., Storkey, A., Darrell, T., & Efros, A. A. (2018) · arXiv · doi:10.48550/arxiv.1808.04355 · Cited by 401
Intrinsic motivation, curiosity, and learning
Oudeyer, P.-Y., Gottlieb, J., & Lopes, M. (2016) · Progress in Brain Research · doi:10.1016/bs.pbr.2016.05.005 · Cited by 387
Extending the Soar Cognitive Architecture
Laird, J. E. (2007) · doi:10.21236/ada473738 · Cited by 364
Machado, M. C., Bellemare, M. G., Talvitie, E., Veness, J., Hausknecht, M., & Bowling, M. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5699 · Cited by 359
Deep reinforcement learning and its neuroscientific implications
Botvinick, M., Wang, J. X., Dabney, W., Miller, K. J., & Kurth-Nelson, Z. (2020) · Neuron · doi:10.1016/j.neuron.2020.06.014 · Cited by 269
Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts
Schmidhuber, J. (2006) · Connection Science · doi:10.1080/09540090600768658 · Cited by 268
Unifying Count-Based Exploration and Intrinsic Motivation
Bellemare, M. G., Srinivasan, S., Ostrovski, G., Schaul, T., Saxton, D., & Munos, R. (2016) · arXiv · doi:10.48550/arxiv.1606.01868 · Cited by 264
Increasing Student Intrinsic Motivation And Self-Efficacy Through Gamification Pedagogy
Banfield, J., & Wilkerson, B. (2014) · Contemporary Issues in Education Research (CIER) · doi:10.19030/cier.v7i4.8843 · Cited by 258
What is reinforced by phasic dopamine signals?
Redgrave, P., Gurney, K., & Reynolds, J. (2008) · Brain Research Reviews · doi:10.1016/j.brainresrev.2007.10.007 · Cited by 261
Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning
Mohamed, S., & Rezende, D. J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.08731 · Cited by 258
Deep Learning for Video Game Playing
Justesen, N., Bontrager, P., Togelius, J., & Risi, S. (2020) · IEEE Transactions on Games · doi:10.1109/tg.2019.2896986 · Cited by 257
Transfer in Reinforcement Learning: A Framework and a Survey
Lazaric, A. (2012) · Adaptation, Learning, and Optimization · doi:10.1007/978-3-642-27645-3_5 · Cited by 256
Towards Continual Reinforcement Learning: A Review and Perspectives
Khetarpal, K., Riemer, M., Rish, I., & Precup, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13673 · Cited by 224
Emotion in reinforcement learning agents and robots: a survey
Moerland, T. M., Broekens, J., & Jonker, C. M. (2018) · Machine Learning · doi:10.1007/s10994-017-5666-0 · Cited by 205
An information-theoretic approach to curiosity-driven reinforcement learning
Still, S., & Precup, D. (2012) · Theory in Biosciences · doi:10.1007/s12064-011-0142-z · Cited by 165
Symbol emergence in robotics: a survey
Taniguchi, T., Nagai, T., Nakamura, T., Iwahashi, N., Ogata, T., & Asoh, H. (2016) · Advanced Robotics · doi:10.1080/01691864.2016.1164622 · Cited by 150
Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play
Sukhbaatar, S., Lin, Z., Kostrikov, I., Synnaeve, G., Szlam, A., & Fergus, R. (2017) · arXiv · doi:10.48550/arxiv.1703.05407 · Cited by 149
Planning to be surprised: optimal bayesian exploration in dynamic environments
Sun, Y., Gomez, F., & Schmidhuber, J. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-22887-2_5 · Cited by 134
“Conceptually, however, this work is closely connected to artificial curiosity and intrinsically motivated reinforcement learning [6,7,9] for agents that actively explore the environment without an external reward signal.”
BackgroundReward learning from human preferences and demonstrations in Atari
Ibarz, B., Leike, J., Pohlen, T., Irving, G., Legg, S., & Amodei, D. (2018) · arXiv · doi:10.48550/arxiv.1811.06521 · Cited by 117
System Design Perspective for Human-Level Agents Using Deep Reinforcement Learning: A Survey
Nguyen, N. D., Nguyen, T., & Nahavandi, S. (2017) · IEEE Access · doi:10.1109/access.2017.2777827 · Cited by 100
EX2: Exploration with Exemplar Models for Deep Reinforcement Learning
Fu, J., Co-Reyes, J. D., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.01260 · Cited by 95
Intrinsic motivations and open-ended development in animals, humans, and robots: an overview
Baldassarre, G., Stafford, T., Mirolli, M., Redgrave, P., Ryan, R. M., & Barto, A. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00985 · Cited by 73
Self-citationInfluence-Based Multi-Agent Exploration
Wang, T., Wang, J., Wu, Y., & Zhang, C. (2019) · arXiv · doi:10.48550/arxiv.1910.05512 · Cited by 69
Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration
Colas, C., Karch, T., Lair, N., Dussoux, J.-M., Moulin-Frier, C., Dominey, P. F., & Oudeyer, P.-Y. (2020) · arXiv · doi:10.48550/arxiv.2002.09253 · Cited by 64
Hierarchical Query Graph Generation for Complex Question Answering over Knowledge Graph
Qiu, Y., Zhang, K., Wang, Y., Jin, X., Bai, L., Guan, S., & Cheng, X. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3411888 · Cited by 52
“Agents with intrinsic reward structures are explored in [37] in order to learn generic options that can apply to a wide variety of tasks.”
BackgroundMethodInfluentialCuriosity-Driven Experience Prioritization via Density Estimation
Zhao, R., & Tresp, V. (2019) · arXiv · doi:10.48550/arxiv.1902.08039 · Cited by 47
Social Diversity and Social Preferences in Mixed-Motive Reinforcement Learning
McKee, K. R., Gemp, I., McWilliams, B., Duèñez-Guzmán, E. A., Hughes, E., & Leibo, J. Z. (2020) · International Joint Conference on Autonomous Agents and Multiagent Systems · doi:10.65109/dnuf7773 · Cited by 33
Modeling Moral Choices in Social Dilemmas with Multi-Agent Reinforcement Learning
Tennant, E., Hailes, S., & Musolesi, M. (2023) · Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/36 · Cited by 21
Learning Reciprocity in Complex Sequential Social Dilemmas
Eccles, T., Hughes, E., Kramár, J., Wheelwright, S., & Leibo, J. Z. (2019) · arXiv · doi:10.48550/arxiv.1903.08082 · Cited by 19
Exploration and preference satisfaction trade-off in reward-free learning
Sajid, N., Tigas, P., Zakharov, A., Fountas, Z., & Friston, K. (2021) · arXiv · doi:10.48550/arxiv.2106.04316 · Cited by 11
Robeyns, M., Szummer, M., & Aitchison, L. (2025) · arXiv · doi:10.48550/arxiv.2504.15228 · Cited by 117
Toward Enactive Artificial Intelligence
Rafiee, B., & Sutton, R. (2026) · arXiv · doi:10.48550/arxiv.2605.24238
Metadata
- Authors
- Singh, S., Barto, A. G., & Chentanez, N.
- Publisher
- Defense Technical Information Center
- Published
- 2005-01-01
- OpenAlex
- W2139612737
- Cited by
- 610
- References
- 16
- Type
- report
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.