Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010)
The DOI matches its Crossref record; 125 of the 126 references it cites were found in a registry or an index.
Abstract
The simple, but general formal theory of fun and intrinsic motivation and creativity (1990-2010) is based on the concept of maximizing intrinsic reward for the active creation or discovery of novel, surprising patterns allowing for improved prediction or data compression. It generalizes the traditional field of active learning, and is related to old, but less formal ideas in aesthetics theory and developmental psychology. It has been argued that the theory explains many essential aspects of intelligence including autonomous development, science, art, music, and humor. This overview first describes theoretically optimal (but not necessarily practical) ways of implementing the basic computational principles on exploratory, intrinsically motivated agents or robots, encouraging them to provoke event sequences exhibiting previously unknown, but learnable algorithmic regularities. Emphasis is put on the importance of limited computational resources for online prediction and compression. Discrete and continuous time formulations are given. Previous practical, but nonoptimal implementations (1991, 1995, and 1997-2002) are reviewed, as well as several recent variants by others (2005-2010). A simplified typology addresses current confusion concerning the precise nature of intrinsic motivation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This underwrites Bayesian experimental design [134] and active learning in statistics [135], intrinsic motivation and artificial curiosity in machine learning and robotics [136, 137, 138, 139, 140].”
From The free energy principle made simpler but not too simple (2023) · cites this paper as [137]
In the citing paper
“This suggests that risk-averse policies minimise expected complexity or computational cost [137].”
From The free energy principle made simpler but not too simple (2023) · cites this paper as [137]
In the citing paper
“Ideally, the learner learns to choose actions that will lead to the largest improvements in its prediction or data compression performance (Schmidhuber, 2010).”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Schmidhuber, 2010
In the citing paper
“In Schmidhuber (2010), this is done in the framework of reinforcement learning, and incorporates a mechanisms for the system to measure its own rate of learning.”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Schmidhuber, 2010
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 7×
“Another potential drawback is the nature of the particular RNN-based RL method.”
BackgroundInfluentialSelf-citationA Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,002 · cited 1×
“iii) An evolutionary algorithm [22], [29], [59],”
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 7×
“Some Q-learning accelerators simply update pairs of actions and states with currently quickly changing Q-values more frequently than others (that is, Q-values with high first derivatives are favored).”
BackgroundMethodInfluentialReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Neural networks for pattern recognition
(1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,538
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Neural Networks for Pattern Recognition
Bishop, C. M. (1995) · doi:10.1093/oso/9780198538493.001.0001 · Cited by 12,287
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · cited 4×
“, some typically suboptimal RL method [35].”
BackgroundMethodInfluential
Research works printed but not linked: 24
- kullback, 1959, Statistics and Information Theory
- hutter, 2004, Universal Artificial Intelligence Sequential Decisions Based on Algorithmic Probability
- holland, 1975, Adaptation in Natural and Artificial Systems
- schmidhuber, 2010, How to Learn Programs with Artificial Recurrent Neural Networks
- harlow, 1950, novelty and curiosity as determinants of exploratory behavior, J Exp Psychol, 41, 68
- berlyne, 1950, novelty and curiosity as determinants of exploratory behavior, Brit J Psychol, 41, 68
- 10.1007/bf00993591 · doi:10.1007/bf00993591 · Paper page
- levin, 1973, universal sequential search problems, Problems Inform Trans, 9, 265
- schmidhuber, 1991, Advances in Neural Information Processing Systems 3, nips 3, 500
- schmidhuber, 1997, Femmes Fractales
- schmidhuber, 2002, Advances in Evolutionary Computing, 579
- ring, 1994, Continual Learning in Reinforcement Environments
- schmidhuber, 1990, Dynamische neuronale Netze und das fundamentale raumzeitliche Lernproblem
- schmidhuber, 1991, Adaptive Curiosity and Adaptive Confidence
- schmidhuber, 2009, Compression Progress The Algorithmic Principle Behind Curiosity and Creativity (With Applications of the Theory of Humor) 40 min Video of Invited Talk at Singularity Summit 2009
- schmidhuber, 2009, Multiple Ways to Design Research Research Cases that Reshape the Design Discipline, 98
- singh, 2005, Advances in neural information processing systems, 17
- schwefel, 1974, Numerische Optimierung von Computer-Modellen
- schmidhuber, 2007, alle berechenbaren universen (all computable universes), Spektrum der Wissenschaft, 75
- storck, 1995, reinforcement driven information acquisition in non-deterministic environments, Proc Int Conf Artif Neural Netw, 2, 159
- brafman, 2002, R-MAX—A general polynomial time algorithm for near-optimal reinforcement learning, J Mach Learn Res, 3, 213
- frank, 2002, sthetische Information Estetika informacio Eine Einfhrung in die Kybernetische sthetik
- franke, 1979, Kybernetische sthetik Phnomen Kunst
- schmidhuber, 2006, Artificial General Intelligence, 175
Cited by
Items 201–300 of 886
Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay ProcessesPreprint
Lesort, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00487 · Cited by 37
“…be defined internally by more abstract rules such as self-supervision Gopnik et al. (2001); Smith and Gasser (2005), intrinsic motivation or curiosity Oudeyer et al. (2007); Schmidhuber (2010) as in Forestier et al. (2017); Colas et al. (2018b); Craye et al. (2019); Laversanne-Finot et al. (2018).”
BackgroundVessel, E. A., Pasqualette, L. B. B. B., Uran, C., Koldehoff, S., Bignardi, G., & Vinck, M. A. (2023) · Psychological Science · doi:10.1177/09567976231188107 · Cited by 36
“Particularly in the context of a predictive-processing account of the mind/brain (Clark, 2013; Rao & Ballard, 1999), this is consistent with theories that posit that sensemaking and uncertainty reduction are positively valued and experienced as pleasurable (Biederman & Vessel, 2006; Schmidhuber, 2010).”
BackgroundLearning to Explore with Meta-Policy GradientPreprint
Xu, T.-B., Liu, Q., Zhao, L. L., & Peng, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.05044 · Cited by 36
“Intrinsic rewards such as prediction gain (Bellemare et al., 2016), learning progress (Oudeyer & Kaplan, 2007), compression progress (Schmidhuber, 2010), variational information maximization (Houthooft et al., 2016; Todd & Peter, 2017), have been employed to augment the environment’s reward signal…”
MethodSynthesising boredom: a predictive processing approach
Darling, T. (2023) · Synthese · doi:10.1007/s11229-023-04380-3 · Cited by 33
“In this literature, boredom is conceived, alongside creativity and fun (e.g. Schmidhuber, 1991, 2010), as a component of intrinsic motivation networks that aim toward maxi-mising epistemic (or informational) gain (e.g. Baldassarre & Mirolli, 2013; Barto et al., 2013; Friston et al., 2017a; Gottlieb…”
BackgroundLuciw, M., Kompella, V. R., Kazerounian, S., & Schmidhuber, J. (2013) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2013.00009 · Cited by 38
“…Lugano-Manno, Switzerland Edited by: Florian Röhrbein, Technische Universität München, Germany Reviewed by: Subramanian Ramamoorthy, The University of Edinburgh, UK Minoru Asada, Osaka University, Japan *Correspondence: Matthew Luciw and Varun Kompella, IDSIA, Galleria 2,…”
BackgroundSelf-citationUnderstanding exploration in humans and machines by formalizing the function of curiosity
Dubey, R., & Griffiths, T. L. (2020) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2020.07.008 · Cited by 38
Dynamic Artificial Neural Networks with Affective Systems
Schuman, C. D., & Birdwell, J. D. (2013) · PLoS ONE · doi:10.1371/journal.pone.0080455 · Cited by 31
Federated inference and belief sharing
Friston, K. J., Parr, T., Heins, C., Constant, A., Friedman, D., Isomura, T., Fields, C., Verbelen, T., Ramstead, M., Clippinger, J., et al. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105500 · Cited by 30
“At the level of learning, action involves updating beliefs about the parameters of the generative model: e.g., active learning (Mackay, 1992; Schmidhuber, 2010; Vigorito and Barto, 2010).”
Your Wit Is My Command: Building AIs with a Sense of Humor
Veale, T. (2021) · The MIT Press eBooks · doi:10.7551/mitpress/12465.001.0001 · Cited by 38
Sun, R., Xi, Y., Stefanidis, A., Jiang, Z., & Su, J. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-01884-y · Cited by 33
“By doing so, HDRL strengthens the learning process of individual abstractions within the hierarchy [64].”
Unsupervised Real-Time Control Through Variational Empowerment
Karl, M., Becker-Ehmck, P., Soelch, M., Benbouzid, D., van der Smagt, P., & Bayer, J. (2022) · Springer proceedings in advanced robotics · doi:10.1007/978-3-030-95459-8_10 · Cited by 37
“Schmidhuber [1991, 2010] proposed to use a formalised notion of interestingness and curiosity.”
Malthusian Reinforcement Learning
Leibo, J. Z., Pérolat, J., Hughes, E., Wheelwright, S., Marblestone, A. H., Duéñez‐Guzmán, E. A., Sunehag, P., Dunning, I., & Graepel, T. (2019) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/fdir9565 · Cited by 43
Learning skills from play: Artificial curiosity on a Katana robot arm
Ngo, H., Luciw, M., Förster, A., & Schmidhuber, J. (2012) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2012.6252824 · Cited by 30
“There is a mathematical theory of how artificial systems can develop skills through playing [14], [15], based on two decades of work on artificial curiosity [15] (AC).”
BackgroundSelf-citationLearning Purposeful Behaviour in the Absence of RewardsPreprint
Machado, M. C., & Bowling, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07700 · Cited by 35
World Discovery ModelsPreprint
Azar, M. G., Piot, B., Pires, B. Á., Grill, J.-B., Altché, F., & Munos, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07685 · Cited by 37
“More specifically, in his survey on the theory of creativity and intrinsic motivation, Schmidhuber (2010) explains how to build the agent that could discover and understand in a self-supervised way its environment.”
InfluentialEditorial: Intrinsically Motivated Open-Ended Learning in Autonomous Robots
Santucci, V. G., Oudeyer, P., Barto, A. G., & Baldassarre, G. (2020) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2019.00115 · Cited by 44
“…expanding into other fields, intrinsically motivated learning (sometimes called “curiosity-driven learning”) has been studied by many researchers as an approach to autonomous lifelong learning in machines (Oudeyer et al., 2007; Schmidhuber, 2010; Barto, 2013; Mirolli and Baldassarre, 2013).”
BackgroundPando‐Naude, V., Matthews, T. E., Højlund, A., Jakobsen, S., Østergaard, K., Johnsen, E., Garza‐Villarreal, E. A., Witek, M. A. G., Penhune, V., & Vuust, P. (2024) · European Journal of Neuroscience · doi:10.1111/ejn.16128 · Cited by 29
Expanding the Active Inference Landscape: More Intrinsic Motivations in the Perception-Action Loop
Biehl, M., Guckelsberger, C., Salge, C., Smith, S. C., & Polani, D. (2018) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2018.00045 · Cited by 37
“Computational approaches to intrinsic motivations (Oudeyer and Kaplan, 2009; Schmidhuber, 2010; Santucci et al., 2013) can be categorized roughly by the psychological motivations they are imitating, e.g., drives to manipulate and explore, the reduction of cognitive dissonance, the achievement of…”
BackgroundResult“Liking” as an early and editable draft of long-run affective value
Dayan, P. (2022) · PLoS Biology · doi:10.1371/journal.pbio.3001476 · Cited by 30
“The worth of this change is exactly what one large class of theories considers to generate aesthetic value for stimuli—these are the theories that concentrate on learning progress or prediction error [76].”
BackgroundThe free energy principle induces neuromorphic development
Fields, C., Friston, K. J., Glazebrook, J. F., Levin, M. G., & Marcianò, A. (2022) · Neuromorphic Computing and Engineering · doi:10.1088/2634-4386/aca7de · Cited by 31
“In turn, this requires the right kind of ‘coarse graining’ or compression [89], to provide an accurate explanation for impressions on the sensory part of the holographic screen implemented by the MB.”
BackgroundEffects of implementing an integrative drama-inquiry learning model in a science classroom
Kolovou, M., & Kim, N. J. (2020) · The Journal of Educational Research · doi:10.1080/00220671.2020.1771673 · Cited by 31
“Likewise, having fun during creative activities maximizes intrinsic motivation, according to the “Formal Theory of Creativity, Fun, and Intrinsic Motivation” (Schmidhuber, 2010).”
BackgroundReady Policy One: World Building Through Active LearningPreprint
Ball, P., Parker-Holder, J., Pacchiano, A., Choromański, K., & Roberts, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.02693 · Cited by 39
“This is likely due to over-emphasis of regions of the state space that are inherently hard to predict, but do not impact the ability to solve the task (Schmidhuber, 2010).”
BackgroundPreschoolers search longer when there is more information to be gained
Ruggeri, A., Stanciu, O., Pelz, M. C., Alison Gopnik, C., & Schulz, E. (2023) · Developmental Science · doi:10.1111/desc.13411 · Cited by 31
“Indeed,Schmidhuber(2010)proposedafor-mal conceptualization of “fun” along similar theoretical lines (i.e., fun measured by the extent to which one’s model of the world improves witheachstep).”
BackgroundArtificial curiosity with planning for autonomous perceptual and cognitive development
Luciw, M., Graziano, V., Ring, M. B., & Schmidhuber, J. (2011) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2011.6037356 · Cited by 30
“In this paper, we use the framework of artificial curiosity [18] to generate the intrinsic reward signal.”
MethodSelf-citationIntrinsically motivated discovered outcomes boost user's goals achievement in a humanoid robot
Seepanomwan, K., Santucci, V. G., & Baldassarre, G. (2017) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2017.8329804 · Cited by 34
“[13], [26], has been successfully introduced in the field of machine learning [36], [6] and developmental robotics [24], [2] to enhance the autonomous development of artificial agents and the open-ended learning of repertoires of skills.”
BackgroundKnow Your Body Through Intrinsic Goals
Mannella, F., Santucci, V. G., Somogyi, E., Jacquey, L., O'Regan, J. K., & Baldassarre, G. (2018) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2018.00030 · Cited by 34
Efficiency Theory : a Unifying Theory for Information, Computation and Intelligence
Yampolskiy, R. V. (2013) · Journal of Discrete Mathematical Sciences and Cryptography · doi:10.1080/09720529.2013.821361 · Cited by 43
“ Art, Beauty, Music, Novelty, Surprise, Interestingness, Attention, Curiosity, Science, Music, Jokes and Creativity are byproducts of our desire to discover novel patterns by representing (compressing) data in efficient ways [72, 73].”
BackgroundBootstrapping intrinsically motivated learning with human demonstration
Nguyen, S. M., Baranes, A., & Oudeyer, P.-Y. (2011) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2011.6037329 · Cited by 30
Moran, M., & Gordon, G. (2019) · Information Sciences · doi:10.1016/j.ins.2019.02.009 · Cited by 32
Creative Outcome as Implausible Utility
Tsao, J. Y., Ting, C. L., & Johnson, C. M. (2019) · Review of General Psychology · doi:10.1177/1089268019857929 · Cited by 31
“From human (Guilford, 1967) to machine (Schmidhuber, 2010) learning, from scientific (Feist, 2008) to artistic (Catmull & Wallace, 2014) innovation, from geniuses (Simonton, 1988) to ordinary mortals (Simon, 1977), creativity helps advance the knowledge that enables humanity to adapt to its environments.”
BackgroundMoran, M., Cohen, T., Ben-Zion, Y., & Gordon, G. (2022) · Information Sciences · doi:10.1016/j.ins.2022.07.025 · Cited by 29
In praise of folly: flexible goals and human cognition
Chu, J., Tenenbaum, J. B., & Schulz, L. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.03.006 · Cited by 41
“…of proxy goals support learning new knowledge and skills, including exploring rare, novel, or surprising events [103], trying to maximize expected information gain or the rate of reducing prediction error [47,49,104], or trying to perform particular actions and reach particular states [105,106].”
The image-scratch paradigm: a new paradigm for evaluating infants' motivated gaze control
Miyazaki, M., Takahashi, H., Rolf, M., Okada, H., & Omori, T. (2014) · Scientific Reports · doi:10.1038/srep05498 · Cited by 32
A Theoretical Framework for How We Learn Aesthetic Values
Aleem, H., Correa-Herran, I., & Grzywacz, N. M. (2020) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2020.00345 · Cited by 29
“Another Frontiers in Human Neuroscience | www.frontiersin.org 15 September 2020 | Volume 14 | Article 345 computational theory comes from Schmidhuber, who contends that aesthetic preferences are largely driven by intrinsic reward (Schmidhuber, 2010).”
BackgroundImprovised Comedy as a Turing TestPreprint
Mathewson, K. W., & Mirowski, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.08819 · Cited by 29
“We plan to conduct an experimental study of the audience beliefs in shared AI and human creativity[10].”
MethodBootstrapping Intrinsically Motivated Learning with Human Demonstration
Nguyen, S. M., Baranès, A., & Oudeyer, P. (2012) · Cited by 26
Would I have gotten that reward? Long-term credit assignment by counterfactual contribution analysis
Meulemans, A., Schug, S., Kobayashi, S., Daw, N. D., & Wayne, G. (2023) · neural information processing systems · doi:10.52202/075280-3006 · Cited by 25
Natural language syntax complies with the free-energy principle
Murphy, E., Holmes, E., & Friston, K. J. (2024) · Synthese · doi:10.1007/s11229-024-04566-3 · Cited by 29
“We find seeds for these ideas in the foundational principles of universal computation, where, as we have noted, free energy is often discussed in terms of minimum description or message lengths (MacKay, 2003; Schmidhuber, 2010).”
BackgroundTanneberg, D., Peters, J., & Rueckert, E. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.10.005 · Cited by 29
Autonomous Reinforcement Learning of Multiple Interrelated Tasks
Santucci, V. G., Baldassarre, G., & Cartoni, E. (2019) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2019.8850713 · Cited by 30
Mindstorms in Natural Language-Based Societies of Mind
Zhuge, M., Liu, H., Faccio, F., Ashley, D. R., Csordás, R., Gopalakrishnan, A., Hamdi, A., Hammoud, H. A. A. K., Herrmann, V., Irie, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.17066 · Cited by 24
Self-citationMindstorms in Natural Language-Based Societies of Mind
Zhuge, M., Liu, H., Faccio, F., Ashley, D. R., Csordás, R., Gopalakrishnan, A., Hamdi, A., Hammoud, H. A. A. K., Herrmann, V., Irie, K., et al. (2025) · Computational Visual Media · doi:10.26599/cvm.2025.9450460 · Cited by 24
Self-citationCuriosity-driven exploration enhances motor skills of continuous actor-critic learner
Hafez, M. B., Weber, C., & Wermter, S. (2017) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2017.8329785 · Cited by 27
Geometric methods for sampling, optimization, inference, and adaptive agents
Barp, A., Costa, L. D., França, G., Friston, K. J., Girolami, M., Jordan, M. I., & Pavliotis, G. A. (2022) · Handbook of statistics · doi:10.1016/bs.host.2022.03.005 · Cited by 32
“Making decisions to maximise information gain leads to a goal-directed form of exploration [231], driven to answer “What would happen if I did that?” [246].”
BackgroundGeneral intelligence requires rethinking exploration
Jiang, M., Rocktäschel, T., & Grefenstette, E. (2023) · Royal Society Open Science · doi:10.1098/rsos.230539 · Cited by 51
“Notably, both Schmidhuber [10,32,33] and Clune [34] have proposed novelty-seeking agents that generate their own challenges, while the work of Stanley and Lehman established the practical value of taming open-endedness [35 – 37].”
BackgroundSantiago-Torner, C. (2023) · Revista iberoamericana de estudios de desarrollo = Iberoamerican journal of development studies · doi:10.26754/ojs_ried/ijds.769 · Cited by 27
Curious Hierarchical Actor-Critic Reinforcement Learning
Röder, F., Eppe, M., Nguyen, P. D. H., & Wermter, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61616-8_33 · Cited by 28
Intrinsically motivated neuroevolution for vision-based reinforcement learning
Cuccu, G., Luciw, M., Schmidhuber, J., & Gomez, F. J. (2011) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2011.6037324 · Cited by 24
Self-citationDuminy, N., Nguyen, S. M., & Duhaut, D. (2019) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2018.00087 · Cited by 23
“Psychological theories such as intrinsic motivation have tried to explain these apparently non-rewarding behaviours and have been successfully inspired learning algorithms Oudeyer et al. (2007); Schmidhuber (2010).”
BackgroundMoran, M., & Gordon, G. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3282564 · Cited by 23
Bridging Information Visualization with Machine Learning (Dagstuhl Seminar 15101)
Keim, D. A., Munzner, T., Fabrice Rossi, A., & Verleysen, M. (2015) · Dagstuhl reports · doi:10.4230/dagrep.5.3.1 · Cited by 23
Intrinsic motivation mechanisms for competence acquisition
Santucci, V. G., Baldassarre, G., & Mirolli, M. (2012) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2012.6400835 · Cited by 25
“For example, in [17], [18] the IM signal is used to improve the acquisition of behaviours mainly driven by extrinsic rewards.”
BackgroundMethodEfficient Novelty Search Through Deep Reinforcement Learning
Shi, L., Li, S., Zheng, Q., Yao, M., & Pan, G. (2020) · IEEE Access · doi:10.1109/access.2020.3008735 · Cited by 27
Scheduled Intrinsic Drive: A Hierarchical Take on Intrinsically Motivated ExplorationPreprint
Zhang, J., Wetzel, N., Dorka, N., Boedecker, J., & Burgard, W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.07400 · Cited by 25
“Intrinsic motivation can be defined as agents conducting actions purely out of the satisfaction of its internal rewarding system rather than the extrinsic rewards (Oudeyer & Kaplan, 2008; Schmidhuber, 2010).”
BackgroundCuriosity and Interactive Learning in Artificial Systems
Haber, N. (2022) · Springer eBooks · doi:10.1007/978-3-031-09687-7_3 · Cited by 26
Learning data teaching strategies via knowledge tracing
Abdelrahman, G. M., & Wang, Q. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110511 · Cited by 24
General Self-Motivation and Strategy Identification: Case Studies Based on Sokoban and Pac-Man
Anthony, T., Polani, D., & Nehaniv, C. L. (2014) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2013.2295372 · Cited by 28
“Take, for example, Schmidhuber’s compression progress framework [39], which bases its measure directly on the progress in compression efficiency in a Kolmogorovtype framework as actions and observations of an agent proceed through time, and which has been extended toward reinforcement-learning-like frameworks (AIXI, […”
BackgroundSchaarschmidt, M., Homscheid, D., & Kilian, T. (2019) · International Journal of Innovation Management · doi:10.1142/s1363919619500336 · Cited by 24
Deep intrinsically motivated continuous actor-critic for efficient robotic visuomotor skill learning
Hafez, M. B., Weber, C., Kerzel, M., & Wermter, S. (2019) · Paladyn Journal of Behavioral Robotics · doi:10.1515/pjbr-2019-0005 · Cited by 24
Memory, attention and prediction: a deep learning architecture for car-following
Wu, Y., Tan, H., Chen, X., & Ran, B. (2019) · Transportmetrica B Transport Dynamics · doi:10.1080/21680566.2019.1650674 · Cited by 30
“In the field of artificial neural networks, the prediction capability of neural networks has long been used to model the intrinstic motivation and regarded as a key factor for many essential aspects of intelligence (Schmidhuber 2010).”
Batch Exploration With Examples for Scalable Robotic Reinforcement Learning
Chen, A. S., Nam, H., Nair, S., & Finn, C. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3068655 · Cited by 31
Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious ExplorationPreprint
Groth, O., Wulfmeier, M., Vezzani, G., Dasagi, V., Hertweck, T., Hafner, R., Heess, N., & Riedmiller, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.08603 · Cited by 26
“Intrinsic motivation [15, 11, 12, 2, 16] can be a powerful concept to endow an agent with an automated mechanism to continuously explore its environment in the absence of task information.”
BackgroundGrace, K. S., & Maher, M. L. (2014) · ICCC · Cited by 28
Butlin, P. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462570 · Cited by 26
“In particular, some recent research proposes that learning is a reward in itself [46, 39, 40, 2, 8].”
BackgroundNovelty as a drive of human exploration in complex stochastic environments
Modirshanechi, A., Lin, W.-H., He, X., Herzog, M. H., & Gerstner, W. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2502193122 · Cited by 24
Conclusions, Future Directions and Outlook
Wiering, M., & van Otterlo, M. (2012) · Adaptation, learning, and optimization · doi:10.1007/978-3-642-27645-3_19 · Cited by 21
Efficient and Robust Reinforcement Learning with Uncertainty-based Value ExpansionPreprint
Zhou, B., Zeng, H.-S., Wang, F., Li, Y.-X., & Tian, H. (2019) · arXiv.org · doi:10.48550/arxiv.1912.05328 · Cited by 18
“The aleatoric uncertainty is more related to the stochastic transition, and the epistemic uncertainty is usually of interest to many works in terms of exploitation & exploration [Pathak et al., 2017, Schmidhuber, 2010, Oudeyer and Kaplan, 2009].”
Novelty Search in Representational Space for Sample Efficient ExplorationPreprint
Tao, R. Y., François-Lavet, V., & Pineau, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.13579 · Cited by 51
“A common approach to exploration is to leverage intrinsic rewards correlated with some metric or score for novelty (Schmidhuber, 2010; Stadie et al., 2015; Houthooft et al., 2016).”
BackgroundSontakke, S., Mehrjou, A., Itti, L., & Schölkopf, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.03110 · Cited by 61
When should agents explore?Preprint
Pislar, M., Szepesvári, D., Ostrovski, G., Borsa, D., & Schaul, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.11811 · Cited by 22
“Exploration with gradual change instead of abrupt mode switches generally appears at long time-scales, such as when pursuing intrinsic rewards (Schmidhuber, 2010; Oudeyer & Kaplan, 2009) but can also be effective at shorter timescales, e.”
BackgroundElpidorou, A. (2025) · doi:10.1093/9780197564363.001.0001 · Cited by 21
Bown, O. (2021) · Springer eBooks · doi:10.1007/978-3-030-72116-9_1 · Cited by 38
Fast adaptation to rule switching using neuronal surprise
Barry, M., & Gerstner, W. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011839 · Cited by 23
Jin, Y., & Lee, M. (2018) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2017.2779474 · Cited by 20
“To be specific, intrinsic motivation employs an internal reward signal to simulate a reinforcement learning agent to explore the environment in a goal-independent manner [32], [33].”
BackgroundSun, Y., Wang, X., Zhang, Z., & Bhattacherjee, A. (2019) · IEEE Access · doi:10.1109/access.2019.2911197 · Cited by 20
“Most studies agree that intrinsic motivation has a direct impact on individual innovation, such as creativity behavior [23], [35], [71], exploration use [13], [14], [72], and creativity performance [61].”
BackgroundNovelty Detection as an Intrinsic Motivation for Cumulative Learning Robots
Nehmzow, U., Gatsoulis, Y., Kerr, E., Condell, J. V., Siddique, N. H., & McGinnity, T. M. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_8 · Cited by 25
“An overview of a formal theory of intrinsic motivations based on prediction improvement and the exploitation of the resulting learning signal to train reinforcement learning systems is reported in Schmidhuber (2010, 2011).”
BackgroundPlanning Goals for ExplorationPreprint
Hu, E. S., Chang, R. C., Rybkin, O., & Jayaraman, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.13002 · Cited by 21
Cultivating creativity: predictive brains and the enlightened room problem
Constant, A., Friston, K. J., & Clark, A. (2024) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2022.0415 · Cited by 19
“This has been discussed in the context of ‘ aha moments ’ [68], and evidenced in van de Cruys et al. [69], as insights that provide a unifying explanation for the way the world works [20,65,66,70].”
BackgroundDeepCreativity: measuring creativity with deep learning techniques
Franceschelli, G., & Musolesi, M. (2022) · Intelligenza Artificiale · doi:10.3233/ia-220136 · Cited by 19
Towards machines that understand people
Howes, A., Jokinen, J., & Oulasvirta, A. (2023) · AI Magazine · doi:10.1002/aaai.12116 · Cited by 34
“Related to this is work on the rational basis of motivation (Dubey and Griffiths 2020a, 2020b; Biehl et al. 2018; Schmidhuber 2010; Roohi et al. 2018; Ecoffet et al. 2021).”
BackgroundThe Curious U : Integrating Theories Linking Knowledge and Information-Seeking Behavior
Ten, A., Oudeyer, P., Sakaki, M., & Murayama, K. (2025) · Open Mind · doi:10.1162/opmi.a.41 · Cited by 19
“Here, we follow the formal framework introduced by Lopes and Oudeyer (2012; also see Schmidhuber, 2010, for a very similar framework).”
InfluentialGAME: Governance for Agile Management of Enterprises: A Management Model for Agile Governance
Luna, A. J. H. D. O., Kruchten, P. B., & de Moura, H. P. (2013) · IEEE International Conference on Global Software Engineering Workshops (ICGSEW) · doi:10.1109/icgsew.2013.20 · Cited by 19
“It generalizes the traditional field of active learning, and is related to old, but less formal ideas in aesthetics theory and developmental psychology [22].”
BackgroundAutonomous Exploration For Navigating In MDPs
Lim, S. H., & Auer, P. L. (2012) · Conference on Learning Theory · Cited by 28
Social behaviour as an emergent property of embodied curiosity: a robotics perspective
Gordon, G. (2019) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2018.0029 · Cited by 22
Cults, Conspiracies, and Fantasies of Knowledge
Munro, D. (2023) · Episteme · doi:10.1017/epi.2022.55 · Cited by 20
A Formal Theory of Creativity to Model the Creation of Art
Schmidhuber, J. (2012) · Springer eBooks · doi:10.1007/978-3-642-31727-9_12 · Cited by 28
Self-citationMoulin-Frier, C., & Oudeyer, P. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-340 · Cited by 21
“Among these guiding mechanisms, intrinsic motivations, generating spontaneous exploration in humans [10, 4], have been transposed in curiosity-driven learning machines [11, 12, 13] and robots [9, 7] and shown to yield highly efficient learning of inverse models in high-dimensional redundant sensorimotor spaces [7].”
BackgroundGrowing Recursive Self-Improvers
Steunebrink, B. R., Þórisson, K. R., & Schmidhuber, J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-41649-6_13 · Cited by 19
Self-citationCosta, L. D., Sajid, N., Parr, T., Friston, K. J., & Smith, R. S. (2020) · arXiv (Cornell University) · Cited by 23
Variational Empowerment as Representation Learning for Goal-Based Reinforcement LearningPreprint
Choi, J., Sharma, A., Lee, H., Levine, S., & Gu, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01404 · Cited by 29
“Task-agnostic reward functions have been proposed to encourage exploration in environments using notions of curiosity or novelty (Schmidhuber, 1991; Oudeyer & Kaplan, 2009; Schmidhuber, 2010; Bellemare et al., 2016; Pathak et al., 2017; Colas et al., 2018).”
BackgroundImaginary Hindsight Experience Replay: Curious Model-based Learning for Sparse Reward TasksPreprint
McCarthy, R., Wang, Q., & Redmond, S. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02414 · Cited by 19
“However, this approach can lead to the agent becoming ‘obsessed’ with inherently unpredictable elements of the environment [31].”
BackgroundActive Inference as a Model of AgencyPreprint
Costa, L. D., Tenka, S., Zhao, D., & Sajid, N. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.12917 · Cited by 19
“[45].”
MethodDupoux, E., LeCun, Y., & Malik, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.15381 · Cited by 21
“System A can help by providing intrinsic reward signals like prediction errors, uncertainty, or novelty, enabling the agent to explore efficiently, and shift to exploitation once it is confident (Oudeyer et al., 2007; Schmidhuber, 2010; Aubret et al., 2023).”
Modelling individual aesthetic judgements over time
Brielmann, A. A., Berentelg, M., & Dayan, P. (2023) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2022.0414 · Cited by 20
Zahedi, A., Lynn, S. J., & Sommer, W. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1388347 · Cited by 19
“In other words, one can also view predictive coding as finding efficient and compressed representations of (the causes of) sensory data (Schmidhuber, 2010), which speaks to the parsimonious way in which we encode our world.”
BackgroundPlay in Cognitive Development: From Rational Constructivism to Predictive Processing
Andersen, M. M., & Kiverstein, J. D. (2024) · Topics in Cognitive Science · doi:10.1111/tops.12752 · Cited by 21
DEMONSTRATING SENSEMAKING EMERGENCE IN ARTIFICIAL AGENTS: A METHOD AND AN EXAMPLE
Georgeon, O. L., & Marshall, J. B. (2013) · International Journal of Machine Consciousness · doi:10.1142/s1793843013500029 · Cited by 18
“For example, Oudeyer, Kaplan, and Hafner [2007] proposed a motivational system based on curiosity; Blank, Lewis, and Marshall [2005] proposed the search for predictability and control; Schmidhuber [2010] proposed the search for simplification of knowledge and compressibility of data.”
Active inference, preference learning and adaptive behaviour
Sajid, N., Tigas, P., & Friston, K. J. (2022) · IOP Conference Series Materials Science and Engineering · doi:10.1088/1757-899x/1261/1/012020 · Cited by 24
“Conceptually, preference learning active inference agents are most similar to open-ended learning algorithms Schmidhuber (2010); Standish (2003); Stanley et al. (2017), where agents are responsible for never-ending learning opportunities.”
BackgroundLearning and control of exploration primitives
Gordon, G., Fonio, E., & Ahissar, E. (2014) · Journal of Computational Neuroscience · doi:10.1007/s10827-014-0500-1 · Cited by 19
“…management There are several models of intrinsic motivation, or artificial curiosity, that attempt to maximize or minimize some form of information gain or surprise (Schmidhuber 1990; Polani 2009; Schmidhuber 2010; Friston 2010; Tishby and Polani 2011; Der and Martius 2012; Little and Sommer 2013).”
BackgroundControl Capacity of Partially Observable Dynamic Systems in Continuous TimePreprint
Tiomkin, S., Polani, D., & Tishby, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.04984 · Cited by 18
Metadata
- Authors
- Schmidhuber, J.
- Affiliations
- Dalle Molle Institute for Artificial Intelligence Research (CH)
- Corresponding author
- Schmidhuber, Jürgen
- ORCID
- Schmidhuber, Jürgen
- Venue
- IEEE Transactions on Autonomous Mental Development
- Published
- 2010-09
- Volume
- 2
- Issue
- 3
- Pages
- 230-247
- ISSN
- 1943-0604 (ISSN-L)
- 1943-0612
- OpenAlex
- W2034806191
- Cited by
- 747
- References
- 126
- Citations by year
- 2026: 29 · 2025: 64 · 2024: 46 · 2023: 50 · 2022: 60 · 2021: 69 · 2020: 92 · 2019: 72 · 2018: 55 · 2017: 37 · 2016: 26 · 2015: 24 · 2014: 30 · 2013: 28 · 2012: 41
- FWCI
- 31.09
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Autonomous Mental Development (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
- Keywords
- intrinsic motivation, creativity, fun, intrinsic rewards, active learning (machine learning), novelty seeking, surprise, pattern discovery, data compression, online prediction, autonomous agents, robots, aesthetics, humor, intelligence, limited computational resources, continuous-time formulation, discrete-time formulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.