Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010)
The DOI matches its Crossref record; 125 of the 126 references it cites were found in a registry or an index.
Abstract
The simple, but general formal theory of fun and intrinsic motivation and creativity (1990-2010) is based on the concept of maximizing intrinsic reward for the active creation or discovery of novel, surprising patterns allowing for improved prediction or data compression. It generalizes the traditional field of active learning, and is related to old, but less formal ideas in aesthetics theory and developmental psychology. It has been argued that the theory explains many essential aspects of intelligence including autonomous development, science, art, music, and humor. This overview first describes theoretically optimal (but not necessarily practical) ways of implementing the basic computational principles on exploratory, intrinsically motivated agents or robots, encouraging them to provoke event sequences exhibiting previously unknown, but learnable algorithmic regularities. Emphasis is put on the importance of limited computational resources for online prediction and compression. Discrete and continuous time formulations are given. Previous practical, but nonoptimal implementations (1991, 1995, and 1997-2002) are reviewed, as well as several recent variants by others (2005-2010). A simplified typology addresses current confusion concerning the precise nature of intrinsic motivation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This underwrites Bayesian experimental design [134] and active learning in statistics [135], intrinsic motivation and artificial curiosity in machine learning and robotics [136, 137, 138, 139, 140].”
From The free energy principle made simpler but not too simple (2023) · cites this paper as [137]
In the citing paper
“This suggests that risk-averse policies minimise expected complexity or computational cost [137].”
From The free energy principle made simpler but not too simple (2023) · cites this paper as [137]
In the citing paper
“Ideally, the learner learns to choose actions that will lead to the largest improvements in its prediction or data compression performance (Schmidhuber, 2010).”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Schmidhuber, 2010
In the citing paper
“In Schmidhuber (2010), this is done in the framework of reinforcement learning, and incorporates a mechanisms for the system to measure its own rate of learning.”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Schmidhuber, 2010
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 7×
“Another potential drawback is the nature of the particular RNN-based RL method.”
BackgroundInfluentialSelf-citationA Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,002 · cited 1×
“iii) An evolutionary algorithm [22], [29], [59],”
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 7×
“Some Q-learning accelerators simply update pairs of actions and states with currently quickly changing Q-values more frequently than others (that is, Q-values with high first derivatives are favored).”
BackgroundMethodInfluentialReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Neural networks for pattern recognition
(1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,538
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Neural Networks for Pattern Recognition
Bishop, C. M. (1995) · doi:10.1093/oso/9780198538493.001.0001 · Cited by 12,287
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · cited 4×
“, some typically suboptimal RL method [35].”
BackgroundMethodInfluential
Research works printed but not linked: 24
- kullback, 1959, Statistics and Information Theory
- hutter, 2004, Universal Artificial Intelligence Sequential Decisions Based on Algorithmic Probability
- holland, 1975, Adaptation in Natural and Artificial Systems
- schmidhuber, 2010, How to Learn Programs with Artificial Recurrent Neural Networks
- harlow, 1950, novelty and curiosity as determinants of exploratory behavior, J Exp Psychol, 41, 68
- berlyne, 1950, novelty and curiosity as determinants of exploratory behavior, Brit J Psychol, 41, 68
- 10.1007/bf00993591 · doi:10.1007/bf00993591 · Paper page
- levin, 1973, universal sequential search problems, Problems Inform Trans, 9, 265
- schmidhuber, 1991, Advances in Neural Information Processing Systems 3, nips 3, 500
- schmidhuber, 1997, Femmes Fractales
- schmidhuber, 2002, Advances in Evolutionary Computing, 579
- ring, 1994, Continual Learning in Reinforcement Environments
- schmidhuber, 1990, Dynamische neuronale Netze und das fundamentale raumzeitliche Lernproblem
- schmidhuber, 1991, Adaptive Curiosity and Adaptive Confidence
- schmidhuber, 2009, Compression Progress The Algorithmic Principle Behind Curiosity and Creativity (With Applications of the Theory of Humor) 40 min Video of Invited Talk at Singularity Summit 2009
- schmidhuber, 2009, Multiple Ways to Design Research Research Cases that Reshape the Design Discipline, 98
- singh, 2005, Advances in neural information processing systems, 17
- schwefel, 1974, Numerische Optimierung von Computer-Modellen
- schmidhuber, 2007, alle berechenbaren universen (all computable universes), Spektrum der Wissenschaft, 75
- storck, 1995, reinforcement driven information acquisition in non-deterministic environments, Proc Int Conf Artif Neural Netw, 2, 159
- brafman, 2002, R-MAX—A general polynomial time algorithm for near-optimal reinforcement learning, J Mach Learn Res, 3, 213
- frank, 2002, sthetische Information Estetika informacio Eine Einfhrung in die Kybernetische sthetik
- franke, 1979, Kybernetische sthetik Phnomen Kunst
- schmidhuber, 2006, Artificial General Intelligence, 175
Cited by
Items 401–500 of 886
Embracing curiosity eliminates the exploration-exploitation dilemmaPreprint
Peterson, E. J., & Verstynen, T. D. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/671362 · Cited by 15
Deep Learning: Our Miraculous Year 1990-1991Preprint
Schmidhuber, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.05744 · Cited by 13
“[RPG] D. Wierstra, A. Foerster, J. Peters, J. Schmidhuber (2010).”
Self-citationJoint Attention for Multi-Agent Coordination and Social LearningPreprint
Lee, D. M., Jaques, N., Kew, J. C., Wu, J., Eck, D., Schuurmans, D., & Faust, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.07750 · Cited by 13
“This can be considered an intrinsic motivation [14], [15] to maintain joint attention with other agents.”
AutoIncSFA and vision-based developmental learning for humanoid robots
Kompella, V. R., Pape, L., Masci, J., Frank, M., & Schmidhuber, J. (2011) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2011.6100865 · Cited by 9
“On humanoid robots, reinforcement learning (RL) [1], [2] could be used to learn skill repertoires, especially if there are self-generated intrinsic curiosity rewards [3]– [7] for action sequences leading to the discovery of new regularities in the observations.”
BackgroundMethodInfluentialSelf-citationUsing perceptual classes to dream policies in open-ended learning robotics
Romero, A., Meden, B., Bellas, F., & Duro, R. J. (2023) · Integrated Computer-Aided Engineering · doi:10.3233/ica-230707 · Cited by 9
“Some of the most influential work, such as those carried out in [16,17] on machine learning or in [18,19] on robots, have led to algorithms that allow autonomous agents to learn models of the domain in which they operate, so that they can "predict" and anticipate the environmental effects of their…”
Parallel development of social behavior in biological and artificial fish
McGraw, J. D., Lee, D., & Wood, J. N. (2024) · Nature Communications · doi:10.1038/s41467-024-52307-4 · Cited by 10
“The second algorithm — intrinsic motivation — provides a self-supervised reward landscape for optimizing deep RL. Prior studies show that intrinsic motivation can drive the development of complex behaviors 24,54 .”
Novelty and Beyond: Towards Combined Motivation Models and Integrated Learning Architectures
Merrick, K. E. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_9 · Cited by 12
Modeling of Cognitive Evolution: Perspective Direction of Interdisciplinary Investigation
Red’ko, V. G. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.12.207 · Cited by 9
“Agents (modeled organisms) can adapt to changing environment [6], have ability to perform the reinforcement learning (RL) [7] and hierarchical RL [8, 9].”
BackgroundMOTIVATION OF PARTICIPANTS IN CROWDSOURCING PLATFORMS USING INTELLIGENT AGENTS
Anand, V., & Hota, C. (2020) · International Journal of Computing · doi:10.47839/ijc.19.1.1696 · Cited by 9
Madeleine: Poetry and Art of an Artificial Intelligence
Revell, G. (2022) · Arts · doi:10.3390/arts11050083 · Cited by 11
Individual and Collective Self-Development: Concepts and Challenges
Lippi, M., Mariani, S., Martinelli, M., & Zambonelli, F. (2022) · Annals of Computer Science and Information Systems · doi:10.15439/2022f301 · Cited by 9
“Reinforcement learning approaches based on intrinsic rewards [20], instead, more closely exploit the idea of exploring the world to develop a sense of agency.”
BackgroundA Definition of Happiness for Reinforcement Learning Agents
Daswani, M., & Leike, J. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-21365-1_24 · Cited by 9
“Schmidhuber’s work on ‘intrinsic motivation’ adds a related component to the reward in order to motivate the agent to explore in interesting directions [9].”
BackgroundShaker, N. (2016) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2016.7860450 · Cited by 10
“developing skills through play and is closely tied to intrinsic motivation [23].”
BackgroundMultiscale Computation and Dynamic Attention in Biological and Artificial Intelligence
Badman, R. P., Hills, T. T., & Akaishi, R. (2020) · Brain Sciences · doi:10.3390/brainsci10060396 · Cited by 13
Toward a 'Standard Model' of Machine Learning
Hu, Z., & Xing, E. P. (2022) · Harvard Data Science Review · doi:10.1162/99608f92.1d34757b · Cited by 16
“…could naturally inspire mutual exchange between the research areas of intrinsic reward and active learning, for example, using the active learning informativeness measure as the intrinsic reward rin, as was studied in earlier work (L. Li et al., 2011; Pathak et al., 2019; Schmidhuber, 2010).”
BackgroundMethodMann, T., Park, Y., Jeong, S., Lee, M., & Choe, Y. (2012) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2012.2216524 · Cited by 9
Empowerment-driven Exploration using Mutual Information EstimationPreprint
Kumar, N. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.05533 · Cited by 9
Beyond Markov Chains, Towards Adaptive Memristor Network-based Music GenerationPreprint
Gale, E., Matthews, O., Costello, B. D. L., & Adamatzky, A. (2013) · Explore Bristol Research · doi:10.48550/arxiv.1302.0785 · Cited by 9
“formalising creativity and describing the actions of creative A.I. agents requires two modes of learning: A. an adaptive predictor or model of the growing data history and B. a reinforcement learner [25]. The agent learns about the world around it, compresses and stores that data and as such makes predictions about th…”
BackgroundExplore and Control with Adversarial SurprisePreprint
Fickinger, A., Jaques, N., Parajuli, S., Chang, M., Rhinehart, N., Berseth, G., Russell, S., & Levine, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.07394 · Cited by 9
“In stochastic environments for example, such methods can suffer from the “noisy TV” problem where the agent becomes distracted by inherently high-entropy elements (Schmidhuber, 2010).”
BackgroundMethodReward Design for Reinforcement Learning AgentsPreprint
Devidze, R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.21949 · Cited by 9
Associative Learning and Active Inference
Anokhin, P., Sorokin, A., Burtsev, M., & Friston, K. J. (2024) · Neural Computation · doi:10.1162/neco_a_01711 · Cited by 9
Divergent Perception: Framing Creative Cognition Through the Lens of Sensory Flexibility
Bellemare‐Pepin, A., & Jerbi, K. (2024) · The Journal of Creative Behavior · doi:10.1002/jocb.1525 · Cited by 8
A Curious Emergence of Reaching
Gordon, G., & Ahissar, E. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-32527-4_1 · Cited by 9
Online-learning and planning in high dimensions with finite element goal babbling
Loviken, P., & Hemion, N. (2017) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2017.8329814 · Cited by 8
Exploring to learn visual saliency: The RL-IAC approach
Craye, C., Lesort, T., Filliat, D., & Goudou, J.-F. (2018) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.11.012 · Cited by 8
Noisy Agents: Self-supervised Exploration by Predicting Auditory Events
Gan, C., Chen, X., Isola, P., Torralba, A. B., & Tenenbaum, J. B. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981614 · Cited by 9
“en an active research topic for decades. There are various solutions that have been investigated for encouraging the agent to explore novel states, including rewarding information gain [24], surprise [46, 47], state visitation counts [55, 6], empowerment [22], curiosity [43, 7] disagreement [44] and so on. A separate…”
BackgroundEvaluación de políticas públicas con técnicas de gamificación para la educación ciudadana
Torres-Toukoumidis, Á. (2016) · Repositorio Institucional de la Universidad de Huelva (Universidad de Huelva) · Cited by 24
Natural language syntax complies with the free-energy principlePreprint
Murphy, E., Holmes, E., & Friston, K. J. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/r3fcx · Cited by 8
Colas, C. (2021) · theses.fr (ABES) · doi:10.70675/58e02a71z71f7z4670z9e24ze039648eac26 · Cited by 8
Yiu, E., Allen, K. R., Ginosar, S., & Gopnik, A. (2026) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2025.0003 · Cited by 8
Exploring AI-based Computational Models of Novelty to Encourage Curiosity in Student Learning
Mohseni, M., Maher, M. L., Grace, K. S., Siddiqui, S., & Najjar, N. A. (2024) · SN Computer Science · doi:10.1007/s42979-024-02837-x · Cited by 7
“These approaches are the basis for computational models of curiosity resulting in AI systems that value the unexplored areas [9–12].”
A Neural Network Account of Kant’s Philosophical Aesthetics
Graben, P. B. (2024) · Mind and Matter · doi:10.53765/mm2024.227 · Cited by 7
“This is achieved by reinforcement learning (Williams 1988, Hertz et al. 1991, Schmidhuber 2010, 2012) through an adversarial minimax game , played by the interacting modules (Russell and Norvig 2010).”
InfluentialCompression Progress-Based Curiosity Drive for Developmental Learning
Mark, R. (2011) · Frontiers in Computational Neuroscience · doi:10.3389/conf.fncom.2011.52.00003 · Cited by 7
“A principled way of motivating such agents is to use subjective compression progress [2] as an intrinsic reward for actions generating learnable but as-yet-unknown regularities in the observation stream.”
BackgroundUsing State Predictions for Value Regularization in Curiosity Driven Deep Reinforcement Learning
Brunner, G., Fritsche, M., Richter, O., & Wattenhofer, R. P. (2018) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai.2018.00015 · Cited by 7
Visceral afferent training in action: The origins of agency in early cognitive development
Corcoran, A. W., Feuerriegel, D., Robinson, J., & Perrykkad, K. (2024) · Developmental Review · doi:10.1016/j.dr.2024.101184 · Cited by 7
A social learning formalism for learners trying to figure out what a teacher wants them to do
Cederborg, T., & Oudeyer, P. (2014) · Paladyn Journal of Behavioral Robotics · doi:10.2478/pjbr-2014-0005 · Cited by 9
Can Machines Design? An Artificial General Intelligence Approach
Hein, A. M., & Condat, H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-97676-1_9 · Cited by 10
Selective Perception as a Mechanism to Adapt Agents to the Environment: An Evolutionary Approach
Ramičić, M., & Bonarini, A. (2019) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2019.2896306 · Cited by 7
Variational Free Energy and Economics Optimizing With Biases and Bounded Rationality
Henriksen, M. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.549187 · Cited by 10
“This is an important quantity in optimal Bayesian design (Lindley, 1956) and many other fields, where it is variously called salience, intrinsic value, intrinsic motivation, and value of information (Oudeyer and Kaplan, 2007; Itti and Baldi, 2009; Schmidhuber, 2010; Barto et al., 2013).”
BackgroundVaskevich, A., & Torres, E. B. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.1033776 · Cited by 8
“…intrinsic motivation and curiosity has become a dominant theme in machine learning and artificial intelligence over the past years (Daw et al., 2006; Baranes and Oudeyer, 2009; Schmidhuber, 2010; Still and Precup, 2012; Little and Sommer, 2013; Friston et al., 2017; Schwartenbeck et al., 2019).”
BackgroundEfficient Interactive Multiclass Learning from Binary Feedback
Ngo, H., Luciw, M., Nagi, J., Förster, A., Schmidhuber, J., & Vien, N. A. (2014) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2629631 · Cited by 7
“In this sense, one can even consider this a form of arti.cial curiosity [Schmidhuber 2010; Ngo et al. 2012, 2013a].”
BackgroundSelf-citationAutonomous reinforcement of behavioral sequences in neural dynamics
Kazerounian, S., Luciw, M., Richter, M., & Sandamirskaya, Y. (2013) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2013.6706877 · Cited by 7
“One could imagine future work using more sophisticated exploration methods [26].”
MethodSimple Aesthetic Sense and Addiction Emerge in Neural Relations of Cost-Benefit Decision in Foraging
Gribkova, E., Catanho, M., & Gillette, R. (2020) · Scientific Reports · doi:10.1038/s41598-020-66465-0 · Cited by 8
An Adversarial Objective for Scalable Exploration
Bucher, B., Schmeckpeper, K., Matni, N., & Daniilidis, K. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636298 · Cited by 8
“This behavioral paradigm is considered a form of curiosity [16], [18], [35], [36].”
BackgroundMethodAutonomous exploration for navigating in non-stationary CMPsPreprint
Gajane, P., Ortner, R., Auer, P. L., & Szepesvári, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.08446 · Cited by 7
“…in the area includes that of Schmidhuber (1991); Singh et al. (2004); Oudeyer and Kaplan (2007); Oudeyer et al. (2007); Baranes and Oudeyer (2009); Schmidhuber (2010); Singh et al. (2010); Lopes et al. (2012); Gottlieb et al. (2013); Stadie et al. (2015); Houthooft et al. (2016); Achiam and…”
BackgroundNovelty is not Surprise: Human exploratory and adaptive behavior in sequential decision-makingPreprint
He, X., Modirshanechi, A., Lehmann, M. P., Gerstner, W., & Herzog, M. H. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.09.24.311084 · Cited by 9
“In psychology, novelty is considered as an intrinsic motivational signal [17, 18] and in reinforcement learning as an exploration bonus [10, 19, 20].”
BackgroundCreativity, Compositionality, and Common Sense in Human Goal GenerationPreprint
Davidson, G., Gureckis, T. M., & Lake, B. M. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/byzs5 · Cited by 12
Embodied intelligence for drumming; a reinforcement learning approach to drumming robots
Karbasi, S. M., Jensenius, A. R., Godøy, R. I., & Tørresen, J. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1450097 · Cited by 9
Active control over exploration improves memory in toddlers
Li, Y.-L., Poli, F., & Ruggeri, A. (2025) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2024.2555 · Cited by 6
“This attentional capture can be characterized in terms of informational surprise [36], with infants showing the most attention to situations of intermediate visual complexity, supposedly to avoid wasting cognitive resources trying to process overly simple or overly complex events [37].”
Barbieri, P., Berto, M., Sarasso, P., Frascaroli, J., Handjaras, G., Piovesan, F., Gnecco, G., Bottari, D., & Ronga, I. (2025) · PNAS Nexus · doi:10.1093/pnasnexus/pgaf288 · Cited by 6
“…pleasure when we are particularly effective in processing sensory information (i.e. we process information flu-ently, find new patterns in our sensorium, make larger updates to our models of the world, experience faster reduction in uncertainty about the causes of our sensations) (7, 58 – 60).”
Philosophers & Futurists, Catch Up! Response to The Singularity
Schmidhuber, J. (2012) · Cited by 8
Self-citationLin, H., He, Y., Li, F., Liu, Q., Wang, B., & Zhu, F. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122145 · Cited by 8
Turing Revisited: A Cognitively-Inspired Decomposition
Besold, T. R. (2012) · Studies in applied philosophy, epistemology and rational ethics · doi:10.1007/978-3-642-31674-6_9 · Cited by 6
“[25]) or the Inventive Machine project [29].”
Emergent vision system of autonomous virtual characters
Nogueira, Y. L. B., de Brito, C. E. F., Vidal, C. A., & Neto, J. B. C. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.09.061 · Cited by 6
“What the RL fi eld calls curiosity is rewarding the agent for fi nding patterns in the environment [10].”
BackgroundNeural Game Engine: Accurate learning of generalizable forward models from pixels
Bamford, C., & Lucas, S. M. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231688 · Cited by 15
Adherence Improves Cooperation in Sequential Social Dilemmas
Yuan, Y., Guo, T., Zhao, P., & Jiang, H. (2022) · Applied Sciences · doi:10.3390/app12168004 · Cited by 8
Making and breaking symmetries in mind and life
Safron, A., Sakthivadivel, D. A. R., Sheikhbahaee, Z., Bein, M., Razi, A., & Levin, M. G. (2023) · Interface Focus · doi:10.1098/rsfs.2023.0015 · Cited by 7
“The sense of aesthetic beauty accompanying such explanations is reminiscent of our understanding of intelligence in terms of the ability to efficiently predict (or compress) data [6,7]; indeed, identifying and building on symmetry can offer a particularly elegant”
BackgroundPredictive World Models from Real-World Partial Observations
Karlsson, R., Carballo, A., Fujii, K., Ohtani, K., & Takeda, K. (2023) · IEEE International Conference on Mobility, Operations, Services and Technologies (MOST) · doi:10.1109/most57249.2023.00024 · Cited by 8
“The problem of learning such simulations is called predictive world modeling in machine learning [6], [7], [12].”
BackgroundA robot to study the development of artwork appreciation through social interactions
Karaouzene, A., Gaussier, P., & Vidal, D. (2013) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2013.6652554 · Cited by 8
“But even at this stage, our approach is different from most other studies about the exploration of aesthetics in robots [4] where the ability to display some sort of aesthetic sense is generally considered as an agent’s intrinsic property - whether it is a human being or a machine - rather than a result of a more gene…”
ResultContinually adding self-invented problems to the repertoire: First experiments with POWERPLAY
Srivastava, R. K., Steunebrink, B. R., Stollenga, M., & Schmidhuber, J. (2012) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2012.6400843 · Cited by 9
“optimal creative agents [20], [21], POWERPLAY’s task search is greedy, yet practically feasible.”
BackgroundMethodSelf-citationScaling All-Goals Updates in Reinforcement Learning Using Convolutional Neural Networks
Pardo, F., Levdik, V., & Kormushev, P. S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5983 · Cited by 6
Reinforcement active learning in the vibrissae system: Optimal object localization
Gordon, G., Dorfman, N., & Ahissar, E. (2012) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2012.06.004 · Cited by 6
“This prediction error is used to adjust the learner parameters, but here it also serves as the intrisic reward signal (Barto et al., 2004; Weng, 2004; Oudeyer et al., 2007; Schmidhuber, 2010).”
BackgroundMethodInfluentialNgo, H. Q., Luciw, M., Vien, N. A., & Schmidhuber, J. (2013) · Research Portal (Queen's University Belfast) · Cited by 6
Self-citationInfluence-Based Reinforcement Learning for Intrinsically-Motivated AgentsPreprint
Fayad, A., & Ibrahim, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.12581 · Cited by 6
Improving Structure of the SLMHM on the Base of the PDCA CyclePreprint
Gakh, D. B. (2022) · Preprints.org · doi:10.20944/preprints202104.0490.v2 · Cited by 6
Simulus: Combining Improvements in Sample-Efficient World Model AgentsPreprint
Cohen, L., Wang, K., Kang, B., Gadot, U., & Mannor, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.11537 · Cited by 7
“An effective approach to this challenge involves estimating the world model’s epistemic uncertainty and directing the controller to gather data from regions where this uncertainty is highest [Schmidhuber, 2010, Shyam et al., 2019, Sekar et al., 2020].”
Forestier, S. (2019) · theses.fr (ABES) · doi:10.70675/abd58967z79d0z4df2zbe94z307821e9b628 · Cited by 7
Filho, R. I. O. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.28886 · Cited by 6
“Schematic memory as reconstruction [1]; unfinished tasks as retained tension [16]; compression progress as intrinsic reward [10].”
Deterding, S., Guckelsberger, C., Lintunen, E. M., & Ady, N. M. (2025) · Motivation and Emotion · doi:10.1007/s11031-025-10156-8 · Cited by 6
Sakamoto, Y., Uchida, T., & Ishiguro, H. (2025) · Scientific Reports · doi:10.1038/s41598-025-25531-1 · Cited by 9
“Studies on agent architectures that integrate internal states and reflection 3,29 , research on intrinsic motivation in artificial intelligence 30,31 , and the use of explicit prompting strategies point toward ways of endowing agents with human-like motivational structures.”
Algebraic Cekirge Method for Deterministic and Energy-efficient Transformer Language Models
Cekirge, H. M. (2025) · American Journal of Artificial Intelligence · doi:10.11648/j.ajai.20250902.25 · Cited by 5
Yin, J., Yuan, T., Zhang, Y., Yang, G., & Wang, X. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1735465 · Cited by 5
“Meanwhile, although the proportion of intrinsic motivation’s effect is the smallest, it remains a factor that cannot be overlooked, as long-term creative behavior often requires the support of intrinsic interest (Schmidhuber, 2010).”
Binary Space Partitioning as Intrinsic Reward
Skaba, W. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35506-6_29 · Cited by 5
“[2], [3].”
Moulin-Frier, C., & Oudeyer, P. (2014) · HAL · Cited by 7
Encouraging p-creative behaviour with computational curiosity
Grace, K. S., Maher, M. L., Mohseni, M., & y Pérez, R. P. (2017) · ICCC · Cited by 5
Nagashima, K., Morita, J., & Takeuchi, Y. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1397860 · Cited by 6
“As we discussed in Section 3.1, this study targeted on intellectual curiosity relating “a desire to bring better form to one’s knowledge structures” (Malone, 1981) or “intrinsic desire to build a better model for the world” (Schmidhuber, 2010).”
BackgroundResultInfluentialSchröder, T., & Diedrich, C. (2019) · at - Automatisierungstechnik · doi:10.1515/auto-2019-0030 · Cited by 5
Evolutionary Computation and the Reinforcement Learning Problem
Kelly, S., & Schossau, J. (2023) · Genetic and evolutionary computation · doi:10.1007/978-981-99-3814-8_4 · Cited by 5
The Inherent Normativity of Concepts
So, W. Y., Friston, K. J., & Neacsu, V. (2024) · Minds and Machines · doi:10.1007/s11023-024-09697-7 · Cited by 5
“To avoid expending excessive energy on processing information that may be redundant, the agent must engage in some form of data compression (Hinton & Zemel, 1993; Schmidhuber, 1991, 2010; Wallace & Dowe, 1999).”
Intrinsic Motivation as Constrained Entropy Maximization
Kiefer, A. B. (2025) · Entropy · doi:10.3390/e27040372 · Cited by 9
“The idea that entropy is maximized “for its own sake” does not, of course, preclude interpretations of this phenomenon in terms of epistemic value [8], curiosity [12], and so on, in various contexts.”
Empowerment as a Generic Utility Function for Agents in a Simple Team Sport Simulation
Clements, M., & Polani, D. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66471-2_5 · Cited by 6
“This can be seen as a drastic abstraction of a “ball” attracting an agent by pure virtue of being able to be kicked around; in a way, an agent detecting that “playing football” with a box is “fun”[22].”
BackgroundThe Role of Social Media in Higher Education
Montebello, M., & Camilleri, V. (2018) · International Journal of Learning and Teaching · doi:10.18178/ijlt.4.4.311-317 · Cited by 5
Policy Shaping with Supervisory Attention Driven Exploration
Faulkner, T. K., Short, E. S., & Thomaz, A. L. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594312 · Cited by 5
Learning the Arrow of Time for Problems in Reinforcement Learning
Rahaman, N., Wolf, S., Goyal, A., Remme, R., & Bengio, Y. (2020) · International Conference on Learning Representations · Cited by 5
Curiosity Driven Deep Reinforcement Learning for Motion Planning in Multi-agent Environment
Perovic, G., & Li, N. (2019) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio49542.2019.8961660 · Cited by 5
Hierarchical Interest-Driven Goal Babbling for Efficient Bootstrapping of Sensorimotor skills
Rayyes, R., Donat, H., & Steil, J. J. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196763 · Cited by 6
Regulating Ruminative Web Browsing Based on the Counterbalance Modeling Approach
Morita, J., Pitakchokchai, T., Raj, G. B., Yamamoto, Y., Yuhashi, H., & Koguchi, T. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.741610 · Cited by 6
Neural foundations of creativity: A systematic review
Raymond, S. M. (2017) · Revista Colombiana de Psiquiatría (English ed ) · doi:10.1016/j.rcpeng.2016.06.001 · Cited by 5
Progress and challenges in adaptive robotics
Nolfi, S. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.1020462 · Cited by 5
“Intrinsic motivation (Badia et al., 2010; Schmidhuber, 2010) achieves this objective by rewarding the robots also for displaying new behaviors and/or experiencing new observations.”
BackgroundIntrinsic motivation learning for real robot applications
Rayyes, R. (2023) · Frontiers in Robotics and AI · doi:10.3389/frobt.2023.1102438 · Cited by 5
“…through driving the robot’s learning and exploration autonomously by internally generated signals in an open-ended (i.e., unbounded) environment (Schmidhuber, 2010; Baranes and Oudeyer, 2013; Santucci et al., 2016; Baldassarre, 2019; Rayyes et al., 2020a; Rayyes, 2020; Rayyes et al., 2021).”
BackgroundMethodInfluentialNagashima, K., Morita, J., & Takeuchi, Y. (2021) · Institutional Repositories DataBase (IRDB) · Cited by 5
A cognitive basis for theories of intrinsic motivation
Srivastava, N., Kapoor, K., & Schrater, P. (2011) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2011.6037327 · Cited by 5
From motor to visually guided bimanual affordance learning
Sánchez-Fibla, M., Forestier, S., Moulin-Frier, C., Puigbò, J.-Y., & Verschure, P. F. M. J. (2019) · Adaptive Behavior · doi:10.1177/1059712319855836 · Cited by 5
“…(Berlyne, 1954; Deci & Ryan, 1985), have been transposed in curiosity-driven learning machines (Barto, Singh, & Chenatez, 2004; Schmidhuber, 1991; Schmidhuber, 2010) and robots (Baranes & Oudeyer, 2012; Oudeyer et al., 2007) and shown to yield highly efficient learning of inverse models in…”
BackgroundModified PPO-RND Method for Solving Sparse Reward Problem in ViZDoom
Chen, J.-C., & Chang, T.-H. (2019) · IEEE Conference on Games (CoG) · doi:10.1109/cig.2019.8847999 · Cited by 5
Towards intrinsic autonomy through evolutionary computation
Nogueira, Y. L. B., de Brito, C. E. F., Vidal, C. A., & Cavalcante-Neto, J. B. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-019-09798-1 · Cited by 9
“Such technique is based on rewarding the curiosity of agents, i.e., the search for novel states (Barto et al. 2004; Schmidhuber 2010; Kompella et al. 2012; Barto 2013; Gay et al. 2016).”
BackgroundMethodDeep Reinforcement Learning From Raw Pixels in DoomPreprint
Hafner, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.02164 · Cited by 5
“It therefore tries to provide new observations to the model that are novel but learnable, inspired by the way humans are bored by both known knowledge and knowledge they cannot understand [20].”
BackgroundKompella, V. R., & Wiskott, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.04663 · Cited by 5
What deep learning can tell us about higher cognitive functions like mindreading?Preprint
Aru, J., & Vicente, R. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/9skeq · Cited by 5
Choi, J., & Yoon, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.03116 · Cited by 5
“The idea of integrating curiosity, and its counterpart boredom, with reinforcement learning was suggested by (Schmidhuber, 1991), and showed that intrinsic reward can be modeled to motivate agents to explore areas with high prediction errors (Schmidhuber, 2010).”
Background
Metadata
- Authors
- Schmidhuber, J.
- Affiliations
- Dalle Molle Institute for Artificial Intelligence Research (CH)
- Corresponding author
- Schmidhuber, Jürgen
- ORCID
- Schmidhuber, Jürgen
- Venue
- IEEE Transactions on Autonomous Mental Development
- Published
- 2010-09
- Volume
- 2
- Issue
- 3
- Pages
- 230-247
- ISSN
- 1943-0604 (ISSN-L)
- 1943-0612
- OpenAlex
- W2034806191
- Cited by
- 747
- References
- 126
- Citations by year
- 2026: 29 · 2025: 64 · 2024: 46 · 2023: 50 · 2022: 60 · 2021: 69 · 2020: 92 · 2019: 72 · 2018: 55 · 2017: 37 · 2016: 26 · 2015: 24 · 2014: 30 · 2013: 28 · 2012: 41
- FWCI
- 31.09
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Autonomous Mental Development (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
- Keywords
- intrinsic motivation, creativity, fun, intrinsic rewards, active learning (machine learning), novelty seeking, surprise, pattern discovery, data compression, online prediction, autonomous agents, robots, aesthetics, humor, intelligence, limited computational resources, continuous-time formulation, discrete-time formulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.