Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010)
The DOI matches its Crossref record; 125 of the 126 references it cites were found in a registry or an index.
Abstract
The simple, but general formal theory of fun and intrinsic motivation and creativity (1990-2010) is based on the concept of maximizing intrinsic reward for the active creation or discovery of novel, surprising patterns allowing for improved prediction or data compression. It generalizes the traditional field of active learning, and is related to old, but less formal ideas in aesthetics theory and developmental psychology. It has been argued that the theory explains many essential aspects of intelligence including autonomous development, science, art, music, and humor. This overview first describes theoretically optimal (but not necessarily practical) ways of implementing the basic computational principles on exploratory, intrinsically motivated agents or robots, encouraging them to provoke event sequences exhibiting previously unknown, but learnable algorithmic regularities. Emphasis is put on the importance of limited computational resources for online prediction and compression. Discrete and continuous time formulations are given. Previous practical, but nonoptimal implementations (1991, 1995, and 1997-2002) are reviewed, as well as several recent variants by others (2005-2010). A simplified typology addresses current confusion concerning the precise nature of intrinsic motivation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This underwrites Bayesian experimental design [134] and active learning in statistics [135], intrinsic motivation and artificial curiosity in machine learning and robotics [136, 137, 138, 139, 140].”
From The free energy principle made simpler but not too simple (2023) · cites this paper as [137]
In the citing paper
“This suggests that risk-averse policies minimise expected complexity or computational cost [137].”
From The free energy principle made simpler but not too simple (2023) · cites this paper as [137]
In the citing paper
“Ideally, the learner learns to choose actions that will lead to the largest improvements in its prediction or data compression performance (Schmidhuber, 2010).”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Schmidhuber, 2010
In the citing paper
“In Schmidhuber (2010), this is done in the framework of reinforcement learning, and incorporates a mechanisms for the system to measure its own rate of learning.”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Schmidhuber, 2010
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 7×
“Another potential drawback is the nature of the particular RNN-based RL method.”
BackgroundInfluentialSelf-citationA Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,002 · cited 1×
“iii) An evolutionary algorithm [22], [29], [59],”
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 7×
“Some Q-learning accelerators simply update pairs of actions and states with currently quickly changing Q-values more frequently than others (that is, Q-values with high first derivatives are favored).”
BackgroundMethodInfluentialReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Neural networks for pattern recognition
(1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,538
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Neural Networks for Pattern Recognition
Bishop, C. M. (1995) · doi:10.1093/oso/9780198538493.001.0001 · Cited by 12,287
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · cited 4×
“, some typically suboptimal RL method [35].”
BackgroundMethodInfluential
Research works printed but not linked: 24
- kullback, 1959, Statistics and Information Theory
- hutter, 2004, Universal Artificial Intelligence Sequential Decisions Based on Algorithmic Probability
- holland, 1975, Adaptation in Natural and Artificial Systems
- schmidhuber, 2010, How to Learn Programs with Artificial Recurrent Neural Networks
- harlow, 1950, novelty and curiosity as determinants of exploratory behavior, J Exp Psychol, 41, 68
- berlyne, 1950, novelty and curiosity as determinants of exploratory behavior, Brit J Psychol, 41, 68
- 10.1007/bf00993591 · doi:10.1007/bf00993591 · Paper page
- levin, 1973, universal sequential search problems, Problems Inform Trans, 9, 265
- schmidhuber, 1991, Advances in Neural Information Processing Systems 3, nips 3, 500
- schmidhuber, 1997, Femmes Fractales
- schmidhuber, 2002, Advances in Evolutionary Computing, 579
- ring, 1994, Continual Learning in Reinforcement Environments
- schmidhuber, 1990, Dynamische neuronale Netze und das fundamentale raumzeitliche Lernproblem
- schmidhuber, 1991, Adaptive Curiosity and Adaptive Confidence
- schmidhuber, 2009, Compression Progress The Algorithmic Principle Behind Curiosity and Creativity (With Applications of the Theory of Humor) 40 min Video of Invited Talk at Singularity Summit 2009
- schmidhuber, 2009, Multiple Ways to Design Research Research Cases that Reshape the Design Discipline, 98
- singh, 2005, Advances in neural information processing systems, 17
- schwefel, 1974, Numerische Optimierung von Computer-Modellen
- schmidhuber, 2007, alle berechenbaren universen (all computable universes), Spektrum der Wissenschaft, 75
- storck, 1995, reinforcement driven information acquisition in non-deterministic environments, Proc Int Conf Artif Neural Netw, 2, 159
- brafman, 2002, R-MAX—A general polynomial time algorithm for near-optimal reinforcement learning, J Mach Learn Res, 3, 213
- frank, 2002, sthetische Information Estetika informacio Eine Einfhrung in die Kybernetische sthetik
- franke, 1979, Kybernetische sthetik Phnomen Kunst
- schmidhuber, 2006, Artificial General Intelligence, 175
Cited by
Items 301–400 of 886
Model Based Planning with Energy Based ModelsPreprint
Du, Y., Lin, T., & Mordatch, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06878 · Cited by 49
“A common approach to achieve exploration behavior in reinforcement learning has been to use explicit rewards, known as intrinsic motivation [24, 25].”
BackgroundSelf Exploration of Sensorimotor Spaces in Robots.
Benureau, F. (2015) · theses.fr (ABES) · doi:10.70675/1d482feaz6e01z4426zbc1cz25388edcb189 · Cited by 19
General intelligence requires rethinking exploration
Jiang, M., Rocktäschel, T., & Grefenstette, E. (2022) · arXiv · doi:10.48550/arxiv.2211.07819 · Cited by 17
“Notably, both Schmidhuber [10,32,33] and Clune [34] have proposed novelty-seeking agents that generate their own challenges, while the work of Stanley and Lehman established the practical value of taming open-endedness [35 – 37].”
BackgroundSpiekermann, S. (2021) · Springer Reference Geisteswissenschaften · doi:10.1007/978-3-658-23715-8_35-1 · Cited by 17
Unified curiosity-Driven learning with smoothed intrinsic reward estimation
Huang, F., Li, W., Cui, J., Fu, Y., & Li, X. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108352 · Cited by 18
Emerging social awareness: Exploring intrinsic motivation in multiagent learning
Sequeira, P., Melo, F. S., Prada, R., & Paiva, A. (2011) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2011.6037325 · Cited by 22
“Several parallels have been drawn between intrinsic motivation systems in the psychology literature and some active learning and experimental design techniques from machine learning [2]-[4].”
BackgroundGoal-Directed Empowerment: Combining Intrinsic Motivation and Task-Oriented Behavior
Volpi, N. C., & Polani, D. (2020) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.3042938 · Cited by 28
Brändle, F., Binz, M., & Schulz, E. (2022) · Cambridge University Press eBooks · doi:10.1017/9781009026949.008 · Cited by 18
Personalised Specific Curiosity for Computational Design Systems
Grace, K. S., Maher, M. L., Wilson, D. C., & Najjar, N. A. (2017) · Springer eBooks · doi:10.1007/978-3-319-44989-0_32 · Cited by 20
FUNIFICATION 2.0: Knowledge mobilization model for corporate and educational game-based learning
Jorge, C. F. B., & Sutton, M. J. D. (2017) · World Journal of Science Technology and Sustainable Development · doi:10.1108/wjstsd-11-2016-0060 · Cited by 16
Automated Curriculum Learning by Rewarding Temporally Rare Events
Justesen, N., & Risi, S. (2018) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490448 · Cited by 18
Explore and Explain: Self-supervised Navigation and Recounting
Bigazzi, R., Landi, F., Cornia, M., Cascianelli, S., Baraldi, L., & Cucchiara, R. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412628 · Cited by 22
Liakoni, V., Lehmann, M. P., Modirshanechi, A., Brea, J., Lutti, A., Gerstner, W., & Preuschoff, K. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118780 · Cited by 18
A Perspective on Lifelong Open-Ended Learning Autonomy for Robotics through Cognitive Architectures
Romero, A., Bellas, F., & Duro, R. J. (2023) · Sensors · doi:10.3390/s23031611 · Cited by 22
“These approaches have been used in a wide variety of applications such as state-space exploration [37–39], knowledge gathering [40,41], autonomous skill learning [18,42–44] or autonomous goal selection [15,18].”
BackgroundThe two-dimensional organization of behavior
Ring, M. B., Schaul, T., & Schmidhuber, J. (2011) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2011.6037326 · Cited by 18
“Thus, it is reasonable to ask, how can the mot system be combined with mechanisms for curiosity [24], [25]? Curiosity could be pursued at either the global or mot level.”
Self-citationLearning and Consolidation as Re-representation: Revising the Meaning of Memory
Wiggins, G. A., & Sanjekdar, A. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00802 · Cited by 17
“Other on-going work in the area relates to the idea of “free energy” (Friston, 2010) and intrinsic motivation (Schmidhuber, 2010).”
BackgroundLearning Representations for Pixel-based Control: What Matters and Why?
Tomar, M., Mishra, U. A., Zhang, A., & Taylor, M. E. (2023) · Transactions on machine learning research · doi:10.48550/arxiv.2111.07775 · Cited by 21
“A related line of work learns representations by constructing world models directly from pixels [44, 40, 21, 24].”
BackgroundIntrinsic Motivation in Dynamical Control Systems
Tiomkin, S., Nemenman, I. M., Polani, D., & Tishby, N. (2024) · PRX Life · doi:10.1103/prxlife.2.033009 · Cited by 27
Confidence-based progress-driven self-generated goals for skill acquisition in developmental robots
Ngo, H., Luciw, M., Förster, A., & Schmidhuber, J. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00833 · Cited by 16
Self-citationMeola, V. C., Caligiore, D., Sperati, V., Zollo, L., Ciancio, A. L., Taffoni, F., Guglielmelli, E., & Baldassarre, G. (2015) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tamd.2015.2494460 · Cited by 17
“possible on the basis of intrinsic motivations [91], [115], [123],”
BackgroundBaldassarre, G., & Mirolli, M. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_11 · Cited by 19
“Intrinsic motivations (IM) are receiving increasing attention for their potential to allow organisms and robots to acquire knowledge and skills cumulatively and in full autonomy (Baldassarre and Mirolli 2010; Barto et al. 2004; Deci et al. 2001; Oudeyer and Kaplan 2007; Schmidhuber 2010).”
BackgroundAutonomous learning of abstractions using Curiosity-Driven Modular Incremental Slow Feature Analysis
Kompella, V. R., Luciw, M., Stollenga, M., Pape, L., & Schmidhuber, J. (2012) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2012.6400829 · Cited by 20
“The Formal Theory of Fun and Creativity [15] mathematically formalizes driving forces behind all kinds of curious and creative behavior.”
BackgroundSelf-citationDuminy, N., Nguyen, S. M., Zhu, J., Duhaut, D., & Kerdreux, J. (2021) · Applied Sciences · doi:10.3390/app11030975 · Cited by 19
“In continual learning in an open-ended world without external rewards, to discover repertoires of skills, agents must be endowed with intrinsic motivation (IM), which is described in psychology as triggering curiosity in human beings [12], to explore the diversity of outcomes it can cause and to control its environmen…”
BackgroundArtificial Curiosity for Autonomous Space Exploration
Graziano, V., Glasmachers, T., Schaul, T., Pape, L., Cuccu, G., Leitner, J., & Schmidhuber, J. (2011) · QUT ePrints (Queensland University of Technology) · doi:10.26180/19959179 · Cited by 15
Self-citationInteractive Language Learning by Question AnsweringPreprint
Yuan, X., Côté, M.-A., Fu, J., Lin, Z., Pal, C., Bengio, Y., & Trischler, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.10909 · Cited by 16
Reward Propagation Using Graph Convolutional NetworksPreprint
Klissarov, M., & Precup, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.02474 · Cited by 28
“…a shaping function, resulting in the following equation: R (cid:48) ( S t , A t , S t +1 ) = r ( S t , A t ) + F ( S t , S t +1 ) where F ( S t , S t +1 ) is the shaping function which can encode expert knowledge or represent concepts such as curiosity [Schmidhuber, 2010, Oudeyer and Kaplan, 2007].”
Efficient Empowerment Estimation for Unsupervised StabilizationPreprint
Zhao, R., Lu, K., Abbeel, P., & Tiomkin, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07356 · Cited by 19
“Intrinsic motivation allows artificial and biological agents to acquire useful behaviours without external knowledge (Barto et al. (2004); Chentanez et al. (2005); Schmidhuber (2010); Barto (2013); Oudeyer et al. (2016)).”
BackgroundHumans monitor learning progress in curiosity-driven explorationPreprint
Ten, A., Kaushik, P., Oudeyer, P., & Gottlieb, J. P. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/7dbr6 · Cited by 15
MERL: Multi-Head Reinforcement LearningPreprint
Flet-Berliac, Y., & Preux, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11939 · Cited by 15
“Auxiliary tasks have been adopted to facilitate representation learning for decades (Suddarth & Kergosien, 1990; Klyubin et al., 2005), along with intrinsic motivation (Schmidhuber, 2010; Pathak et al., 2017) and artificial curiosity (Schmidhuber, 1991; Oudeyer & Kaplan, 2007).”
MethodThe Benefits of Model-Based Generalization in Reinforcement LearningPreprint
Young, K., Ramesh, A., Kirsch, L., & Schmidhuber, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.02222 · Cited by 18
“On a practical note, there is significant work on exploration with sparse (or no) reward (Schmidhuber, 1991a;b; Thrun, 1992; Schmidhuber, 2010; Amin et al., 2021) which is orthogonal to our focus.”
BackgroundSelf-citationCognition all the way down 2.0: neuroscience beyond neurons in the diverse intelligence era
Chis-Ciure, R., & Levin, M. G. (2025) · Synthese · doi:10.1007/s11229-025-05319-6 · Cited by 15
“This complexity minimization is addressed in universal computation through the notion of compression, which figures in many accounts of efficiency, e.g., Schmidhuber ( 2010 ), Mehta et al. ( 2014 ), Ruffini (2017), Grün-wald and Roos ( 2019 ), and Friston et al. ( 2025 ).”
Dynamics-Aware Unsupervised Skill Discovery
Sharma, A., Gu, S., Levine, S., Kumar, V., & Hausman, K. (2020) · International Conference on Learning Representations · Cited by 21
Okada, A., & Sheehy, K. (2020) · Revista e-Curriculum · doi:10.23925/1809-3876.2020v18i2p590-613 · Cited by 14
Don’t Do What Doesn’t Matter: Intrinsic Motivation with Action Usefulness
Seurin, M., Strub, F., Preux, P., & Pietquin, O. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/406 · Cited by 15
Autonomous development of goals: From generic rewards to goal and self detection
Rolf, M., & Asada, M. (2014) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2014.6982980 · Cited by 14
“With respect to the autonomous development of goals already Prince [14] noted that goals are related to selfsupervision, but missed the point that it is not the goals themselves, but rather the self-detection (in relation to the goals) that enables the supervision.”
BackgroundHafez, M. B., Weber, C., Kerzel, M., & Wermter, S. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852254 · Cited by 17
“Examples of intrinsic reward functions include model prediction error [7], [8], model learning progress [9]–[12], change in policy value [13], and information-theoretic-based dynamics uncertainty [14].”
BackgroundMaximizing Fun by Creating Data with Easily Reducible Subjective Complexity
Schmidhuber, J. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_5 · Cited by 16
“Since 1990, agents were built that implement this idea (Schmidhuber 2010b).”
BackgroundMethodSelf-citationBoedecker, J., Lampe, T., & Riedmiller, M. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00739 · Cited by 19
Curiosity-Driven Multi-Criteria Hindsight Experience ReplayPreprint
Lanier, J. B., McAleer, S., & Baldi, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.03710 · Cited by 17
“We focus on exploration by performing actions that both challenge and improve an agent’s ability to model the world [21, 17, 38, 39].”
BackgroundGuided Exploration with Proximal Policy Optimization using a Single DemonstrationPreprint
Libardi, G., & Fabritiis, G. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.03328 · Cited by 22
“Self-supervised prediction approaches, such as [4, 3, 13, 1] have been used successfully in stochastic environments, although they are less effective in three-dimensional environments.”
MethodDevelopment of collective behavior in newborn artificial agentsPreprint
Lee, D., Wood, S. M. W., & Wood, J. N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.03796 · Cited by 15
“The second mechanism—curiosity-driven learning—can drive the development of complex behaviors through selfsupervised learning (34, 35).”
The neuroscientific basis of flow: Learning progress guides task engagement and cognitive control
Lu, H., van der Linden, D., & Bakker, A. B. (2025) · NeuroImage · doi:10.1016/j.neuroimage.2025.121076 · Cited by 13
“Research about intrinsic motivation highlights the importance of learning progress (LP) as an intrinsic motivator in human behavior 22–25 .”
Adaptive compression as a unifying framework for episodic and semantic memory
Nagy, D., Orbán, G., & Wu, C. M. (2025) · Nature Reviews Psychology · doi:10.1038/s44159-025-00458-6 · Cited by 22
“…selectively because of its cost, simulated learners with limited episodic memory capacity performed better in online structure learning when they selectively prioritized experiences with high Bayesian surprise 77–79 , compared to learners that applied the same capacity indiscriminately 39 (Fig.”
A Performance-Based Start State Curriculum Framework for Reinforcement Learning
Wöhlke, J., Schmitt, F., & van Hoof, H. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/llzi8531 · Cited by 23
“…works have suggested a variety of ideas to improve data-efficiency in sparse reward reinforcement learning [26]: Intrinsic motivation [5, 12, 15, 27, 32], diversity [7, 14], return decomposition [2], auxiliary tasks [16, 30], use of demonstrations [29, 31], or curriculum learning [8, 9, 19, 30].”
BackgroundAffordance and Active Inference
Friston, K. J. (2022) · Springer eBooks · doi:10.1007/978-3-031-08629-8_20 · Cited by 18
Explore to see, learn to perceive, get the actions for free: SKILLABILITY
Kompella, V. R., Stollenga, M., Luciw, M., & Schmidhuber, J. (2014) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889784 · Cited by 13
“Curious Dr. MISFA comprises Incremental Slow Feature Analysis (IncSFA; [12, 13]), and Artificial Curiosity (AC; [14, 15]), IncSFA is used for learning slow features incrementally from high-dimensional sensory input.”
MethodSelf-citationReinforcement active learning hierarchical loops
Gordon, G., & Ahissar, E. (2011) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2011.6033617 · Cited by 15
“Finally, each hierarchical loop learns its own critic and actor, where the latter controls (either directly or via an inverse model) only external actions and not internal states [ 24 ].”
BackgroundMethodInfluentialNeural Foundations of Creativity: A Systematic Review
Raymond, S. M. (2016) · Revista Colombiana de Psiquiatría · doi:10.1016/j.rcp.2016.06.003 · Cited by 19
Generative adversarial exploration for reinforcement learning
Hong, W., Zhu, M., Liu, M., Zhang, W., Zhou, M. Q., Yu, Y. H., & Sun, P. (2019) · International Conference on Distributed Artificial Intelligence · doi:10.1145/3356464.3357706 · Cited by 15
“Hence, most research work focuses on supplementing by intrinsic reward [19, 22], which is also called exploration bonus, as an alternative to reward engineering, mostly inspired by the concepts of curiosity and surprise [8, 21].”
BackgroundKnowledge-level Creativity in Game Design.
Smith, A. M., & Mateas, M. (2011) · ICCC · Cited by 14
Dark Control: Towards a Unified Account of Default Mode Function by Markov Decision ProcessesPreprint
Dohmatob, E., Dumas, G., & Bzdok, D. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/148890 · Cited by 13
“Such a surprise-guided reinforcement learning scheme has previously been advocated under the equivalent framework of information compression (Schmidhuber, 2010; Mohamed and Rezende, 2015).”
MethodSeparation of Concerns in Reinforcement LearningPreprint
van Seijen, H., Fatemi, M., Romoff, J., & Laroche, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.05159 · Cited by 13
Towards safe and sustainable reinforcement learning for real-time strategy games
Andersen, P., Goodwin, M., & Granmo, O. (2024) · Information Sciences · doi:10.1016/j.ins.2024.120980 · Cited by 12
Creativity and Perception: Unveiling the Role of Cross‐Modal Audiovisual Integration
Xiao, X., Wang, J., Shu, Y., & Tan, J. (2024) · The Journal of Creative Behavior · doi:10.1002/jocb.668 · Cited by 13
Survival in a world of complex dangers
Mobbs, D., Wise, T., Tashjian, S. M., Zhang, J., Friston, K. J., & Headley, D. B. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105924 · Cited by 14
Zhang, S., Tian, Y., Liu, Q., & Wu, H. (2025) · eLife · doi:10.7554/elife.92892.4 · Cited by 12
A Survey of Human Intelligence Augmented Artificial Intelligence: An Autonomous Driving Perspective
Li, S., Yuan, K., Tang, Z., Zhang, Y., Ji, P., Ge, Q., Du, S., Huang, Y., & Chen, H. (2025) · Automotive Innovation · doi:10.1007/s42154-025-00361-z · Cited by 13
“Infants’ intense intrinsic curiosity is a key driver of early cognitive development and plays a major role in the continuous enhancement of HI [132, 133].”
C-ELM: A Curious Extreme Learning Machine for Classification Problems
Wu, Q., & Miao, C. (2014) · Proceedings in adaptation, learning and optimization · doi:10.1007/978-3-319-14063-6_30 · Cited by 12
Online Safety Assurance for Learning-Augmented Systems
Rotman, N. H., Schapira, M., & Tamar, A. (2020) · ACM Workshop on Hot Topics in Networks (HotNets) · doi:10.1145/3422604.3425940 · Cited by 17
“Curiosity-based exploration in RL involves guiding the agent to unvisited states during training [35, 43].”
BackgroundCartoni, E., Montella, D., Triesch, J., & Baldassarre, G. (2023) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2023.3270081 · Cited by 13
“The field of developmental robotics [6], [7] has developed algorithms for autonomous learning based on intrinsic motivations (e.g., [8], [9], [10], [11], [12], [13], [14], [15], and [16]).”
BackgroundResource-Bounded Machines are Motivated to be Effective, Efficient, and Curious
Steunebrink, B. R., Koutník, J., Þórisson, K. R., Nivel, E., & Schmidhuber, J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39521-5_13 · Cited by 13
Self-citationToward Computational Motivation for Multi-Agent Systems and Swarms
Khan, M. M., Kasmarik, K., & Barlow, M. (2018) · Frontiers in Robotics and AI · doi:10.3389/frobt.2018.00134 · Cited by 14
“Schmidhuber (2010) proposed a typology of intrinsic motivation from a different perspective.”
BackgroundBebopNet: Deep neural models for personalized jazz improvisations
Hakimi, S. H., Bhonker, N., & El‐Yaniv, R. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.4245561 · Cited by 12
Interdependence as the key for an ethical artificial autonomy
Pianca, F., & Santucci, V. G. (2022) · AI & Society · doi:10.1007/s00146-021-01313-x · Cited by 12
“2020), knowledge gathering (Schmidhuber 2010), learning repertoire of skills (Singh et al.”
BackgroundMcCoy, J., Paul, L. A., & Ullman, T. D. (2019) · Oxford University Press eBooks · doi:10.1093/oso/9780190639679.003.0010 · Cited by 16
New Millennium AI and the Convergence of History: Update of 2012
Schmidhuber, J. (2012) · The frontiers collection · doi:10.1007/978-3-642-32560-1_4 · Cited by 12
“More recent, mathematically optimal, creative explor-ers [69, 75] are based on universal RL methods [32, 64, 70] that are not yet computationally tractable (Section 3).”
BackgroundInfluentialSelf-citationTowards Game-based Metrics for Computational Co-Creativity
Canaan, R., Menzel, S., Togelius, J., & Nealen, A. (2018) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490429 · Cited by 13
Towards Information-Seeking AgentsPreprint
Bachman, P., Sordoni, A., & Trischler, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.02605 · Cited by 12
Trick or TReAT: Thematic Reinforcement for Artistic TypographyPreprint
Tendulkar, P., Krishna, K., Selvaraju, R. R., & Parikh, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.07820 · Cited by 12
Hierarchical principles of embodied reinforcement learning: A reviewPreprint
Eppe, M., Gumbsch, C., Kerzel, M., Nguyen, P. D. H., Butz, M. V., & Wermter, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.10147 · Cited by 12
Collect & Infer -- a fresh look at data-efficient Reinforcement LearningPreprint
Riedmiller, M., Springenberg, J. T., Hafner, R., & Heess, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.10273 · Cited by 16
BYOL-Explore: Exploration by Bootstrapped Prediction
Guo, Z. D., Thakoor, S., Pîslar, M., Pires, B. A., Altché, F., Tallec, C., Saade, A., Calandriello, D., Grill, J.-B., Tang, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.08332 · Cited by 12
“While our approach belongs to the curiosity-driven exploration paradigm [50, 42, 49, 59, 5, 68, 60, 34, 51, 52, 2, 63], other exploration paradigms have also been proposed.”
BackgroundMethodExploration and preference satisfaction trade-off in reward-free learning
Sajid, N., Tigas, P., Zakharov, A., Fountas, Z., & Friston, K. (2021) · arXiv · doi:10.48550/arxiv.2106.04316 · Cited by 11
“Conceptually, open-ended learning [50, 60, 61], where agents are responsible for never-ending learning opportunities, is closest to our formulation.”
BackgroundA motivational-based learning model for mobile robots
Berto, L. M., Costa, P. D. P., Simões, A. D. S., Gudwin, R., & Colombini, E. L. (2024) · Cognitive Systems Research · doi:10.1016/j.cogsys.2024.101278 · Cited by 11
Specific curiosity as a cause and consequence of transformational creativity
Grace, K. S., & Maher, M. L. (2015) · ICCC · Cited by 26
Online Learning of Body Orientation Control on a Humanoid Robot Using Finite Element Goal Babbling
Loviken, P., Hemion, N., Laflaquière, A., Spranger, M., & Cangelosi, A. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593762 · Cited by 11
An experimental study of action selection mechanisms to create an entertaining opponent
Sephton, N., Cowling, P., & Slaven, N. H. (2015) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2015.7317939 · Cited by 11
“Schmidhuber’s formal definition of creativity and fun [16] states that the interest or fun associated with a given piece of information is related to the compressibility of that information, such that the compressibility should neither be too high (preventing easy compression through being semi-random), or too low (ma…”
BackgroundMethodInfluentialThe Theory of Neural Cognition Applied to Robotics
Touzet, C. F. (2015) · International Journal of Advanced Robotic Systems · doi:10.5772/60693 · Cited by 12
“In fact, the notion of joy only reflects the limited capacities of the observer [23].”
BackgroundExploring Through Random Curiosity with General Value Functions
Ramesh, A. A., Kirsch, L., van Steenkiste, S., & Schmidhuber, J. (2022) · neural information processing systems · doi:10.52202/068431-1361 · Cited by 11
Self-citationConstructivist ambient intelligent agent for smart environments
Najjar, A., & Reignier, P. (2013) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomw.2013.6529515 · Cited by 11
Mechanizing exploratory game design
Mateas, M., & Smith, A. M. (2012) · Cited by 12
Zhu, W., & Wang, X. (2021) · Springer eBooks · doi:10.1007/978-3-030-88132-0_1 · Cited by 11
Beyond Exponentially Discounted Sum: Automatic Learning of Return FunctionPreprint
Wang, Y., Ye, Q., & Liu, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11591 · Cited by 11
“, [25, 26, 27] used hand-engineered intrinsic reward that greatly accelerate the learning.”
MethodBrain signatures of surprise in EEG and MEG dataPreprint
Mousavi, Z., Kiani, M. M., & Aghajan, H. K. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.01.06.895664 · Cited by 12
“In these models, 439 surprise has been regarded as a parameter which the brain tries to minimize during 440 learning [14, 16, 17].”
MethodIntrinsic motivations and open-ended learningPreprint
Baldassarre, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.13263 · Cited by 13
Intrinsic Motivation in Computational Creativity Applied to Videogames
Guckelsberger, C. (2020) · Queen Mary Research Online (Queen Mary University of London) · Cited by 12
H-GRAIL: A Robotic Motivational Architecture to Tackle Open-Ended Learning Challenges
Romero, A., Baldassarre, G., Duro, R. J., & Santucci, V. G. (2025) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2025.3569352 · Cited by 17
“Some works have focused on developing strategies to enhance state-space exploration by leveraging heuristics such as novelty, prediction error of future states, or information-theoretic metrics [14], [15], [16], [17], [18], [19].”
Keller, R., Kirsch, A., Pei, F., Pitkow, X., Kozachkov, L., & Nayebi, A. (2025) · neural information processing systems · doi:10.52202/085713-1205 · Cited by 10
“…motivation using world-models —such as curiosity-driven exploration and variants [3–6]—exhibit inconsistent behavioral patterns due to difficulties distinguishing between controllable and uncontrollable stimuli, coping with non-stationary environments, and avoiding the “noisy TV” problem [7].”
Interactive learning gives the tempo to an intrinsically motivated robot learner
Nguyen, S. M., & Oudeyer, P. (2012) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2012.6651588 · Cited by 11
Towards Computational Co-Creation in Modding Communities
Grace, K. S., & Maher, M. L. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v10i3.12750 · Cited by 10
“Togelius and Schmidhuber (2008) present a method for evolving game rulesets using the latter’s model of artificial curiosity (Schmidhuber 2010).”
MethodYannakakis, G. N., & Liapis, A. (2016) · OAR@UM (University of Malta) · Cited by 10
Encouraging Curiosity in Case-Based Reasoning and Recommender Systems
Maher, M. L., & Grace, K. S. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-61030-6_1 · Cited by 10
Efficient Online Interest-Driven Exploration for Developmental Robots
Rayyes, R., Donat, H., & Steil, J. J. (2020) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.3001633 · Cited by 12
Stochasticity, Nonlinear Value Functions, and Update Rules in Learning Aesthetic Biases
Grzywacz, N. M. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.639081 · Cited by 11
Literature review of procedural content generation in puzzle games
Khalifa, A., & Fayek, M. B. (2015) · OAR@UM (University of Malta) · Cited by 10
Approaches to Learning to Control Dynamic Uncertainty
Osman, M., Glass, B. D., & Hola, Z. (2015) · Systems · doi:10.3390/systems3040211 · Cited by 11
Inverting the Interaction Cycle to Model Embodied Agents
Georgeon, O. L., & Cordier, A. (2014) · Procedia Computer Science · doi:10.1016/j.procs.2014.11.109 · Cited by 10
Artificial Curiosity for Autonomous Space Exploration
Futura, A. (2011) · Cited by 10
Where do goals come from? A Generic Approach to Autonomous Goal-System DevelopmentPreprint
Rolf, M., & Asada, M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.5557 · Cited by 10
“However, they might also be purely internal or intrinsic as it is often considered in the contexts of intrinsic motivation [27], [28], [29], [30] or information seeking [31].”
Strategies for Conceptual Change in Convolutional Neural NetworksPreprint
Grachten, M., & Cancino-Chacón, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.01634 · Cited by 12
Metadata
- Authors
- Schmidhuber, J.
- Affiliations
- Dalle Molle Institute for Artificial Intelligence Research (CH)
- Corresponding author
- Schmidhuber, Jürgen
- ORCID
- Schmidhuber, Jürgen
- Venue
- IEEE Transactions on Autonomous Mental Development
- Published
- 2010-09
- Volume
- 2
- Issue
- 3
- Pages
- 230-247
- ISSN
- 1943-0604 (ISSN-L)
- 1943-0612
- OpenAlex
- W2034806191
- Cited by
- 747
- References
- 126
- Citations by year
- 2026: 29 · 2025: 64 · 2024: 46 · 2023: 50 · 2022: 60 · 2021: 69 · 2020: 92 · 2019: 72 · 2018: 55 · 2017: 37 · 2016: 26 · 2015: 24 · 2014: 30 · 2013: 28 · 2012: 41
- FWCI
- 31.09
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Autonomous Mental Development (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
- Keywords
- intrinsic motivation, creativity, fun, intrinsic rewards, active learning (machine learning), novelty seeking, surprise, pattern discovery, data compression, online prediction, autonomous agents, robots, aesthetics, humor, intelligence, limited computational resources, continuous-time formulation, discrete-time formulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.