Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010)
The DOI matches its Crossref record; 125 of the 126 references it cites were found in a registry or an index.
Abstract
The simple, but general formal theory of fun and intrinsic motivation and creativity (1990-2010) is based on the concept of maximizing intrinsic reward for the active creation or discovery of novel, surprising patterns allowing for improved prediction or data compression. It generalizes the traditional field of active learning, and is related to old, but less formal ideas in aesthetics theory and developmental psychology. It has been argued that the theory explains many essential aspects of intelligence including autonomous development, science, art, music, and humor. This overview first describes theoretically optimal (but not necessarily practical) ways of implementing the basic computational principles on exploratory, intrinsically motivated agents or robots, encouraging them to provoke event sequences exhibiting previously unknown, but learnable algorithmic regularities. Emphasis is put on the importance of limited computational resources for online prediction and compression. Discrete and continuous time formulations are given. Previous practical, but nonoptimal implementations (1991, 1995, and 1997-2002) are reviewed, as well as several recent variants by others (2005-2010). A simplified typology addresses current confusion concerning the precise nature of intrinsic motivation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This underwrites Bayesian experimental design [134] and active learning in statistics [135], intrinsic motivation and artificial curiosity in machine learning and robotics [136, 137, 138, 139, 140].”
From The free energy principle made simpler but not too simple (2023) · cites this paper as [137]
In the citing paper
“This suggests that risk-averse policies minimise expected complexity or computational cost [137].”
From The free energy principle made simpler but not too simple (2023) · cites this paper as [137]
In the citing paper
“Ideally, the learner learns to choose actions that will lead to the largest improvements in its prediction or data compression performance (Schmidhuber, 2010).”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Schmidhuber, 2010
In the citing paper
“In Schmidhuber (2010), this is done in the framework of reinforcement learning, and incorporates a mechanisms for the system to measure its own rate of learning.”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Schmidhuber, 2010
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 7×
“Another potential drawback is the nature of the particular RNN-based RL method.”
BackgroundInfluentialSelf-citationA Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,002 · cited 1×
“iii) An evolutionary algorithm [22], [29], [59],”
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 7×
“Some Q-learning accelerators simply update pairs of actions and states with currently quickly changing Q-values more frequently than others (that is, Q-values with high first derivatives are favored).”
BackgroundMethodInfluentialReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Neural networks for pattern recognition
(1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,538
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Neural Networks for Pattern Recognition
Bishop, C. M. (1995) · doi:10.1093/oso/9780198538493.001.0001 · Cited by 12,287
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · cited 4×
“, some typically suboptimal RL method [35].”
BackgroundMethodInfluential
Research works printed but not linked: 24
- kullback, 1959, Statistics and Information Theory
- hutter, 2004, Universal Artificial Intelligence Sequential Decisions Based on Algorithmic Probability
- holland, 1975, Adaptation in Natural and Artificial Systems
- schmidhuber, 2010, How to Learn Programs with Artificial Recurrent Neural Networks
- harlow, 1950, novelty and curiosity as determinants of exploratory behavior, J Exp Psychol, 41, 68
- berlyne, 1950, novelty and curiosity as determinants of exploratory behavior, Brit J Psychol, 41, 68
- 10.1007/bf00993591 · doi:10.1007/bf00993591 · Paper page
- levin, 1973, universal sequential search problems, Problems Inform Trans, 9, 265
- schmidhuber, 1991, Advances in Neural Information Processing Systems 3, nips 3, 500
- schmidhuber, 1997, Femmes Fractales
- schmidhuber, 2002, Advances in Evolutionary Computing, 579
- ring, 1994, Continual Learning in Reinforcement Environments
- schmidhuber, 1990, Dynamische neuronale Netze und das fundamentale raumzeitliche Lernproblem
- schmidhuber, 1991, Adaptive Curiosity and Adaptive Confidence
- schmidhuber, 2009, Compression Progress The Algorithmic Principle Behind Curiosity and Creativity (With Applications of the Theory of Humor) 40 min Video of Invited Talk at Singularity Summit 2009
- schmidhuber, 2009, Multiple Ways to Design Research Research Cases that Reshape the Design Discipline, 98
- singh, 2005, Advances in neural information processing systems, 17
- schwefel, 1974, Numerische Optimierung von Computer-Modellen
- schmidhuber, 2007, alle berechenbaren universen (all computable universes), Spektrum der Wissenschaft, 75
- storck, 1995, reinforcement driven information acquisition in non-deterministic environments, Proc Int Conf Artif Neural Netw, 2, 159
- brafman, 2002, R-MAX—A general polynomial time algorithm for near-optimal reinforcement learning, J Mach Learn Res, 3, 213
- frank, 2002, sthetische Information Estetika informacio Eine Einfhrung in die Kybernetische sthetik
- franke, 1979, Kybernetische sthetik Phnomen Kunst
- schmidhuber, 2006, Artificial General Intelligence, 175
Cited by
Items 501–600 of 886
Online Safety Assurance for Deep Reinforcement LearningPreprint
Rotman, N. H., Schapira, M., & Tamar, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.03625 · Cited by 5
“Curiosity-based exploration in RL involves guiding the agent to unvisited states during training [35, 43].”
BackgroundToddlers search longer when there is more information to be gainedPreprint
Ruggeri, A., Pelz, M. C., Alison Gopnik, C., & Schulz, E. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/uzdvp · Cited by 5
The "Tau" of Science - How to Measure, Study, and Integrate Quantitative and Qualitative KnowledgePreprint
Fanelli, D. (2022) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/67sak · Cited by 5
Intrinsically-Motivated Reinforcement Learning: A Brief IntroductionPreprint
Yuan, M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.02298 · Cited by 5
“Exploration methods encourage RL agents to fully explore the state space in various way, for instance by rewarding surprise [11, 12], curiosity [13, 14], information gain (IG) [15, 16], feature control [17] or diversity [18].”
Learning to Deceive in Multi-agent Hidden Role Games
Aitchison, M., Benke, L., & Sweetser, P. (2021) · Communications in Computer and Information Science · doi:10.1007/978-3-030-91779-1_5 · Cited by 4
“To this end, we trained two sets of agents to play RTG, one in which the agent pursues the rewards of the game as per normal, and another where the agents are provided an intrinsic reward [31] to incentivise deceptive actions.”
MethodLeveling up fun: learning progress, expectations, and success influence enjoyment in video games
Brändle, F., Wu, C. M., & Schulz, E. (2025) · Scientific Reports · doi:10.1038/s41598-025-14628-2 · Cited by 5
“Here, we test the theory of intrinsic reward as learning progress 15,16,35,36 to inform our simulations and to compare them with human behaviour.”
Active inference and artificial reasoning
Friston, K. J., Costa, L. D., Tschantz, A., Heins, C., Buckley, C., Verbelen, T., & Parr, T. (2026) · Nature Communications · doi:10.1038/s41467-026-77209-5 · Cited by 4
“In robotics, this kind of motivation is sometimes referred to as intrinsic — c.f., 7 — that manifests as arti fi cial curiosity 8 .”
Discriminative reward co-training
Altmann, P. J., Ritz, F., Zorn, M., Kölle, M., Phan, T., Gabor, T., & Linnhoff–Popien, C. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10512-8 · Cited by 4
Farooq, S. S., Rahman, H., Wahid, S. A., Ansari, M. A., Wahid, S. A., & Lee, H. (2025) · Computers · doi:10.3390/computers14100434 · Cited by 6
“For instance, Schmidhuber et al. [42,43] and others [44] used surprise and compression progress as intrinsic rewards.”
Let it Learn - A Curious Vision System for Autonomous Object Learning
Chandrashekhariah, P., Spina, G., & Triesch, J. (2013) · International Conference on Computer Vision Theory and Applications · doi:10.5220/0004294101690176 · Cited by 4
“The central inspiration of our approach is the concept of intrinsic motivation (Baranes and Oudeyer, 2009; Schmidhuber, 2010; Baldassarre, 2011).”
BackgroundAutomatic Prediction of Aesthetics and Interestingness of Text Passages
Ganguly, D., Leveling, J., & Jones, G. J. F. (2014) · International Conference on Computational Linguistics · Cited by 4
Modeling of Cognitive Evolution: Agent-Based Investigations in Cognitive Science
Red’ko, V. G. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-40663-3_83 · Cited by 4
Artificial Interactionism: Avoiding Isolating Perception From Cognition in AI
Guillermin, M., & Georgeon, O. L. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.806041 · Cited by 4
“…have been proposed, among which curiosity (Oudeyer et al., 2007), the autotelic principle (Steels, 2004), the principle of data compressibility (Schmidhuber, 2010), predictability-based exploration ( Bugur et al., 2021), minimization of uncertainty through minimization of free energy (Friston…”
BackgroundReinforcement Learning Based Dance Movement Generation
Ruud, M. T., Sandberg, T. H., Tranvaag, U. J. V., Wallace, B., Karbasi, S. M., & Tørresen, J. (2022) · International Conference on Movement and Computing (MOCO) · doi:10.1145/3537972.3538007 · Cited by 4
“Schmidhuber using the concept of curiosity and intrinsic motivation (internal rewards as opposed to external rewards, affecting behaviour) [12].”
MethodNagel, J. (2024) · Oxford University Press eBooks · doi:10.1093/9780191976766.003.0007 · Cited by 4
Touzet, C. F. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-22887-2_43 · Cited by 4
Automatic Prediction of Text Aesthetics and Interestingness
Ganguly, D., Leveling, J., & Jones, G. J. F. (2014) · Dublin City University Open Access Institutional Repository (Dublin City University) · Cited by 4
Grzyb, B. J., Boedecker, J., Asada, M., del Pobil, A. P., & Smith, L. B. (2011) · National Conference on Artificial Intelligence · Cited by 5
Online Learning with Stochastic Recurrent Neural Networks using Intrinsic Motivation Signals
Tanneberg, D., Peters, J., & Rueckert, E. (2022) · Technischen Universität Darmstadt · doi:10.26083/tuprints-00020580 · Cited by 4
Schneider, M. (2018) · Cognitive Systems Research · doi:10.1016/j.cogsys.2018.07.005 · Cited by 4
Abdelfattah, S. M., Kasmarik, K., & Hu, J. (2018) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2018.00065 · Cited by 5
“Schmidhuber (2010) describes the learning assumption of IMRL as “maximizing the fun or internal joy for the discovery or creation of novel patterns.””
BackgroundDiscovering Unexpected Local Nonlinear Interactions in Scientific Black-box Models
Doron, M., Segev, I., & Shahaf, D. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330886 · Cited by 4
“The works of [25, 38] suggest that an observed sample is interesting if the observers update their understanding of the system after viewing the sample.”
BackgroundMethodSelf-guided deep deterministic policy gradient with multi-actor
Chen, H., Liu, Q., & Zhong, S. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-05738-9 · Cited by 4
Combining Social and Intrinsically Motivated Learning for Multitask Robot Skill Acquisition
Kulak, T., & Calinon, S. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3069341 · Cited by 5
Spatiotemporal Signatures of Surprise Captured by Magnetoencephalography
Mousavi, Z., Kiani, M. M., & Aghajan, H. K. (2022) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2022.865453 · Cited by 5
“In this context, surprise is often represented by a parameter that the brain attempts to minimize during the process of learning and perceptual inference (Schmidhuber, 2010; Roesch et al., 2012; Friston and Frith, 2015; Friston et al., 2017; Faraji et al., 2018).”
BackgroundLearning One Abstract Bit at a Time Through Self-invented Experiments Encoded as Neural Networks
Herrmann, V., Kirsch, L., & Schmidhuber, J. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-47958-8_16 · Cited by 5
Self-citationLatent Learning Progress Drives Autonomous Goal Selection in Human Reinforcement Learning
Molinaro, G., Colas, C., Oudeyer, P.-Y., & Collins, A. G. E. (2024) · neural information processing systems · doi:10.52202/079017-1013 · Cited by 15
“As a result, researchers and engineers started endowing artificial systems with intrinsic reward signals that aim to reproduce aspects of human curiosity-driven learning [11–17].”
The Three Factors of Creativity Management: Visual, Number, and Word Creativity
Yu, J. W. (2015) · DLSU Business & Economics Review · doi:10.59588/2243-786x.1608 · Cited by 4
Autonomous reuse of motor exploration trajectories
Benureau, F., & Oudeyer, P. (2013) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2013.6652567 · Cited by 4
Learning where to look with movement-based intrinsic motivations: A bio-inspired model
Sperati, V., & Baldassarre, G. (2014) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2014.6983024 · Cited by 5
“The most studied are: (b) Prediction-based IMs : these are triggered by stimuli that violate the animal’s expectations [6], [7], [8]; (a) Novelty-based IMs : these are triggered by stimuli that are not in the animal’s memory [5]; (c) Competence-based IMs : these are related to the animal’s capacity…”
BackgroundHigh Performance Computing of Possible Minds
Ziesche, S., & Yampolskiy, R. V. (2017) · International Journal of Grid and High Performance Computing · doi:10.4018/ijghpc.2017010104 · Cited by 4
Efficient online adaptation with stochastic recurrent neural networks
Tanneberg, D., Peters, J., & Rueckert, E. (2017) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2017.8246875 · Cited by 5
“A coherent theory and fundamental investigation of using intrinsic motivation in machine learning is given in [21], where it is stated that the improvement of prediction can be used as an intrinsic reinforcement for efficient learning.”
BackgroundBioVision: A Biomimetics Platform for Intrinsically Motivated Visual Saliency Learning
Craye, C., Filliat, D., & Goudou, J.-F. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2018.2806227 · Cited by 5
“To overcome limitations related to novelty or error in unlearnable situations, intrinsic motivation based on progress has been proposed [1], [46], [56].”
Evaluation of Internal Models in Autonomous Learning
Smith, S. C., & Herrmann, J. M. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2018.2865999 · Cited by 4
SEDRo: A Simulated Environment for Developmental Robotics
Pothula, A., Mondol, M. A. R., Narasimhan, S., Islam, S. M., & Park, D. (2020) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/icdl-epirob48136.2020.9278036 · Cited by 4
“flexes, hippocampus, or limbic systems. Universal principles explain the driving force behind learning and can be usually written as succinct mathematical formulation such as intrinsic motivation [9], [10], Bayesian statistics [11], or the free energy principles [12]. As we can see, there are many candidate components,…”
BackgroundModelling and Generating Strategy Games Mechanics
Mahlmann, T. (2013) · IT University Of Copenhagen (IT University of Copenhagen) · Cited by 7
Assessment of fun from the analysis of facial images
Vieira, L. C. (2017) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.45.2017.tde-05072017-212226 · Cited by 5
Goal-oriented Trajectories for Efficient ExplorationPreprint
Pardo, F., Levdik, V., & Kormushev, P. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.02078 · Cited by 4
The complexity dividend: when sophisticated inference mattersPreprint
Tavoni, G., Balasubramanian, V., & Gold, J. I. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/563346 · Cited by 4
“10 can also be interpreted as an estimate of the reward one can obtain per unit time (Shenhav et al., 2017; Vul et al., 2014; Schmidhuber, 2010; Gold & Shadlen, 2002).”
BackgroundEfficient Online Estimation of Empowerment for Reinforcement LearningPreprint
Zhao, R., Abbeel, P., & Tiomkin, S. (2020) · arXiv (Cornell University) · Cited by 6
The societal and ethical relevance of computational creativityPreprint
Loi, M., Viganò, E., & van der Plas, L. (2020) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2007.11973 · Cited by 4
“Intrinsically Motivated Reinforcement Learning (Kaplan and Oudeyer (2007) among others) basically works as follows: the agent is rewarded for discovering new patterns in the environment (Schmidhuber 2010), and it is always in search of novelty.”
BackgroundUnderstanding exploration in humans and machines by formalizing the function of curiosityPreprint
Dubey, R., & Griffiths, T. L. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/uz5wp · Cited by 4
“Algorithms based on prediction errors rely on the hypothesis that the brain is intrinsically motivated to pursue tasks in which one’s predictions are always improving [20,25,26,27 ].”
BackgroundAnticipation in multiple criteria decision-making under uncertainty
Azevedo, C. R. B. (2012) · doi:10.47749/t/unicamp.2012.938003 · Cited by 4
The World Is Bigger! A Computationally-Embedded Perspective on the Big World HypothesisPreprint
Lewandowski, A., Ramesh, A. A., Meyer, E., Schuurmans, D., & Machado, M. C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.23419 · Cited by 4
“Previous work has explored several intrinsic drives that can guide agent behaviour without the need for explicit external rewards (Schmidhuber, 2010; Barto, 2013).”
Cherti, M. (2018) · theses.fr (ABES) · doi:10.70675/d040b5b4zd926z4f6ez9870z7af586f801ab · Cited by 4
“Similarly to (Schmidhuber, 2010), they also mention intrinsic motivation as a reward for doing experiments in their environments that lead to new situations while still making sure these new situations are not too much different from the known ones.”
BackgroundRajendran, J., Lewis, R. L., Veeriah, V., Lee, H., & Singh, S. P. (2020) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v34i04.5995 · Cited by 7
Peng, Z., & Braun, D. A. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01879 · Cited by 3
Artificial Creativity: from predictive AI to Generative System 3
Chávez-Autor, J. C. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1654716 · Cited by 3
“Information-theoretic and intrinsic-motivation accounts explain why systems seek novelty or compressive structure (Schmidhuber, 2010).”
Flimm, H., Tuzsus, D., Pappas, I., & Peters, J. (2025) · Computational Brain & Behavior · doi:10.1007/s42113-025-00258-4 · Cited by 3
“Implementing curiosity in artificial agents via curiosity-driven algorithms, as has already been proposed (Ruggeri et al., 2024; Schmidhuber, 2010), might lead to a higher exploration rate and a lower level of perseveration.”
An Active Inference Model of Meter Perception and the Urge to Move to Music
Matthews, T. E., Vuust, P., & Cannon, J. J. (2025) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.70129 · Cited by 3
Raihan, A. S., Liu, Z., Bhuiyan, T. H., & Ahmed, I. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.115463 · Cited by 3
Learning Intrinsically Motivated Options to Stimulate Policy Exploration
Bagot, L., Mets, K., & Latré, S. (2020) · HAL · Cited by 5
C-GRAIL: Autonomous Reinforcement Learning of Multiple and Context-Dependent Goals
Santucci, V. G., Montella, D., & Baldassarre, G. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2022.3152081 · Cited by 3
Milano, N., & Nolfi, S. (2022) · Evolutionary Intelligence · doi:10.1007/s12065-022-00801-3 · Cited by 3
E-learning in the Transformation of Education in Digital Society
Smyrnova‐Trybulska, E. (2022) · E-learning · doi:10.34916/el.2022.14 · Cited by 4
AI, Popular Culture, Semiotics
Danesi, M. (2024) · Springer eBooks · doi:10.1007/978-3-031-54752-2_1 · Cited by 3
Tinker, T. J., Doya, K., & Tani, J. (2024) · Neural Computation · doi:10.1162/neco_a_01690 · Cited by 4
“Among various definitions of curiosity such as those from Oudeyer and Kaplan (2007), Schmidhuber (2010), Pathak et al. (2017), and Schwartenbeck et al. (2019), the most relevant to this letter is that of Kawahara et al. (2022) because it is directly derived from the FEP for RL.”
BackgroundMethodImproving sample efficiency and exploration in upside-down reinforcement learning
Nakhaei, M., & Moghadam, R. A. (2025) · Journal of Information and Intelligence · doi:10.1016/j.jiixd.2025.04.004 · Cited by 4
Decision, Inference, and Information: Formal Equivalences Under Active Inference
Sweeney, P. C., Ruiz-Serra, J., & Harré, M. (2025) · Entropy · doi:10.3390/e28010001 · Cited by 3
“These included entropy-based experiment design [68], active learning with uncertainty-aware models [69], and exploratory reinforcement learning through information gain [70].”
The World Is Bigger! A Computationally-Embedded Perspective on the Big World Hypothesis
Lewandowski, A., Ramesh, A., Meyer, E., Schuurmans, D., & Machado, M. C. (2025) · neural information processing systems · doi:10.52202/085713-0951 · Cited by 3
Many Meanings of Intentionality: A Brief Disambiguation for Computational Creativity
Linkola, S., & Hauhio, I. (2023) · Työväentutkimus Vuosikirja · Cited by 3
Gatsoulis, Y., Burbridge, C., & McGinnity, T. M. (2012) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio.2012.6491010 · Cited by 3
“The methodology and the experimental setup are then described in Sections IV and V, followed by the results in Section VI and the conclusions in Section VII.”
BackgroundMethodStradella, E., Salvini, P., Pirni, A., Carlo, A. D., Oddo, C. M., Dario, P., & Palmerini, E. (2012) · CINECA IRIS Institutial research information system (University of Pisa) · Cited by 3
Exploring Criticality as a Generic Adaptive Mechanism
Aguilera, M., & Bedía, M. G. (2018) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2018.00055 · Cited by 5
“For example, Schmidhuber (2010) addresses the problem of how to model aspects of human player behavior that are not explained by either rational or goal-driven decision making behavior and without extrinsic reinforcement such as game score.”
BackgroundEntropic boundary conditions towards safe artificial superintelligence
Núñez-Corrales, S., & Jakobsson, E. (2021) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2021.1952653 · Cited by 5
“In this context, life will pose a plethora of surprising patterns or learnable regularities (Schmidhuber, 2010) that will naturally, we hope, prompt the agent to prevent irrecoverable risks to life forms stated in terms of diminished learning opportunities.”
BackgroundDiscovering the Neuroanatomical Correlates of Music with Machine Learning
Daikoku, T. (2021) · Springer eBooks · doi:10.1007/978-3-030-72116-9_6 · Cited by 3
A Model-Based Exploration Policy in Deep Q-Network
Li, S., Zhang, W., Leng, Y., & Zhang, X. (2021) · International Conference on Digital Society and Intelligent Systems (DSInS) · doi:10.1109/dsins54396.2021.9670573 · Cited by 3
CandyRL: A Hybrid Reinforcement Learning Model for Gameplay
Karimi, S., Asadi, S., Lorenzo, F., & Payberah, A. H. (2022) · International Conference on Machine Learning and Applications · doi:10.1109/icmla55696.2022.00098 · Cited by 3
Надутенко, М. В., Надутенко, М. В., & ФАСТ, О. Л. (2024) · doi:10.52726/as.pedagogy/2024.2.14 · Cited by 3
Tong, Y. (2024) · Journal of big data and computing. · doi:10.62517/jbdc.202401318 · Cited by 4
Unsupervised Modeling of Partially Observable Environments
Graziano, V., Koutník, J., & Schmidhuber, J. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23780-5_42 · Cited by 3
“The use of artificial curiosity [13] with planning to address this problem has been carried out in [9], where the sensory layer was built-up using vector quantization (a SOM without neighborhoods).”
MethodSelf-citationFinding minimal action sequences with a simple evaluation of actions
Shah, A., & GURNEY, K. N. (2014) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2014.00151 · Cited by 3
Autonomous reinforcement of behavioral sequences in neural dynamics
Kazerounian, S., Luciw, M., Sandamirskaya, Y., Richter, M., Schmidhuber, J., & Schöner, G. (2012) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2012.6400831 · Cited by 3
“One could imagine future work using more sophisticated exploration methods [26].”
MethodSelf-citationDissociable Forms of Repetition Priming: A Computational Model
Makukhin, K., & Bolland, S. (2014) · Neural Computation · doi:10.1162/neco_a_00569 · Cited by 3
“Interestingly, in the field of developmental robotics, it has been widely assumed that exploring on the periphery of knowledge is a preferred strategy for the gaining of competence (Oudeyer, Kaplan, & Hafner, 2007; Kaplan & Oudeyer, 2007; Ryan & Deci, 2000; Schmidhuber, 2010).”
BackgroundOptimal Curiosity-Driven Modular Incremental Slow Feature Analysis
Kompella, V. R., Luciw, M., Stollenga, M., & Schmidhuber, J. (2016) · Neural Computation · doi:10.1162/neco_a_00855 · Cited by 5
“This work is based on the theory of artificial curiosity (AC; Schmidhuber, 2006a, 2010a).”
BackgroundSelf-citationArtificial aesthetic: An interesting framework for epigenetic robotics
Karaouzene, A., Gaussier, P., & Vidal, D. (2012) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2012.6400587 · Cited by 3
“But even at this stage, our approach is different from most other studies about the exploration of aesthetics in robots [1] where the ability to display some sort”
ResultAn Examination of High School Students' Online Engagement in Mathematics Problems
Lim, W., Son, J., Gregson, S., & Kim, J. (2018) · International Journal of Web-Based Learning and Teaching Technologies · doi:10.4018/ijwltt.2018040101 · Cited by 3
A Dynamic Neural Network Approach to Generating Robot's Novel Actions: A Simulation Experiment
Hwang, J., & Tani, J. (2018) · International Conference on Ubiquitous Robots (UR) · doi:10.1109/urai.2018.8441824 · Cited by 3
ExTra: Transfer-guided Exploration
Santara, A., Madan, R., Mitra, P., & Ravindran, B. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/dgit5441 · Cited by 3
Exploration in Feature Space for Reinforcement LearningPreprint
Sasikumar, S. N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.02210 · Cited by 3
“Schmidhuber (2010) proposed a measure for intrinsic motivation by taking into account the improvement a learning algorithm effected on its predictive world model.”
BackgroundPrioritizing Starting States for Reinforcement Learning.Preprint
Tavakoli, A., Levdik, V., Islam, R., & Kormushev, P. S. (2018) · arXiv (Cornell University) · Cited by 3
Flexible and Efficient Long-Range Planning Through Curious ExplorationPreprint
Curtis, A., Xin, M., Arumugam, D., Feigelis, K., & Yamins, D. L. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.10876 · Cited by 5
“We introduce the Curious Sample Planner (CSP), which uses curiosity (Schmidhuber, 2010; Pathak et al., 2017) to bias the search toward novel points in the state space.”
MethodTowards Informed Exploration for Deep Reinforcement Learning
Tang, H. (2019) · eScholarship (California Digital Library) · Cited by 3
The societal and ethical relevance of computational creativityPreprint
Loi, M., Viganò, E., & van der Plas, L. (2020) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-199189 · Cited by 3
Learning to Plan Optimistically: Uncertainty-Guided Deep Exploration via Latent Model EnsemblesPreprint
Seyde, T., Schwarting, W., Karaman, S., & Rus, D. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.14641 · Cited by 5
A Brief Survey of Associations Between Meta-Learning and General AIPreprint
Peng, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.04283 · Cited by 3
“For example, [19] adds curiosity or creativity reward in the objective function of reinforcement learning so that robots’ behaviors are more diverse.”
Humans monitor learning progress in curiosity-driven explorationPreprint
Ten, A., Kaushik, P., Oudeyer, P., & Gottlieb, J. P. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/d4wat · Cited by 3
Developmental Curiosity and Social Interaction in Virtual AgentsPreprint
Doyle, C., Shader, S., Lau, M., Sano, M., Yamins, D. L. K., & Haber, N. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.13396 · Cited by 3
“Working to improve these predictions (the agent’s “world model”) can create a self-generated learning curriculum, through a cycle of evaluating deficiencies in the model, seeking out information, updating the model, and gaining new capabilities (Schmidhuber, 2010).”
BackgroundAdvancing Self-Determination Theory With Computational Intrinsic Motivation: The Case of CompetencePreprint
Lintunen, E. M., Ady, N. M., Guckelsberger, C., & Deterding, S. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/7qy35 · Cited by 4
ViVa: Video-Trained Value Functions for Guiding Online RL from Diverse DataPreprint
Dashora, N., Ghosh, D., & Levine, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.18210 · Cited by 3
Learning Interestingness in Automated Mathematical Theory FormationPreprint
Tsoukalas, G., Saha, R., Thakur, A., Reguyal, S., & Chaudhuri, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.14778 · Cited by 3
“Such internal rewards can be critical for driving exploration and acquisition of general knowledge about the environment and discovery of useful subgoals, especially when external feedback is infrequent or absent, by promoting behaviors like curiosity or novelty-seeking [1, 3, 32, 33, 41, 43].”
Nguyen, S. M. (2013) · theses.fr (ABES) · doi:10.70675/644a094ez08b3z4dbaz8542z13a9be39e42c · Cited by 4
“Indeed, transposed in curiosity-driven learning machines (Schmidhuber, 1991b; Barto et al., 2004b; Schmidhuber, 2010; Schembri et al., 2007; Merrick and Maher, 2009; Hart, 2009; Stout and Barto, 2010) and robots (Oudeyer et al.”
BackgroundDroniou, A. (2015) · theses.fr (ABES) · doi:10.70675/396a49b1z7704z4ad3z83b9z2ad6ced4083c · Cited by 3
Lesort, T. (2020) · theses.fr (ABES) · doi:10.70675/e714c94ezb6d9z4adbz9daaz60088df88f5f · Cited by 3
Teodorescu, L. (2023) · theses.fr (ABES) · doi:10.70675/08e0bf33z987fz4ff6zbdd7zbf6cf87e81f0 · Cited by 3
Beyond Fixed Representations: The Vocabulary and Verifier Gaps in Open-Ended AIPreprint
Cao, Y., & Yang, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.09560 · Cited by 3
“Updating θ t − 1 to θ t so that the same history becomes more compressible is an abstraction step guided by a simplicity prior. closely related to the compression view of MDL and AIT [50, 53].”
Playing Games Across the Superintelligence Divide
Isaksen, A., Togelius, J., Lantz, F., & Nealen, A. (2016) · National Conference on Artificial Intelligence · Cited by 2
Pragmatic information of aesthetic appraisal
Graben, P. B. (2025) · Cognitive Neurodynamics · doi:10.1007/s11571-025-10225-1 · Cited by 2
Chen, Q., & Prokopenko, M. (2025) · Royal Society Open Science · doi:10.1098/rsos.241655 · Cited by 3
“The Venn diagram of predictive information shown as the mutual information (the overlap area) between past and future sensory states: it represents how useful the past is for predicting the future.”
Cognitive Based Detection of Anomalous Sequences Using Bayesian Surprise
McGarry, K., & Nelson, D. (2025) · Expert Systems · doi:10.1111/exsy.70106 · Cited by 2
“Surprise as a criterion for judging differences in belief is finding applications in reinforcement-based learning for automating the learning process (Schmidhuber 2010; Gottlieb et al. 2013) and autonomous agents (Rhienberger and Hammitt 2018; Maguire et al. 2019).”
Metadata
- Authors
- Schmidhuber, J.
- Affiliations
- Dalle Molle Institute for Artificial Intelligence Research (CH)
- Corresponding author
- Schmidhuber, Jürgen
- ORCID
- Schmidhuber, Jürgen
- Venue
- IEEE Transactions on Autonomous Mental Development
- Published
- 2010-09
- Volume
- 2
- Issue
- 3
- Pages
- 230-247
- ISSN
- 1943-0604 (ISSN-L)
- 1943-0612
- OpenAlex
- W2034806191
- Cited by
- 747
- References
- 126
- Citations by year
- 2026: 29 · 2025: 64 · 2024: 46 · 2023: 50 · 2022: 60 · 2021: 69 · 2020: 92 · 2019: 72 · 2018: 55 · 2017: 37 · 2016: 26 · 2015: 24 · 2014: 30 · 2013: 28 · 2012: 41
- FWCI
- 31.09
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Autonomous Mental Development (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
- Keywords
- intrinsic motivation, creativity, fun, intrinsic rewards, active learning (machine learning), novelty seeking, surprise, pattern discovery, data compression, online prediction, autonomous agents, robots, aesthetics, humor, intelligence, limited computational resources, continuous-time formulation, discrete-time formulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.