The DOI matches its Crossref record; 109 of the 132 references it cites were found in a registry or an index; none of the 51 sources its claims cite is checked for support yet.
Information-seeking, curiosity, and attention: computational and neural mechanisms
Open accessAbstract
Summary Intelligent animals devote much time and energy to exploring and obtaining information, but the underlying mechanisms are poorly understood. We review recent developments on this topic that have emerged from the traditionally separate fields of machine learning, eye movements in natural behavior, and studies of curiosity in psychology and neuroscience. These studies show that exploration may be guided by a family of mechanisms that range from automatic biases toward novelty or surprise, to systematic search for learning progress and information gain in curiosity-driven behavior. In addition, eye movements reflect visual information search in multiple conditions and are amenable for cellular-level investigations, suggesting that the oculomotor system is an excellent model system for understanding information sampling mechanisms.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This review integrates machine learning, eye-movement, and curiosity research to argue that information seeking ranges from automatic novelty/surprise biases to targeted search for learning progress and information gain. It proposes the oculomotor system as a model for information sampling and highlights intrinsic rewards and uncertainty monitoring as key open questions.
Key findings
- Information seeking is defined as actions whose primary goal is to obtain information and reduce uncertainty; it can be extrinsically motivated, as when a driver glances at a traffic light, or intrinsically motivated, as in curiosity or interest.
- Task-related gaze is proactively guided by uncertainty: in visuo-manual tasks the eyes move ahead of the hand to collision or target points and wait there, and in a virtual-reality walking study subjects preferentially deployed gaze to oncoming pedestrians with a history of veering onto a collision course.
- Curiosity-like mechanisms include Lowenstein’s information gap hypothesis, illustrated by a mystery novel with 10 equally likely suspects where the reader moves from high entropy to a single culprit, and developmental robotics approaches that reward learning progress rather than absolute competence.
- Novelty, surprise, and uncertainty heuristics help discover new tasks but are inefficient in large open-ended spaces and risk trapping agents in unlearnable tasks; learning-progress systems may support efficient exploration but a quantitative neural response to learning progress has not been demonstrated empirically.
Methods
- Narrative review synthesizing three traditionally separate lines: active learning and exploration in machine learning and robotics, eye movements in natural behavior, and curiosity in psychology and neuroscience.
- No primary empirical sample or new data are reported; the paper uses conceptual and computational examples, including a POMDP traffic-light example, the information gap mystery-novel illustration, and developmental robotics simulations/robots.
- Main analysis is conceptual integration and comparison of mechanisms across fields, not statistical meta-analysis.
Limitations
- The paper states that understanding information-seeking mechanisms is in its infancy, that a quantitative response to learning progress distinct from phasic responses to novelty, surprise, or arousal has not been demonstrated empirically, and that the learning-progress approach entails a complex meta-cognitive architecture awaiting empirical verification.
- The paper also states that it has not been empirically explored to what extent task-related informational eye movements are habitual, immutable aspects of a task versus rapidly responsive to changing epistemic conditions.
- Evident from the text: as a review, it provides no new experimental data, and many claims about intrinsic rewards and their neural implementation remain speculative; the paper itself notes the question of active exploration is vast and cannot be exhaustively covered.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 51 cited sources checked · 0 supported · 0 flagged
In the paper
“In well-practiced tasks that involve visuo-manual coordination (such as moving an object to a target point) the eyes move ahead of the hand to critical locations such as potential collision or target points, and wait there until the hand has cleared those locations”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Task-directed search for information through the prism of eye movements > Eye movements reflect active information search
What the sources say
- Not checked yetcites Johansson, Westling et al. 2001
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIn the paper
“Singh et al. (Singh, Lewis et al. 2010) have shown with computer simulations that even if the objective fitness/reward function of an organism is to survive and reproduce, it may be more efficient to evolve a control architecture that encodes an innate surrogate reward function rewarding learning per se.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > The benefits and challenges of information seeking
What the sources say
- Not checked yetcites Singh, Lewis et al. 2010
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“Finally, the eye movements patterns made while acquiring a task differ greatly from those made after learning”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Task-directed search for information through the prism of eye movements > Eye movements reflect active information search
What the sources say
- Not checked yetcites Sailer, Flanagan et al. 2005
- Not checked yetcites Land 2006 source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIn the paper
“Subjects preferentially deployed gaze to oncoming pedestrians whose trajectory was expected to be uncertain (i.e., who had a history of veering onto a collision course) relative to those who had never shown such deviations.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Task-directed search for information through the prism of eye movements > Eye movements reflect active information search
What the sources say
- Not checked yetcites Jovancevic-Misic and Hayhoe 2009
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIn the paper
“Sensory novelty, defined as a small number of stimulus exposures, is known to enhance neural responses throughout visual, frontal and temporal areas”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > Randomness, novelty, surprise and uncertainty
What the sources say
- Not checked yetcites Ranganath and Rainer 2003 source
Strength:boosted· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIn the paper
“Sensory novelty, defined as a small number of stimulus exposures, is known to enhance neural responses throughout visual, frontal and temporal areas (Ranganath and Rainer 2003) and activate reward responsive dopamine-recipient areas.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > Randomness, novelty, surprise and uncertainty
What the sources say
- Not checked yetcites Ranganath and Rainer 2003 source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“information seeking in its many manifestations is related to the ability of an observer to estimate his or her uncertainty and to find strategies that reduce that uncertainty.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Information seeking obeys the imperative to reduce uncertainty and can be extrinsically or intrinsically motivated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“Converging evidence suggests that, when deployed in the service of a task, eye movements may be explained by the simple imperative to sample information in order to reduce uncertainty regarding future states”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Task-directed search for information through the prism of eye movements > Eye movements reflect active information search
What the sources say
- Not checked yetcites Tatler, Hayhoe et al. 2011 source
- Not checked yetcites Friston and Ao 2012
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Our focus here is on evaluating recent developments related to this question from three lines of investigation that have remained largely separate – namely, studies of active learning and exploration in the machine learning and robotics fields, studies of eye movements in natural behavior, and studies of curiosity in psychology and neuroscience.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Information seeking in machine learning, psychology and neuroscience
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“These observations, together with the fact that eye movements are well investigated at the single neuron level in experimental animals and use value-based decision mechanisms”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Task-directed search for information through the prism of eye movements > Eye movements reflect active information search
What the sources say
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- It may state two separate claims as one. 12
- Probably not a substantial claim. 22
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Notably, the eyes never track the hand, which relies on motor and proprioceptive guidance and, for short periods of time, follows a predictable path; instead, they are strategically deployed to acquire new information.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Task-directed search for information through the prism of eye movements > Eye movements reflect active information search
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“One salient fact that emerges regarding this question is the sheer vastness of the learning space, especially given the scarce time and energy available for learning.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > The benefits and challenges of information seeking
Strength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Deciding how to explore optimally when searching for strategy is a very difficult question, and is almost intractable in the general case.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Task-directed search for information through the prism of eye movements > Information sampling while searching for a task strategy
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A number of processes for open-ended exploration have been described in the literature, which, as we describe below, have individual strengths and weaknesses and may act in complementary fashion to accomplish these goals.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > The benefits and challenges of information seeking
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Information gathering actions are a special type of intermediate step that obey the imperative to reduce uncertainty and adjudicate among competing interpretations.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Task-directed search for information through the prism of eye movements > Information sampling while executing a known task
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This suggests that the observers monitor the uncertainty or predictability of external items and use these quantities proactively to deploy gaze”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Task-directed search for information through the prism of eye movements > Eye movements reflect active information search
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Strategies for solving a task, including those for generating informational actions, are not known in advance but must also be learnt.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Task-directed search for information through the prism of eye movements > Information sampling while searching for a task strategy
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is important to note that, while these processes require significant exploration, they are goal directed in the sense that they seek to maximize a separate, or extrinsic, reward”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Task-directed search for information through the prism of eye movements > Information sampling while searching for a task strategy
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, the challenge of an information seeking mechanism is to learn efficiently a large repertoire of diverse skills given its limited resources, and avoid being trapped in unlearnable situations.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > The benefits and challenges of information seeking
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Similarly in the cognitive domain, infants must acquire a vast amount of factual knowledge, rules and social skills.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > The benefits and challenges of information seeking
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“although using different terminology and methods, these three lines of research grapple with strikingly similar questions and propose overlapping mechanisms.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Information seeking in machine learning, psychology and neuroscience
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The benefits of such a system arise because of the limited cognitive capacities of the agent (i.e., its inability to solve the fitness function directly) (Sorg, Lewis et al. 2010; Lehman and Stanley 2011; Sequeira, Melo et al. 2011) or because information or skills that are not immediately useful may be re-used in the future.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > The benefits and challenges of information seeking
What the sources say
- Not checked yetcites Sorg, Lewis et al. 2010
- Not checked yetcites Lehman and Stanley 2011
- Not checked yetcites Sequeira, Melo et al. 2011
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although exploratory actions can involve physical acts, they are distinct from other motor acts in that their primary goal is not to exert force on the world, but to alter the observer’s epistemic state.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Information seeking obeys the imperative to reduce uncertainty and can be extrinsically or intrinsically motivated
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“eye movements reflect visual information search in multiple conditions and are amenable for cellular-level investigations”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In intrinsically motivated contexts, on the other hand, the search for information is a goal in itself – a process we would intuitively call “curiosity” or “interest”.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Information seeking obeys the imperative to reduce uncertainty and can be extrinsically or intrinsically motivated
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The fact that animals, and particularly humans, seem avidly to seek out information without an apparent ulterior motive suggests that the brain generates intrinsic rewards that assign value to information”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Information seeking obeys the imperative to reduce uncertainty and can be extrinsically or intrinsically motivated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
- The authors state this as an assumption: The brain generates intrinsic rewards related to learning or acquiring information.
None of the sources cited for this claim have been checked yet.
In the paper
“This idea resonates with the free-energy principle, which states that the possession of a large array of skills can be useful to avoid future surprises by “anticipating a changing and itinerant world””
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > The benefits and challenges of information seeking
What the sources say
- Not checked yetcites Friston 2010 source
- Not checked yetcites Friston, Thornton et al. 2012
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“A more complex form of “contextual novelty” (also called surprise) has been suggested to account for attentional attraction toward salient events”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > Randomness, novelty, surprise and uncertainty
What the sources say
- Not checked yetcites Boehnke, Berg et al. 2011
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“may be quantified using Bayesian inference as a difference between a prior and posterior world model (Itti and Baldi 2009) or as a high prediction error for high-confidence states (Oudeyer and Kaplan 2007).”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > Randomness, novelty, surprise and uncertainty
What the sources say
- Not checked yetcites Itti and Baldi 2009 source
- Not checked yetcites Oudeyer and Kaplan 2007
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Just as animals seek to fill gaps in their physical resources (e.g., energy, sex or wealth) they seek to fill gaps in their knowledge by taking learning-oriented actions.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > Information gap hypothesis of curiosity
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In neuroscience research, the most commonly considered exploration strategies are based on random action selection or automatic biases toward novel, surprising or uncertain events.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > Randomness, novelty, surprise and uncertainty
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Defined in this way, curiosity can be viewed as a deprivation phenomenon that seeks to fill a need similar to other reference-point phenomena or biological drives.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > Information gap hypothesis of curiosity
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This brings us back again to the imperative to minimize uncertainty about the state of the world, and suggests that this imperative is similar to a biological drive.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > Information gap hypothesis of curiosity
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, to define an information gap one has to monitor one’s level of uncertainty, again similar to eye movement control.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > Information gap hypothesis of curiosity
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, understanding information seeking requires that we understand how agents monitor their own competence and epistemic states, and specifically how they estimate their uncertainty and generate strategies for reducing that uncertainty.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Conclusions
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Finally, eye movements are natural indicators of the brain’s active information search.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Conclusions
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
In the paper
“According to the information gap theory, this type of curiosity arises because of a discrepancy between what the observer knows and what he would like to know, where knowledge can be measured with traditional measures of information.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > Information gap hypothesis of curiosity
What the sources say
- Not checked yetcites Lowenstein 1994 source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, this question requires that we understand the nature of intrinsic rewards that motivate information seeking and learning.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Conclusions
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By virtue of their amenability to neurophysiological investigations, may be an excellent model system for tackling this question, especially if studied in conjunction with computational approaches and the brain’s intrinsic reward and cognitive control mechanisms.”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Conclusions
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
- Rests on an assumption: The brain has intrinsic reward and cognitive control mechanisms.
- Rests on an assumption: Eye movements are amenable to neurophysiological investigations.
None of the sources cited for this claim have been checked yet.
In the paper
“or which respond selectively for behavioral change or the beginning of exploratory episodes (Isoda and Hikosaka 2007; Quilodran, Rothe et al. 2008).”
§ Information seeking, curiosity and attention: computational and neural mechanisms > Curiosity and autonomous exploration > Exploration based on learning progress (LP)
What the sources say
- Not checked yetcites Isoda and Hikosaka 2007
- Not checked yetcites Quilodran, Rothe et al. 2008
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The authors state this as an assumption.
The brain generates intrinsic rewards related to learning or acquiring information.
“it seems necessary to assume that the brain generates intrinsic rewards related to learning or acquiring information (Berlyne 1960).”
The paper relies on this without support.
Biological drives are prompted by salient and easily recognizable signs.
“while biological drives are prompted by salient and easily recognizable signs (e.g., somatic signals for hunger or sex)”
The paper relies on this without support.
In scientific research, the results of an experiment typically open up new questions that had not been foreseen, and it is not possible to estimate in advance the current entropy and the final desired state.
“In scientific research for example, the results of an experiment typically open up new questions that had not been foreseen, and it is not possible to estimate in advance what is the current entropy and the final, desired, state.”
The paper relies on this without support.
The central objective of developmental robotics is to design agents that can explore in open-ended environments and develop autonomously without a pre-programmed trajectory, based on their intrinsic interest.
“The central objective of developmental robotics is to design agents that can explore in open-ended environments and develop autonomously without a pre-programmed trajectory, based on their intrinsic interest.”
The paper relies on this without support.
The brain has intrinsic reward and cognitive control mechanisms.
“the brain’s intrinsic reward and cognitive control mechanisms.”
The paper relies on this without support.
Eye movements are amenable to neurophysiological investigations.
“By virtue of their amenability to neurophysiological investigations”
Citations
Cites
Genetic algorithms in search, optimization, and machine learning
(1989) · Choice Reviews Online · doi:10.5860/choice.27-0936 · Cited by 49,178
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 1×
“The search may use approaches ranging from reinforcement learning (Kaelbling, Littman et al. 1998; Sutton and Barto 1998; Jens Kober 2012), to stochastic optimization (Spall 2005), evolutionary techniques (Goldberg 1989), or Bayesian optimization (Jones, Schonlau et al.”
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Genetic Algorithms in Search, Optimization and Machine Learning
Goldberg, D. E., & Robson, D. (1988) · Cited by 18,391
Goldberg, D. E., & Shakespeare, W. (2002) · Cited by 16,088
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 1×
“This idea resonates with the free-energy principle, which states that the possession of a large array of skills can be useful to avoid future surprises by “anticipating a changing and itinerant world” (Friston 2010; Friston, Thornton et al. 2012).”
Efficient Global Optimization of Expensive Black-Box Functions
Jones, D. R., Schonlau, M., & Welch, W. J. (1998) · Journal of Global Optimization · doi:10.1023/a:1008306431147 · Cited by 8,070
Conflict, arousal, and curiosity.
Berlyne, D. E. (1960) · McGraw-Hill Book Company · doi:10.1037/11164-000 · Cited by 6,018 · cited 1×
“To explain such behaviors and the high degree of motivation associated with them, it seems necessary to assume that the brain generates intrinsic rewards related to learning or acquiring information (Berlyne 1960).”
Marr, D. (1982) · Cumulative Index of Computer Aided Architectural Design · Cited by 5,733
Proceedings of the 25th international conference on Machine learning
Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691
Barlow, H. B. (1983) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(83)90030-5 · Cited by 5,506
(1999) · Cited by 5,147
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160
The psychology of curiosity: A review and reinterpretation.
Loewenstein, G. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.116.1.75 · Cited by 2,738 · cited 2×
“In psychology, research on curiosity surged during the 1960s and 1970s and subsequently waned (for a comprehensive review, see (Lowenstein 1994)), and shows a mild revival in neuroscience in recent years (Kang, Hsu et al.”
Active Learning with Statistical Models
Cohn, D. A., Zoubin Ghahramani, A., & Jordan, M. I. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.295 · Cited by 2,054
Introduction to Stochastic Search and Optimization
Spall, J. C. (2003) · Wiley eBooks · doi:10.1002/0471722138 · Cited by 1,912
Proceedings of the 20th conference on Uncertainty in artificial intelligence
Meek, C., Chickering, M., & Halpern, J. Y. (2004) · Cited by 1,548
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
Bubeck, S. (2012) · now publishers, Inc. eBooks · doi:10.1561/9781601986276 · Cited by 1,500
Blake, A., & Yuille, A. (1993) · Cited by 1,383
Sutton, R. S. (1990) · Elsevier eBooks · doi:10.1016/b978-1-55860-141-3.50030-4 · Cited by 1,381
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
Bubeck, S., & Cesa-Bianchi, N. (2012) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000024 · Cited by 1,295
Bayesian surprise attracts human attention
Itti, L., & Baldi, P. (2009) · Vision Research · doi:10.1016/j.visres.2008.09.007 · Cited by 1,224 · cited 1×
“2011) and may be quantified using Bayesian inference as a difference between a prior and posterior world model (Itti and Baldi 2009) or as a high prediction error for high-confidence states (Oudeyer and Kaplan 2007).”
Abandoning Objectives: Evolution Through the Search for Novelty Alone
Lehman, J., & Stanley, K. O. (2011) · Evolutionary Computation · doi:10.1162/evco_a_00025 · Cited by 1,216
Development as a dynamic system
Smith, L. B., & Thelen, E. (2003) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(03)00156-6 · Cited by 1,132
Intrinsic Motivation Systems for Autonomous Mental Development
Oudeyer, P.-Y., Kaplan, F., & Hafner, V. V. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2006.890271 · Cited by 1,126
Self-citationINTRODUCTION TO STOCHASTIC SEARCH AND OPTIMIZATION: ESTIMATION, SIMULATION, AND CONTROL
Nowak, W. (2012) · Medical Entomology and Zoology · Cited by 1,030
The Neurobiology of Decision: Consensus and Controversy
Kable, J. W., & Glimcher, P. W. (2009) · Neuron · doi:10.1016/j.neuron.2009.09.003 · Cited by 928
Kang, M. J., Hsu, M., Krajbich, I. M., Loewenstein, G., McClure, S. M., Wang, J. T.-Y., & Camerer, C. F. (2009) · Psychological Science · doi:10.1111/j.1467-9280.2009.02402.x · Cited by 882
Eye–Hand Coordination in Object Manipulation
Johansson, R. S., Westling, G., Bäckström, A., & Flanagan, J. R. (2001) · Journal of Neuroscience · doi:10.1523/jneurosci.21-17-06917.2001 · Cited by 873
Near-Optimal Reinforcement Learning in Polynomial Time
Kearns, M. J., & Singh, S. P. (2002) · Machine Learning · doi:10.1023/a:1017984413808 · Cited by 861
What is intrinsic motivation? A typology of computational approaches
Oudeyer, P.-Y. (2007) · Frontiers in Neurorobotics · doi:10.3389/neuro.12.006.2007 · Cited by 811 · cited 1×
“2011) and may be quantified using Bayesian inference as a difference between a prior and posterior world model (Itti and Baldi 2009) or as a high prediction error for high-confidence states (Oudeyer and Kaplan 2007).”
Relating Introspective Accuracy to Individual Differences in Brain Structure
Fleming, S. M., Weil, R. S., Nagy, Z., Dolan, R. J., & Rees, G. (2010) · Science · doi:10.1126/science.1191883 · Cited by 809
Neural mechanisms for detecting and remembering novel events
Ranganath, C., & Rainer, G. (2003) · Nature Reviews Neuroscience · doi:10.1038/nrn1052 · Cited by 779 · cited 1×
“Sensory novelty, defined as a small number of stimulus exposures, is known to enhance neural responses throughout visual, frontal and temporal areas (Ranganath and Rainer 2003) and activate reward responsive dopamine-recipient areas.”
Eye guidance in natural vision: reinterpreting salience
Tatler, B. W., Hayhoe, M. M., Land, M. F., & Ballard, D. H. (2011) · Journal of Vision · doi:10.1167/11.5.5 · Cited by 775
Eye movements and the control of actions in everyday life
Land, M. F. (2006) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2006.01.002 · Cited by 667
Autonomous Mental Development by Robots and Animals
Weng, J., McClelland, J. L., Pentland, A., Sporns, O., Stockman, I. J., Sur, M., & Thelen, E. (2001) · Science · doi:10.1126/science.291.5504.599 · Cited by 655
The neurobiology of saccadic eye movements.
Wurtz, R. H., & Goldberg, M. E. (1989) · PubMed · Cited by 631
Curious model-building control systems
Schmidhuber, J. (1991) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1991.170605 · Cited by 601
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580
Cognitive Developmental Robotics: A Survey
Asada, M., Hosoda, K., Kuniyoshi, Y., Ishiguro, H., Inui, T., Yoshikawa, Y., Ogino, M., & Yoshida, C. T. (2009) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2009.2021702 · Cited by 574
Proceedings of the International Joint Conference on Neural Networks (IJCNN) 2005
(2005) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2005.1556327 · Cited by 553
Midbrain Dopamine Neurons Signal Preference for Advance Information about Upcoming Rewards
Bromberg-Martin, E. S., & Hikosaka, O. (2009) · Neuron · doi:10.1016/j.neuron.2009.06.009 · Cited by 548
Confidence in value-based choice
De Martino, B., Fleming, S. M., Garrett, N., & Dolan, R. J. (2013) · Nature Neuroscience · doi:10.1038/nn.3279 · Cited by 534
Predictability, Complexity, and Learning
Bialek, W., Nemenman, I. M., & Tishby, N. (2001) · Neural Computation · doi:10.1162/089976601753195969 · Cited by 524
Active learning of inverse models with intrinsically motivated goal exploration in robots
Baranès, A., & Oudeyer, P. (2012) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2012.05.008 · Cited by 498
Self-citationIntrinsically Motivated Reinforcement Learning: An Evolutionary Perspective
Singh, S. P., Lewis, R. L., Barto, A. G., & Sorg, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2051031 · Cited by 444
Neuronal Basis of Covert Spatial Attention in the Frontal Eye Field
Thompson, K., Biscoe, K. L., & Sato, T. R. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.0741-05.2005 · Cited by 440
A visual salience map in the primate frontal eye field
Thompson, K. G., & Bichot, N. P. (2005) · Progress in Brain Research · doi:10.1016/s0079-6123(04)47019-8 · Cited by 431
Switching from automatic to controlled action by monkey medial frontal cortex
Isoda, M., & Hikosaka, O. (2007) · Nature Neuroscience · doi:10.1038/nn1830 · Cited by 430 · cited 1×
“2013) or which respond selectively for behavioral change or the beginning of exploratory episodes (Isoda and Hikosaka 2007; Quilodran, Rothe et al. 2008).”
Prefrontal Contributions to Metacognition in Perceptual Decision Making
Fleming, S. M., Huijgen, J., & Dolan, R. J. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.6489-11.2012 · Cited by 391
Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control
Frey, J. C. (2004) · Journal of the American Statistical Association · doi:10.1198/jasa.2004.s362 · Cited by 383
Intrinsically Motivated Learning in Natural and Artificial Systems
Baldassarre, G., & Mirolli, M. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1 · Cited by 380
Free-Energy Minimization and the Dark-Room Problem
Friston, K., Thornton, C., & Clark, A. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00130 · Cited by 367
R-MAX: a general polynomial time algorithm for near-optimal reinforcement learning
Brafman, R. I., & Tennenholtz, M. (2001) · Cited by 366
Behavioral Shifts and Action Valuation in the Anterior Cingulate Cortex
Quilodran, R., Rothé, M., & Procyk, E. (2008) · Neuron · doi:10.1016/j.neuron.2007.11.031 · Cited by 359
Knowing how much you don't know: a neural organization of uncertainty estimates
Bach, D. R., & Dolan, R. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3289 · Cited by 339
Eye–Hand Coordination during Learning of a Novel Visuomotor Task
Sailer, U., Flanagan, J., & Johansson, R. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.2658-05.2005 · Cited by 313
A Computational Perspective on Visual Attention
Tsotsos, J. K. (2011) · The MIT Press eBooks · doi:10.7551/mitpress/9780262015417.001.0001 · Cited by 307
Why Red Doesn’t Sound Like a BellUnderstanding the feel of consciousness
O'Regan, J. K. (2011) · doi:10.1093/acprof:oso/9780199775224.001.0001 · Cited by 285
Distinct neural mechanisms of distractor suppression in the frontal and parietal lobe
Suzuki, M., & Gottlieb, J. (2013) · Nature Neuroscience · doi:10.1038/nn.3282 · Cited by 281
Self-citationNeural mechanisms underlying the induction and relief of perceptual curiosity
Jepma, M., Verdonschot, R. G., van Steenbergen, H., Rombouts, S. A. R. B., & Nieuwenhuis, S. (2012) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2012.00005 · Cited by 281
Information Theory of Decisions and Actions
Tishby, N., & Polani, D. (2011) · Perception-Action Cycle · doi:10.1007/978-1-4419-1452-1_19 · Cited by 267 · cited 1×
“Balancing the costs and benefits of information sampling can also be cast in an information theoretic perspective (Tishby and Polani 2011).”
What is reinforced by phasic dopamine signals?
Redgrave, P., GURNEY, K. N., & Reynolds, J. N. J. (2007) · Brain Research Reviews · doi:10.1016/j.brainresrev.2007.10.007 · Cited by 261
Düzel, E., Bunzeck, N., Guitart-Masip, M., & Düzel, S. (2010) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2009.08.006 · Cited by 249
Attention, learning, and the value of information
Gottlieb, J. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.034 · Cited by 241
Self-citationPredictive State Representations: A New Theory for Modeling Dynamical SystemsPreprint
Singh, S. P., James, M., & Rudary, M. R. (2012) · arXiv (Cornell University) · doi:10.5555/1036843.1036905 · Cited by 233
Reward modulates attention independently of action value in posterior parietal cortex
Peck, C. J., Jangraw, D. C., Suzuki, M., Efem, R., & Gottlieb, J. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.1929-09.2009 · Cited by 219
Self-citationNear-Bayesian exploration in polynomial time
Kolter, J. Z., & Ng, A. Y. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553441 · Cited by 218
Reinforcement Learning in Robotics: A Survey
Kober, J., & Peters, J. (2013) · Springer tracts in advanced robotics · doi:10.1007/978-3-319-03194-1_2 · Cited by 213
Adaptive Gaze Control in Natural Environments
Jovancevic, J., & Hayhoe, M. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.5570-08.2009 · Cited by 171 · cited 1×
“gaze is proactively guided by estimates of uncertainty comes from a virtual reality study where groups of observers walked along a track (Jovancevic-Misic and Hayhoe 2009).”
In search of the neural circuits of intrinsic motivation
Kaplan, F. (2007) · Frontiers in Neuroscience · doi:10.3389/neuro.01.1.1.017.2007 · Cited by 151
Attention and Associative Learning: From Brain to Behaviour
Mitchell, C. J., & Pelley, M. E. L. (2010) · Cited by 146
Free Energy, Value, and Attractors
Friston, K. J., & Ao, P. (2011) · Computational and Mathematical Methods in Medicine · doi:10.1155/2012/937860 · Cited by 146 · cited 1×
“Converging evidence suggests that, when deployed in the service of a task, eye movements may be explained by the simple imperative to sample information in order to reduce uncertainty regarding future states (Tatler, Hayhoe et al. 2011; Friston and Ao 2012).”
Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning Progress
Lopes, M. C., Lang, T., Toussaint, M., & Oudeyer, P. (2012) · Cited by 145
Self-citationIn Monkeys Making Value-Based Decisions, LIP Neurons Encode Cue Salience and Not Action Value
Leathers, M. L., & Olson, C. R. (2012) · Science · doi:10.1126/science.1226405 · Cited by 133
Comparison of Reward Modulation in the Frontal Eye Field and Caudate of the Macaque
Ding, L., & Hikosaka, O. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.0836-06.2006 · Cited by 132
Attention as a decision in information space
Gottlieb, J. P., & Balan, P. F. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.03.001 · Cited by 125
Self-citationTwo theories of attention: a review and a possible integration
Pearce, J. M., & Mackintosh, N. J. (2010) · Oxford University Press eBooks · Cited by 123
Exploration Bonuses and Dual Control
Dayan, P., & B, T. J. S. A. (1996) · Machine Learning · doi:10.1023/a:1018357105171 · Cited by 120
Robust Representation of Stable Object Values in the Oculomotor Basal Ganglia
Yasuda, M., Yamamoto, S., & Hikosaka, O. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.3438-12.2012 · Cited by 104
Oristaglio, J., Schneider, D. M., Balan, P. F., & Gottlieb, J. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1779-06.2006 · Cited by 102
The Neurobiology of Saccadic Eye Movements (Reviews of Oculomotor Research Vol. 3)
Donaldson, I. M. L. (1990) · Trends in Neurosciences · doi:10.1016/0166-2236(90)90154-3 · Cited by 80
Exploration in active learning
Thrun, S. (1998) · Cited by 78
Visualization of unsteady viscous flow around turbine blade
Jimbo, T., Biswas, D., Yokono, Y., & Niizeki, Y. (2008) · Journal of Visualization · doi:10.1007/bf03181903 · Cited by 78
Oudeyer, P., & Kaplan, F. (2006) · Connection Science · doi:10.1080/09540090600768567 · Cited by 72
Self-citationReward Design via Online Gradient Ascent
Sorg, J., Lewis, R. L., & Singh, S. P. (2010) · Cited by 67
Visual adaptation and novelty responses in the superior colliculus: Adaptation in the SC
Boehnke, S. E., Berg, D. J., Marino, R. A., Baldi, P. F., Itti, L., & Munoz, D. P. (2011) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07805.x · Cited by 60
Intrinsically Motivated Learning of Real-World Sensorimotor Skills with Developmental Constraints
Oudeyer, P., Baranès, A., & Kaplan, F. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_13 · Cited by 59
Self-citationProceedings of the International Joint Conference on Neural Networks 2003
(2003) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2003.1223795 · Cited by 58
The strategic student approach for life-long exploration and learning
Lopes, M. C., & Oudeyer, P. (2012) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2012.6400807 · Cited by 56
Self-citationLearning tactile skills through curious exploration
Pape, L., Oddo, C. M., Controzzi, M., Cipriani, C., Förster, A., Carrozza, M. C., & Schmidhuber, J. (2012) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2012.00006 · Cited by 51
Neuromorphic and Brain-Based Robots
(2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511994838 · Cited by 45
First experiments with PowerPlay
Srivastava, R. K., Steunebrink, B. R., & Schmidhuber, J. (2013) · Neural Networks · doi:10.1016/j.neunet.2013.01.022 · Cited by 42
Credit Assignment in Multiple Goal Embodied Visuomotor Behavior
Rothkopf, C. A., & Ballard, D. H. (2010) · Frontiers in Psychology · doi:10.3389/fpsyg.2010.00173 · Cited by 38 · cited 1×
“Computational models have also incorporated uncertainty-based strategies, generating biases toward actions or states that have high variance or entropy (Cohn, Ghahramani et al. 1996; Rothkopf and Ballard 2010).”
Moore, B. L., Doufas, A. G., & Pyeatt, L. D. (2010) · Anesthesia & Analgesia · doi:10.1213/ane.0b013e31820334a7 · Cited by 36
Curiosity-driven phonetic learning
Moulin-Frier, C., & Oudeyer, P. (2012) · doi:10.1109/devlrn.2012.6400583 · Cited by 36
Self-citationA Neural Correlate of Motivational Conflict in the Superior Colliculus of the Macaque
Isoda, M., & Hikosaka, O. (2008) · Journal of Neurophysiology · doi:10.1152/jn.90275.2008 · Cited by 34
Learning skills from play: Artificial curiosity on a Katana robot arm
Ngo, H., Luciw, M., Förster, A., et al. (2012) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2012.6252824 · Cited by 28
Oudeyer, P. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2009.2039057 · Cited by 26
Self-citationEmerging social awareness: Exploring intrinsic motivation in multiagent learning
Sequeira, P., Melo, F. S., Prada, R., & Paiva, A. (2011) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2011.6037325 · Cited by 17
Exploration from Generalization Mediated by Multiple Controllers
Dayan, P. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_4 · Cited by 15
Maximizing Fun by Creating Data with Easily Reducible Subjective Complexity
Schmidhuber, J. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_5 · Cited by 14 · cited 2×
“An alternative mechanism for targeted learning has been proposed in the field of developmental robotics, which eschews this difficulty by tracking an agent’s local learning progress without setting an absolute goal (Oudeyer, Kaplan et al. 2007; Oudeyer, Baranes et al. 2013; Schmidhuber 2013) (following an early formul…”
Haykin, S. (2012) · Cambridge University Press eBooks · doi:10.1017/cbo9780511818363.003 · Cited by 10
Kaplan, F., & Oudeyer, P. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511994838.011 · Cited by 10
Self-citationHogarth, L., Dickinson, A., & Duka, T. (2010) · Open Research Exeter (University of Exeter) · Cited by 9
First Experiments with PowerPlayPreprint
Srivastava, R. K., Steunebrink, B. R., & Schmidhuber, J. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1210.8385 · Cited by 7
Flagged first, then by citations
23 of the 132 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Nguyen, 2013, Socially guided intrinsic motivation for robot learning of motor skills, Auton Robots
Cited by
Sorted by citations
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
The Psychology and Neuroscience of Curiosity
Kidd, C., & Hayden, B. Y. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.010 · Cited by 1,026
Lieder, F., & Griffiths, T. L. (2020) · Behavioral and Brain Sciences · doi:10.1017/s0140525x1900061x · Cited by 892
What is consciousness, and could machines have it?
Dehaene, S., Lau, H., & Kouider, S. (2017) · Science · doi:10.1126/science.aan8871 · Cited by 855
Electronic skins and machine learning for intelligent soft robots
Shih, B., Shah, D., Li, J., Thuruthel, T. G., Park, Y.-L., Iida, F., Bao, Z., Kramer-Bottiglio, R., & Tolley, M. T. (2020) · Science Robotics · doi:10.1126/scirobotics.aaz9239 · Cited by 715
Metadata
- Authors
- Gottlieb, J., Oudeyer, P.-Y., Lopes, M., & Baranes, A.
- Affiliations
- Columbia University (US)
- Institut national de recherche en sciences et technologies du numérique (FR)
- Corresponding author
- Gottlieb, Jacqueline
- Venue
- Trends in Cognitive Sciences
- Publisher
- Elsevier BV
- Published
- 2013-11
- Volume
- 17
- Issue
- 11
- Pages
- 585-593
- ISSN
- 1364-6613 (ISSN-L)
- PMC
- PMC4193662
- OpenAlex
- W2013830186
- Semantic Scholar
- 61667ccec10748b39e274f2a13fcf9b7c4a9b275
- Cited by
- 784
- References
- 132
- Citations by year
- 2026: 43 · 2025: 57 · 2024: 77 · 2023: 165 · 2022: 70 · 2021: 68 · 2020: 64 · 2019: 74 · 2018: 42 · 2017: 43 · 2016: 31 · 2015: 21 · 2014: 26 · 2013: 2
- FWCI
- 4.42
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Trends in Cognitive Sciences (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- HAL (submittedVersion)
- Funders
- National Institutes of Health (4r01mh098039-05)
- National Institute of Mental Health (R01 MH-098039, R03MH091590)
- National Institute on Drug Abuse (R21DA030095)
- National Eye Institute (P30EY019007)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychological and Educational Research Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Keywords
- information seeking, curiosity, attention, exploration, novelty bias, surprise, information gain, eye movements, oculomotor system
- MeSH terms
- Animals, Artificial Intelligence, Attention, Brain, Exploratory Behavior, Humans, Information Seeking Behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
