CoChat
Sign Up
Cited by 804Psychological and Educational Research StudiesReinforcement Learning in RoboticsVisual perception and processing mechanisms

Information-seeking, curiosity, and attention: computational and neural mechanisms

Open access
Gottlieb, Jacqueline, Oudeyer, Pierre-Yves, Lopes, Manuel, Baranes, Adrien·2013·Trends in Cognitive Sciences
DOI 10.1016/j.tics.2013.09.001PMC PMC4193662
Open this paper in CoChat — free account

Abstract

Summary Intelligent animals devote much time and energy to exploring and obtaining information, but the underlying mechanisms are poorly understood. We review recent developments on this topic that have emerged from the traditionally separate fields of machine learning, eye movements in natural behavior, and studies of curiosity in psychology and neuroscience. These studies show that exploration may be guided by a family of mechanisms that range from automatic biases toward novelty or surprise, to systematic search for learning progress and information gain in curiosity-driven behavior. In addition, eye movements reflect visual information search in multiple conditions and are amenable for cellular-level investigations, suggesting that the oculomotor system is an excellent model system for understanding information sampling mechanisms.

AI snapshot

Written from the paper’s full text.

TL;DR

This review integrates machine learning, eye-movement, and curiosity research to argue that information seeking ranges from automatic novelty/surprise biases to targeted search for learning progress and information gain. It proposes the oculomotor system as a model for information sampling and highlights intrinsic rewards and uncertainty monitoring as key open questions.

Key findings

  • Information seeking is defined as actions whose primary goal is to obtain information and reduce uncertainty; it can be extrinsically motivated, as when a driver glances at a traffic light, or intrinsically motivated, as in curiosity or interest.
  • Task-related gaze is proactively guided by uncertainty: in visuo-manual tasks the eyes move ahead of the hand to collision or target points and wait there, and in a virtual-reality walking study subjects preferentially deployed gaze to oncoming pedestrians with a history of veering onto a collision course.
  • Curiosity-like mechanisms include Lowenstein’s information gap hypothesis, illustrated by a mystery novel with 10 equally likely suspects where the reader moves from high entropy to a single culprit, and developmental robotics approaches that reward learning progress rather than absolute competence.
  • Novelty, surprise, and uncertainty heuristics help discover new tasks but are inefficient in large open-ended spaces and risk trapping agents in unlearnable tasks; learning-progress systems may support efficient exploration but a quantitative neural response to learning progress has not been demonstrated empirically.

Methods

  • Narrative review synthesizing three traditionally separate lines: active learning and exploration in machine learning and robotics, eye movements in natural behavior, and curiosity in psychology and neuroscience.
  • No primary empirical sample or new data are reported; the paper uses conceptual and computational examples, including a POMDP traffic-light example, the information gap mystery-novel illustration, and developmental robotics simulations/robots.
  • Main analysis is conceptual integration and comparison of mechanisms across fields, not statistical meta-analysis.

Limitations

  • The paper states that understanding information-seeking mechanisms is in its infancy, that a quantitative response to learning progress distinct from phasic responses to novelty, surprise, or arousal has not been demonstrated empirically, and that the learning-progress approach entails a complex meta-cognitive architecture awaiting empirical verification.
  • The paper also states that it has not been empirically explored to what extent task-related informational eye movements are habitual, immutable aspects of a task versus rapidly responsive to changing epistemic conditions.
  • Evident from the text: as a review, it provides no new experimental data, and many claims about intrinsic rewards and their neural implementation remain speculative; the paper itself notes the question of active exploration is vast and cannot be exhaustively covered.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.tics.2013.09.001

Cites

110 of 110 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 51 cited sources checked · 0 supported · 0 flagged

  • Findingcitation

    In well-practiced tasks involving visuo-manual coordination, the eyes move ahead of the hand to critical locations such as potential collision or target points and wait there until the hand has cleared those locations.

    “In well-practiced tasks that involve visuo-manual coordination (such as moving an object to a target point) the eyes move ahead of the hand to critical locations such as potential collision or target points, and wait there until the hand has cleared those locations”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Johansson, Westling et al. 2001

      Not checked in the free run.

  • Findingcitation

    Computer simulations have shown that even if the objective fitness/reward function of an organism is to survive and reproduce, it may be more efficient to evolve a control architecture that encodes an innate surrogate reward function rewarding learning per se.

    “Singh et al. (Singh, Lewis et al. 2010) have shown with computer simulations that even if the objective fitness/reward function of an organism is to survive and reproduce, it may be more efficient to evolve a control architecture that encodes an innate surrogate reward function rewarding learning per se.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Singh, Lewis et al. 2010

      Not checked in the free run.

  • Findingcitation

    Eye movement patterns made while acquiring a task differ greatly from those made after learning.

    “Finally, the eye movements patterns made while acquiring a task differ greatly from those made after learning”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Sailer, Flanagan et al. 2005

      Not checked in the free run.

    • cites Land 2006 source

      Not checked in the free run.

  • Findingcitation

    Observers preferentially deploy gaze to oncoming pedestrians whose trajectory is expected to be uncertain relative to those who have never shown such deviations.

    “Subjects preferentially deployed gaze to oncoming pedestrians whose trajectory was expected to be uncertain (i.e., who had a history of veering onto a collision course) relative to those who had never shown such deviations.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Jovancevic-Misic and Hayhoe 2009

      Not checked in the free run.

  • Findingcitation

    Sensory novelty, defined as a small number of stimulus exposures, enhances neural responses throughout visual, frontal and temporal areas.

    “Sensory novelty, defined as a small number of stimulus exposures, is known to enhance neural responses throughout visual, frontal and temporal areas”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ranganath and Rainer 2003 source

      Not checked in the free run.

  • May combine two claims. 12
  • Classified as probably not a substantial claim. 22

  • May combine two claims.

    Findingreasoning

    The eyes never track the hand; instead they are strategically deployed to acquire new information.

    “Notably, the eyes never track the hand, which relies on motor and proprioceptive guidance and, for short periods of time, follows a predictable path; instead, they are strategically deployed to acquire new information.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisereasoning

    The learning space is vast, especially given the scarce time and energy available for learning.

    “One salient fact that emerges regarding this question is the sheer vastness of the learning space, especially given the scarce time and energy available for learning.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Deciding how to explore optimally when searching for a strategy is very difficult and almost intractable in the general case.

    “Deciding how to explore optimally when searching for strategy is a very difficult question, and is almost intractable in the general case.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisenone given

    A number of processes for open-ended exploration have been described that have individual strengths and weaknesses and may act in complementary fashion to accomplish the goals of information seeking.

    “A number of processes for open-ended exploration have been described in the literature, which, as we describe below, have individual strengths and weaknesses and may act in complementary fashion to accomplish these goals.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisereasoning

    Information gathering actions are a special type of intermediate step that obey the imperative to reduce uncertainty and adjudicate among competing interpretations.

    “Information gathering actions are a special type of intermediate step that obey the imperative to reduce uncertainty and adjudicate among competing interpretations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    Observers monitor the uncertainty or predictability of external items and use these quantities proactively to deploy gaze.

    “This suggests that the observers monitor the uncertainty or predictability of external items and use these quantities proactively to deploy gaze”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Strategies for solving a task, including those for generating informational actions, are not known in advance but must be learnt.

    “Strategies for solving a task, including those for generating informational actions, are not known in advance but must also be learnt.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Strategy-learning processes require significant exploration but are goal directed in that they seek to maximize a separate, extrinsic reward.

    “It is important to note that, while these processes require significant exploration, they are goal directed in the sense that they seek to maximize a separate, or extrinsic, reward”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The challenge of an information seeking mechanism is to learn efficiently a large repertoire of diverse skills given limited resources, and to avoid being trapped in unlearnable situations.

    “Thus, the challenge of an information seeking mechanism is to learn efficiently a large repertoire of diverse skills given its limited resources, and avoid being trapped in unlearnable situations.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    In the cognitive domain, infants must acquire a vast amount of factual knowledge, rules and social skills.

    “Similarly in the cognitive domain, infants must acquire a vast amount of factual knowledge, rules and social skills.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    The three lines of research on information seeking grapple with strikingly similar questions and propose overlapping mechanisms.

    “although using different terminology and methods, these three lines of research grapple with strikingly similar questions and propose overlapping mechanisms.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    The benefits of an intrinsic learning reward arise because of the limited cognitive capacities of the agent or because information or skills that are not immediately useful may be re-used in the future.

    “The benefits of such a system arise because of the limited cognitive capacities of the agent (i.e., its inability to solve the fitness function directly) (Sorg, Lewis et al. 2010; Lehman and Stanley 2011; Sequeira, Melo et al. 2011) or because information or skills that are not immediately useful may be re-used in the future.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Sorg, Lewis et al. 2010

      Not checked in the free run.

    • cites Lehman and Stanley 2011

      Not checked in the free run.

    • cites Sequeira, Melo et al. 2011

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisereasoning

    Exploratory actions are distinct from other motor acts in that their primary goal is not to exert force on the world but to alter the observer's epistemic state.

    “Although exploratory actions can involve physical acts, they are distinct from other motor acts in that their primary goal is not to exert force on the world, but to alter the observer’s epistemic state.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    Eye movements reflect visual information search in multiple conditions and are amenable to cellular-level investigations.

    “eye movements reflect visual information search in multiple conditions and are amenable for cellular-level investigations”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisenone given

    In intrinsically motivated contexts, the search for information is a goal in itself, a process called curiosity or interest.

    “In intrinsically motivated contexts, on the other hand, the search for information is a goal in itself – a process we would intuitively call “curiosity” or “interest”.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    The fact that animals and humans seek information without an apparent ulterior motive suggests that the brain generates intrinsic rewards that assign value to information.

    “The fact that animals, and particularly humans, seem avidly to seek out information without an apparent ulterior motive suggests that the brain generates intrinsic rewards that assign value to information”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The brain generates intrinsic rewards related to learning or acquiring information.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The free-energy principle states that the possession of a large array of skills can be useful to avoid future surprises by anticipating a changing and itinerant world.

    “This idea resonates with the free-energy principle, which states that the possession of a large array of skills can be useful to avoid future surprises by “anticipating a changing and itinerant world””
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Friston 2010 source

      Not checked in the free run.

    • cites Friston, Thornton et al. 2012

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Contextual novelty, also called surprise, has been suggested to account for attentional attraction toward salient events.

    “A more complex form of “contextual novelty” (also called surprise) has been suggested to account for attentional attraction toward salient events”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Boehnke, Berg et al. 2011

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Contextual novelty may be quantified using Bayesian inference as a difference between a prior and posterior world model or as a high prediction error for high-confidence states.

    “may be quantified using Bayesian inference as a difference between a prior and posterior world model (Itti and Baldi 2009) or as a high prediction error for high-confidence states (Oudeyer and Kaplan 2007).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Itti and Baldi 2009 source

      Not checked in the free run.

    • cites Oudeyer and Kaplan 2007

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Animals seek to fill gaps in their physical resources and seek to fill gaps in their knowledge by taking learning-oriented actions.

    “Just as animals seek to fill gaps in their physical resources (e.g., energy, sex or wealth) they seek to fill gaps in their knowledge by taking learning-oriented actions.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    In neuroscience research, the most commonly considered exploration strategies are based on random action selection or automatic biases toward novel, surprising or uncertain events.

    “In neuroscience research, the most commonly considered exploration strategies are based on random action selection or automatic biases toward novel, surprising or uncertain events.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Curiosity can be viewed as a deprivation phenomenon that seeks to fill a need similar to other reference-point phenomena or biological drives.

    “Defined in this way, curiosity can be viewed as a deprivation phenomenon that seeks to fill a need similar to other reference-point phenomena or biological drives.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The imperative to minimize uncertainty about the state of the world is similar to a biological drive.

    “This brings us back again to the imperative to minimize uncertainty about the state of the world, and suggests that this imperative is similar to a biological drive.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    To define an information gap one has to monitor one's level of uncertainty.

    “Second, to define an information gap one has to monitor one’s level of uncertainty, again similar to eye movement control.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Understanding information seeking requires understanding how agents monitor their own competence and epistemic states, and specifically how they estimate their uncertainty and generate strategies for reducing that uncertainty.

    “First, understanding information seeking requires that we understand how agents monitor their own competence and epistemic states, and specifically how they estimate their uncertainty and generate strategies for reducing that uncertainty.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Eye movements are natural indicators of the brain's active information search.

    “Finally, eye movements are natural indicators of the brain’s active information search.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Specific epistemic curiosity arises because of a discrepancy between what the observer knows and what he would like to know, where knowledge can be measured with traditional measures of information.

    “According to the information gap theory, this type of curiosity arises because of a discrepancy between what the observer knows and what he would like to know, where knowledge can be measured with traditional measures of information.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Lowenstein 1994 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Understanding information seeking requires understanding the nature of intrinsic rewards that motivate information seeking and learning.

    “Second, this question requires that we understand the nature of intrinsic rewards that motivate information seeking and learning.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Eye movements may be an excellent model system for tackling the question of active exploration, especially if studied in conjunction with computational approaches and the brain's intrinsic reward and cognitive control mechanisms.

    “By virtue of their amenability to neurophysiological investigations, may be an excellent model system for tackling this question, especially if studied in conjunction with computational approaches and the brain’s intrinsic reward and cognitive control mechanisms.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The brain has intrinsic reward and cognitive control mechanisms.
    • Rests on an assumption: Eye movements are amenable to neurophysiological investigations.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Possible candidates for a meta-cognitive system for monitoring learning progress include frontal systems that respond selectively for behavioral change or the beginning of exploratory episodes.

    “or which respond selectively for behavioral change or the beginning of exploratory episodes (Isoda and Hikosaka 2007; Quilodran, Rothe et al. 2008).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Isoda and Hikosaka 2007

      Not checked in the free run.

    • cites Quilodran, Rothe et al. 2008

      Not checked in the free run.

30 of 30

  • The authors state this as an assumption.

    The brain generates intrinsic rewards related to learning or acquiring information.

    “it seems necessary to assume that the brain generates intrinsic rewards related to learning or acquiring information (Berlyne 1960).”

  • The paper relies on this without support.

    Biological drives are prompted by salient and easily recognizable signs.

    “while biological drives are prompted by salient and easily recognizable signs (e.g., somatic signals for hunger or sex)”

  • The paper relies on this without support.

    In scientific research, the results of an experiment typically open up new questions that had not been foreseen, and it is not possible to estimate in advance the current entropy and the final desired state.

    “In scientific research for example, the results of an experiment typically open up new questions that had not been foreseen, and it is not possible to estimate in advance what is the current entropy and the final, desired, state.”

  • The paper relies on this without support.

    The central objective of developmental robotics is to design agents that can explore in open-ended environments and develop autonomously without a pre-programmed trajectory, based on their intrinsic interest.

    “The central objective of developmental robotics is to design agents that can explore in open-ended environments and develop autonomously without a pre-programmed trajectory, based on their intrinsic interest.”

  • The paper relies on this without support.

    The brain has intrinsic reward and cognitive control mechanisms.

    “the brain’s intrinsic reward and cognitive control mechanisms.”

  • The paper relies on this without support.

    Eye movements are amenable to neurophysiological investigations.

    “By virtue of their amenability to neurophysiological investigations”
6 of 6

Cites

Flagged first, then by citations

22 of the 132 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. Nguyen, 2013, Socially guided intrinsic motivation for robot learning of motor skills, Auton Robots

Cited by

Sorted by citations

Related

Authors
Gottlieb, J., Oudeyer, P.-Y., Lopes, M., & Baranes, A.
Affiliations
  • Columbia University (US)
  • Institut national de recherche en sciences et technologies du numérique (FR)
Corresponding author
Gottlieb, Jacqueline
ORCID
Venue
Trends in Cognitive Sciences
Publisher
Elsevier BV
Published
2013-11
Volume
17
Issue
11
Pages
585-593
ISSN
1364-6613 (ISSN-L)
DOI
10.1016/j.tics.2013.09.001
PMC
PMC4193662
OpenAlex
W2013830186
Semantic Scholar
61667ccec10748b39e274f2a13fcf9b7c4a9b275
Cited by
784
References
132
Citations by year
2026: 43 · 2025: 57 · 2024: 77 · 2023: 165 · 2022: 70 · 2021: 68 · 2020: 64 · 2019: 74 · 2018: 42 · 2017: 43 · 2016: 31 · 2015: 21 · 2014: 26 · 2013: 2
FWCI
4.42
Type
journal article
Language
en
Open access
Licence
www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations
  • Trends in Cognitive Sciences (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • HAL (submittedVersion)
Funders
  • National Institutes of Health (4r01mh098039-05)
  • National Institute of Mental Health (R01 MH-098039, R03MH091590)
  • National Institute on Drug Abuse (R21DA030095)
  • National Eye Institute (P30EY019007)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychological and Educational Research Studies
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
Keywords
information seeking, curiosity, attention, exploration, novelty bias, surprise, information gain, eye movements, oculomotor system
MeSH terms
Animals, Artificial Intelligence, Attention, Brain, Exploratory Behavior, Humans, Information Seeking Behavior

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review