The DOI matches its Crossref record; 128 of the 128 references it cites were found in a registry or an index.
Towards a neuroscience of active sampling and curiosity
Abstract
In natural behaviour, animals actively interrogate their environments using endogenously generated 'question-and-answer' strategies. However, in laboratory settings participants typically engage with externally imposed stimuli and tasks, and the mechanisms of active sampling remain poorly understood. We review a nascent neuroscientific literature that examines active-sampling policies and their relation to attention and curiosity. We distinguish between information sampling, in which organisms reduce uncertainty relevant to a familiar task, and information search, in which they investigate in an open-ended fashion to discover new tasks. We review evidence that both sampling and search depend on individual preferences over cognitive states, including attitudes towards uncertainty, learning progress and types of information. We propose that, although these preferences are non-instrumental and can on occasion interfere with external goals, they are important heuristics that allow organisms to cope with the high complexity of both sampling and search, and generate curiosity-driven investigations in large, open environments in which rewards are sparse and ex ante unknown.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Research has shown that because agents find pleasure in reducing manageable uncertainty, they are attracted to just-uncertain-enough environments”
From Active inference goes to school: the importance of active learning in the age of large language models (2024) · cites this paper as [51]
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Conflict, arousal, and curiosity.
Berlyne, D. E. (1960) · McGraw-Hill Book Company · doi:10.1037/11164-000 · Cited by 6,018
Humans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844
The psychology of curiosity: A review and reinterpretation.
Loewenstein, G. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.116.1.75 · Cited by 2,738
Sweet anticipation: music and the psychology of expectation
Huron, D., Marcus, R., Pearce, T., & Müllensiefen, D. (2007) · Choice Reviews Online · doi:10.5860/choice.44-4368 · Cited by 1,830
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779
Temporal Resolution of Uncertainty and Dynamic Choice Theory
Kreps, D. M., & Porteus, E. L. (1978) · Econometrica · doi:10.2307/1913656 · Cited by 1,753
The Normalization Model of Attention
Reynolds, J. H., & Heeger, D. J. (2009) · Neuron · doi:10.1016/j.neuron.2009.01.002 · Cited by 1,457
Huron, D. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/6575.001.0001 · Cited by 1,413
Research works printed but not linked: 7
- Forestier, S. & Oudeyer, P. Y. in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS) 3965–3972 (IEEE, 2016). · DOI
- Forestier, S. & Oudeyer, P. Y. in Proc. 39th Annual Meeting of the Cognitive Science Soc. 2013–2018 (Cogsci, 2017).
- Lopes, M. & Oudeyer, P.-Y. in Proc. IEEE Int. Conf. on Development and Learning and Epigenetic Robotics (ICDL) 1–8 (IEEE, 2012). · DOI
- Barto, A., Singh, S. & Chenatez, N. in Proc. 3rd Int. Conf. Dvp. Learn 112–119 (San Diego, CA, 2004).
- Schmidhuber, J. in Proc. Int. Joint Conf. Neural Networks 2, 1458–1463 (IEEE, 1991). · DOI
- Bellemare, M. et al. in Proc. Advances in Neural Information Processing Systems 29 Conf. 1471–1479 (NIPS, 2016).
- Kulkarni, T. D., Narasimhan, K., Saeedi, A. & Tenenbaum, J. B. in Proc. Advances in Neural Information Processing Systems 29 Conf. 3675–3683 (NIPS, 2016).
Cited by
Sorted by citations
Process Account of Curiosity and Interest: A Reward-Learning Perspective
Murayama, K., FitzGibbon, L., & Sakaki, M. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09499-9 · Cited by 334
Murayama, K. (2022) · Psychological Review · doi:10.1037/rev0000349 · Cited by 350
Hierarchical motor control in mammals and machines
Merel, J., Botvinick, M., & Wayne, G. (2019) · Nature Communications · doi:10.1038/s41467-019-13239-6 · Cited by 274
Gruber, M. J., & Ranganath, C. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.10.003 · Cited by 302
When Is More Uncertainty Better? A Model of Uncertainty Regulation and Effectiveness
Griffin, M. A., & Grote, G. (2020) · Academy of Management Review · doi:10.5465/amr.2018.0271 · Cited by 223
Metadata
- Authors
- Gottlieb, J., & Oudeyer, P.-Y.
- Affiliations
- Mortimer B. Zuckerman Mind Brain Behavior Institute (US)
- Institut national de recherche en sciences et technologies du numérique (FR)
- Corresponding author
- Gottlieb, Jacqueline
- Venue
- Nature Reviews Neuroscience
- Published
- 2018-11-05
- Volume
- 19
- Issue
- 12
- Pages
- 758-770
- ISSN
- 1471-003X (ISSN-L)
- 1471-0048
- OpenAlex
- W2899529983
- Cited by
- 494
- References
- 128
- Citations by year
- 2026: 39 · 2025: 68 · 2024: 53 · 2023: 136 · 2022: 49 · 2021: 67 · 2020: 55 · 2019: 26
- FWCI
- 5.90
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature reviews. Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- HAL (submittedVersion)
- Funders
- National Eye Institute (P30EY019007)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychological and Educational Research Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- active sampling, curiosity, information sampling, information search, attention, uncertainty, cognitive states, exploration, learning progress, sparse rewards, open environments
- MeSH terms
- Animals, Attention, Brain, Cognition, Exploratory Behavior, Humans, Neurosciences, Reward
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
