Curiosity Search: Producing Generalists by Encouraging Individuals to Continually Explore and Acquire Skills throughout Their Lifetime
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Natural animals are renowned for their ability to acquire a diverse and general skill set over the course of their lifetime. However, research in artificial intelligence has yet to produce agents that acquire all or even most of the available skills in non-trivial environments. One candidate algorithm for encouraging the production of such individuals is Novelty Search, which pressures organisms to exhibit different behaviors from other individuals. However, we hypothesized that Novelty Search would produce sub-populations of specialists, in which each individual possesses a subset of skills, but no one organism acquires all or most of the skills. In this paper, we propose a new algorithm called Curiosity Search, which is designed to produce individuals that acquire as many skills as possible during their lifetime. We show that in a multiple-skill maze environment, Curiosity Search does produce individuals that explore their entire domain, while a traditional implementation of Novelty Search produces specialists. However, we reveal that when modified to encourage intra-life behavioral diversity, Novelty Search can produce organisms that explore almost as much of their environment as Curiosity Search, although Curiosity Search retains a significant performance edge. Finally, we show that Curiosity Search is a useful helper objective when combined with Novelty Search, producing individuals that acquire significantly more skills than either algorithm alone.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Intrinsic Motivation Systems for Autonomous Mental Development
Oudeyer, P.-Y., Kaplan, F., & Hafner, V. V. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2006.890271 · Cited by 1,126
Robots that can adapt like animals
Cully, A., Clune, J., Tarapore, D., & Mouret, J.-B. (2015) · Nature · doi:10.1038/nature14422 · Cited by 979 · cited 2×
“Evolutionary Algorithms (EAs) have proven useful in a variety of fields, including space exploration [1], automated damage recovery [2], and evolving gaits for robots [3, 4].”
BackgroundMethodSelf-citationMesolimbocortical and nigrostriatal dopamine responses to salient non-reward events
Horvitz, J. C. (2000) · Neuroscience · doi:10.1016/s0306-4522(00)00019-1 · Cited by 967
Reward, Motivation, and Reinforcement Learning
Dayan, P., & Balleine, B. W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00963-7 · Cited by 855
Mammalian Play: Training for the Unexpected
Spinka, M., Newberry, R. C., & Bekoff, M. (2001) · The Quarterly Review of Biology · doi:10.1086/393866 · Cited by 817 · cited 3×
“For example, many juvenile animals spend much of their time playing, which is thought to facilitate their ability to learn skills for surviving in unforeseen, stressful, or chaotic situations [8].”
BackgroundLearning tactile skills through curious exploration
Pape, L., Oddo, C. M., Controzzi, M., Cipriani, C., Förster, A., Carrozza, M. C., & Schmidhuber, J. (2012) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2012.00006 · Cited by 51 · cited 1×
“…motivation, which attempt to imitate the curiosity seen in animals, have resulted in agents that can successfully interact with objects in a playpen environment [7], autonomously identify and pursue goals to maximize their understanding of the world [21], and even learn tactile skills [22].”
Background
Cited by
Clune, J. (2019) · arXiv · doi:10.48550/arxiv.1905.10985 · Cited by 82
“In an algorithm called Curiosity Search [173], we later showed that encouraging an agent to visit as many different locations within its lifetime (i.e. within an episode) as possible improved performance on the Atari game Montezuma’s Revenge, a notoriously difficult, sparse, hard-exploration…”
MethodSelf-citationEvolution Through Large Models
Lehman, J., Gordon, J., Jain, S., Ndousse, K., Yeh, C., & Stanley, K. O. (2024) · Genetic and Evolutionary Computation · doi:10.1007/978-981-99-3814-8_11 · Cited by 74
Evolution through Large Models
Lehman, J., Gordon, J., Jain, S., Ndousse, K., Yeh, C., & Stanley, K. O. (2022) · arXiv · doi:10.48550/arxiv.2206.08896 · Cited by 74
Huizinga, J., & Clune, J. (2022) · Evolutionary Computation · doi:10.1162/evco_a_00301 · Cited by 20
Self-citationMulti-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft
Kanitscheider, I., Huizinga, J., Farhi, D., Guss, W. H., Houghton, B., Sampedro, R., Zhokhov, P., Baker, B., Ecoffet, A., Tang, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.14876 · Cited by 9
“• An exploration bonus (as an auxiliary reward) to perform tasks unrelated to the current goal (a.k.a. curiosity search [2, 3]) substantially improves performance (Figure 1, green).”
Self-citation
Metadata
- Authors
- Stanton, C., & Clune, J.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-09-02
- Volume
- 11
- Issue
- 9
- Pages
- e0162235
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2518130841
- Cited by
- 27
- References
- 6
- FWCI
- 3.04
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Curiosity Search: Producing Generalists by Encouraging Individuals to Continually Explore and Acquire Skills throughout Their Lifetime. https://app.cochat.ai/papers/10.1371/journal.pone.0162235
- The paper
- Stanton, C., & Clune, J. (2016). Curiosity Search: Producing Generalists by Encouraging Individuals to Continually Explore and Acquire Skills throughout Their Lifetime. PLOS ONE. https://doi.org/10.1371/journal.pone.0162235
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.