Evolving internal reinforcers for an intrinsically motivated reinforcement-learning robot
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
Intrinsically motivated reinforcement learning (IMRL) has been proposed as a framework within which agents exploit "internal reinforcement" to acquire general-purpose building-block behaviors ("skills") which can be later combined for solving several specific tasks. The architectures so far proposed within this framework are limited in that: (1) they use hardwired "salient events" to form and train skills, and this limits agents' autonomy; (2) they are applicable only to problems with abstract states and actions, as grid-world problems. This paper proposes solutions to these problems in the form of a hierarchical reinforcement-learning architecture that: (1) exploits evolutionary robotics techniques so to allow the system to autonomously discover "salient events"; (2) uses neural networks so to allow the system to cope with continuous states and noisy environments. The viability of the proposed approach is demonstrated with a simulated robotic scenario.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Motivation reconsidered: The concept of competence.
White, R. W. (1959) · Psychological Review · doi:10.1037/h0040934 · Cited by 5,681 · cited 2×
“Intrinsically Motivated Reinforcement Learning The approach proposed here is inspired by the IMRL framework [3, 8, 9], which in turn builds upon psychological theories of motivation [11, 12], recent advances in the neuroscience of reward systems [13-16], and machine learning research on reinforcement learning [17, 18].”
BackgroundMethodBetween MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 2×
“In this respect, it was suggested that the phasic release of the neuromodulator dopamine by midbrain neurons serves the function of signaling the occurrence of unpredicted reward, in a way closely similar to the temporal difference prediction error posed by standard reinforcement learning algorithms [13, 14].”
MethodResultIntrinsic Motivation Systems for Autonomous Mental Development
Oudeyer, P.-Y., Kaplan, F., & Hafner, V. V. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2006.890271 · Cited by 1,126
Reward, Motivation, and Reinforcement Learning
Dayan, P., & Balleine, B. W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00963-7 · Cited by 855
The short-latency dopamine signal: a role in discovering novel actions?
Redgrave, P., & Gurney, K. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn2022 · Cited by 762
Cited by
Formal theory of creativity, fun, and intrinsic motivation (1990–2010)
Schmidhuber, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2056368 · Cited by 747
“They speed up skill learning by rewarding a top level, module-selecting RL agent in proport ion to the TD error of the selected module [63]—compare Section III-B.”
BackgroundActive inference and epistemic value
Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., & Pezzulo, G. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1020053 · Cited by 731
Intrinsically Motivated Reinforcement Learning: An Evolutionary Perspective
Singh, S., Lewis, R. L., Barto, A. G., & Sorg, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2051031 · Cited by 417
Murayama, K. (2022) · Psychological Review · doi:10.1037/rev0000349 · Cited by 304
Intrinsic motivations and open-ended development in animals, humans, and robots: an overview
Baldassarre, G., Stafford, T., Mirolli, M., Redgrave, P., Ryan, R. M., & Barto, A. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00985 · Cited by 73
Self-citation
Metadata
- Authors
- Schembri, M., Mirolli, M., & Baldassarre, G.
- Venue
- 2007 IEEE 6th International Conference on Development and Learning
- Publisher
- IEEE
- Published
- 2007-07
- Pages
- 282-287
- OpenAlex
- W2167815629
- Cited by
- 77
- References
- 7
- FWCI
- 4.34
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.