The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index.
Meta-learning in Reinforcement Learning
Abstract
Meta-parameters in reinforcement learning should be tuned to the environmental dynamics and the animal performance. Here, we propose a biologically plausible meta-reinforcement learning algorithm for tuning these meta-parameters in a dynamic, adaptive manner. We tested our algorithm in both a simulation of a Markov decision task and in a non-linear control task. Our results show that the algorithm robustly finds appropriate meta-parameter values, and controls the meta-parameter time course, in both static and dynamic environments. We suggest that the phasic and tonic components of dopamine neuron firing can encode the signal required for meta-learning of reinforcement learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Aston-Jones, G., & Bloom, F. (1981) · The Journal of Neuroscience · doi:10.1523/jneurosci.01-08-00876.1981 · Cited by 1,960
An opponent-process theory of motivation. I. Temporal dynamics of affect
Solomon, R. L., & Corbit, J. D. (1974) · Psychological Review · doi:10.1037/h0036128 · Cited by 1,704
Reinforcement Learning in Continuous Time and Space
Doya, K. (2000) · Neural Computation · doi:10.1162/089976600300015961 · Cited by 995
Self-citationOpponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844
Metalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704
Self-citation5-HT and motor control: a hypothesis
Jacobs, B. L., & Fornal, C. A. (1993) · Trends in Neurosciences · doi:10.1016/0166-2236(93)90090-9 · Cited by 452
On the Complexity of Solving Markov Decision ProblemsPreprint
Littman, M. L., Dean, T., & Kaelbling, L. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.4971 · Cited by 434
A stochastic reinforcement learning algorithm for learning real-valued functions
Gullapalli, V. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90056-q · Cited by 292
The role of exploration in learning control
Thrun, S. (1992) · Cited by 226
Research works printed but not linked: 1
- Tanaka, 2002, Functional MRI study of short-term and long-term prediction of reward, Proceedings of the Eighth International Conference on Functional Mapping of the Human Brain, Sendai, Japan, 1062
Cited by
Sorted by citations
Meta-learning with memory-augmented neural networks
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016) · International Conference on Machine Learning · Cited by 1,504
Meta-Learning in Neural Networks: A Survey
Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3079209 · Cited by 886
Deep Reinforcement Learning: A Survey
Wang, X., Wang, S., Liang, X., Zhao, D., Huang, J., Xu, X., Dai, B., & Miao, Q. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3207346 · Cited by 972
“The main idea of meta-reinforcement learning [149] is to apply the prior knowledge learned from a large number of RL tasks to new RL tasks, and to improve the learning speed and generalization ability of the agent.”
BackgroundReinforcement learning, fast and slow
Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.006 · Cited by 667
“Meta-learning occurs when one learning system progressively adjusts the operation of a second learning system, such that the latter operates with increasing speed and efficiency [83, 84, 85, 86, 87].”
Neuroeconomics: Decision Making and the Brain
Rushworth, M. F. S., & Walton, M. E. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.005 · Cited by 669
Metadata
- Authors
- Schweighofer, N., & Doya, K.
- Affiliations
- Advanced Telecommunications Research Institute International (JP)
- Corresponding author
- Schweighofer, Nicolas
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2003-01
- Volume
- 16
- Issue
- 1
- Pages
- 5-9
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W1996579288
- Cited by
- 261
- References
- 17
- Citations by year
- 2026: 7 · 2025: 22 · 2024: 20 · 2023: 22 · 2022: 20 · 2021: 26 · 2020: 17 · 2019: 13 · 2018: 16 · 2017: 16 · 2016: 8 · 2015: 7 · 2014: 11 · 2013: 7 · 2012: 6
- FWCI
- 2.84
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Networks (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Core Research for Evolutional Science and Technology
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Keywords
- meta-learning, meta-reinforcement learning, meta-parameters, dopaminergic neurons, parameter adaptation, Markov decision process, dynamic environments
- MeSH terms
- Algorithms, Artificial Intelligence, Decision Theory, Markov Chains, Reinforcement, Psychology, Stochastic Processes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
