Meta-learning in Reinforcement Learning
Abstract
Meta-parameters in reinforcement learning should be tuned to the environmental dynamics and the animal performance. Here, we propose a biologically plausible meta-reinforcement learning algorithm for tuning these meta-parameters in a dynamic, adaptive manner. We tested our algorithm in both a simulation of a Markov decision task and in a non-linear control task. Our results show that the algorithm robustly finds appropriate meta-parameter values, and controls the meta-parameter time course, in both static and dynamic environments. We suggest that the phasic and tonic components of dopamine neuron firing can encode the signal required for meta-learning of reinforcement learning.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564 · see citing works
Aston-Jones, G., & Bloom, F. (1981) · The Journal of Neuroscience · doi:10.1523/jneurosci.01-08-00876.1981 · Cited by 1,960 · see citing works
An opponent-process theory of motivation. I. Temporal dynamics of affect
Solomon, R. L., & Corbit, J. D. (1974) · Psychological Review · doi:10.1037/h0036128 · Cited by 1,704 · see citing works
Reinforcement Learning in Continuous Time and Space
Doya, K. (2000) · Neural Computation · doi:10.1162/089976600300015961 · Cited by 961 · see citing works
Self-citationOpponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844 · see citing works
Metalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704 · see citing works
Self-citation5-HT and motor control: a hypothesis
Jacobs, B. L., & Fornal, C. A. (1993) · Trends in Neurosciences · doi:10.1016/0166-2236(93)90090-9 · Cited by 452 · see citing works
On the Complexity of Solving Markov Decision ProblemsPreprint
Littman, M. L., Dean, T., & Kaelbling, L. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.4971 · Cited by 434 · see citing works
A stochastic reinforcement learning algorithm for learning real-valued functions
Gullapalli, V. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90056-q · Cited by 292 · see citing works
The role of exploration in learning control
Thrun, S. (1992) · Cited by 227 · see citing works
Control of exploitation–exploration meta-parameter in reinforcement learning
Ishii, S., Yoshida, W., & Yoshimoto, J. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00056-4 · Cited by 209 · see citing works
Adapting bias by gradient descent: an incremental version of delta-bar-delta
Sutton, R. S. (1992) · Cited by 169 · see citing works
D2‐like dopamine receptor activation excites rat dorsal raphe 5‐HT neurons in vitro
Haj‐Dahmane, S. (2001) · European Journal of Neuroscience · doi:10.1046/j.0953-816x.2001.01616.x · Cited by 85 · see citing works
Dopaminergic regulation of cortical acetylcholine release: Effects of dopamine receptor agonists
Day, J. C., & Fibiger, H. C. (1993) · Neuroscience · doi:10.1016/0306-4522(93)90235-8 · Cited by 84 · see citing works
Maeda, T., Kojima, Y., Arai, R., Fujimiya, M., Kimura, H., Kitahama, K., & Geffard, M. (1991) · Comparative Biochemistry and Physiology Part C Comparative Pharmacology · doi:10.1016/0742-8413(91)90195-y · Cited by 49 · see citing works
A model of cerebellar metaplasticity.Preprint
Schweighofer, N., & Arbib, M. A. (1998) · Learning & Memory · doi:10.1101/lm.4.5.421 · Cited by 47 · see citing works
Self-citation
Flagged first, then by citations
1 of the 17 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Meta-learning with memory-augmented neural networks
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016) · International Conference on Machine Learning · Cited by 1,504 · see citing works
Meta-Learning in Neural Networks: A Survey
Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3079209 · Cited by 886 · see citing works
Deep Reinforcement Learning: A Survey
Wang, X., Wang, S., Liang, X., Zhao, D., Huang, J., Xu, X., Dai, B., & Miao, Q. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3207346 · Cited by 972 · see citing works
Reinforcement learning, fast and slow
Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.006 · Cited by 667 · see citing works
Neuroeconomics: Decision Making and the Brain
Rushworth, M. F. S., & Walton, M. E. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.005 · Cited by 669 · see citing works
Learning to reinforcement learn
Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., & Botvinick, M. (2016) · arXiv · doi:10.48550/arxiv.1611.05763 · Cited by 579 · see citing works
Prefrontal cortex as a meta-reinforcement learning system
Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., Hassabis, D., & Botvinick, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0147-8 · Cited by 596 · see citing works
One-shot Learning with Memory-Augmented Neural NetworksPreprint
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.06065 · Cited by 435 · see citing works
Medial and orbital frontal cortex in decision-making and flexible behavior
Klein-Flügge, M. C., Bongioanni, A., & Rushworth, M. F. S. (2022) · Neuron · doi:10.1016/j.neuron.2022.05.022 · Cited by 244 · see citing works
Deep reinforcement learning in computer vision: a comprehensive survey
Le, N., Rathour, V. S., Yamazaki, K., Luu, K., & Savvides, M. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-10061-9 · Cited by 265 · see citing works
Low-Serotonin Levels Increase Delayed Reward Discounting in Humans
Schweighofer, N., Bertin, M., Shishida, K., Okamoto, Y., Tanaka, S., Yamawaki, S., & Doya, K. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.4982-07.2008 · Cited by 255 · see citing works
Self-citationThe Exploration-Exploitation Dilemma: A Multidisciplinary Framework
Berger‐Tal, O., Nathan, J., Meroni, E., & Saltz, D. (2014) · PLoS ONE · doi:10.1371/journal.pone.0095693 · Cited by 256 · see citing works
Deep reinforcement learning in medical imaging: A literature review
Zhou, S., Le, N., Luu, K., Nguyen, H. V., & Ayache, N. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102193 · Cited by 222 · see citing works
Dual Adaptation Supports a Parallel Architecture of Motor Memory
Lee, J.-Y., & Schweighofer, N. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.1294-09.2009 · Cited by 202 · see citing works
Self-citationFusi, S., Asaad, W. F., Miller, E. K., & Wang, X.-J. (2007) · Neuron · doi:10.1016/j.neuron.2007.03.017 · Cited by 199 · see citing works
Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A. (2021) · IEEE Access · doi:10.1109/access.2021.3126658 · Cited by 200 · see citing works
“A promising research direction to tackle the RG problem could be the meta-learning strategy application in RL, in order to quickly adapt experience gained in simulation on the real system [115], [116].”
BackgroundTwenty-Five Lessons from Computational Neuromodulation
Dayan, P. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.027 · Cited by 190 · see citing works
Meta-learning in natural and artificial intelligence
Wang, J. X. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.01.002 · Cited by 171 · see citing works
Emotion in reinforcement learning agents and robots: a survey
Moerland, T. M., Broekens, J., & Jonker, C. M. (2018) · Machine Learning · doi:10.1007/s10994-017-5666-0 · Cited by 205 · see citing works
Teaching recurrent neural networks to infer global temporal structure from local examples
Kim, J. Z., Lu, Z., Nozari, E., Pappas, G. J., & Bassett, D. S. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00321-2 · Cited by 143 · see citing works
Reinforcement learning for HVAC control in intelligent buildings: A technical and conceptual review
Al Sayed, K., Boodi, A., Sadeghian Broujeny, R., & Beddiar, K. (2024) · Journal of Building Engineering · doi:10.1016/j.jobe.2024.110085 · Cited by 140 · see citing works
A Biophysically Based Neural Model of Matching Law Behavior: Melioration by Stochastic Synapses
Soltani, A., & Wang, X. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.5159-05.2006 · Cited by 127 · see citing works
Dorsal anterior cingulate-brainstem ensemble as a reinforcement meta-learner
Silvetti, M., Vassena, E., Abrahamse, E. L., & Verguts, T. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006370 · Cited by 130 · see citing works
Qiao, H., Chen, J., & Huang, X. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3071312 · Cited by 119 · see citing works
Federated Learning and Meta Learning: Approaches, Applications, and Directions
Liu, X., Deng, Y., Nallanathan, A., & Bennis, M. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3330910 · Cited by 116 · see citing works
Metaplasticity as a Neural Substrate for Adaptive Learning and Choice under Uncertainty
Farashahi, S. G., Donahue, C. H., Khorsand, P., Seo, H., Lee, D., & Soltani, A. (2017) · Neuron · doi:10.1016/j.neuron.2017.03.044 · Cited by 98 · see citing works
Adaptive properties of differential learning rates for positive and negative outcomes
Cazé, R. D., & van der Meer, M. A. A. (2013) · Biological Cybernetics · doi:10.1007/s00422-013-0571-5 · Cited by 100 · see citing works
Dopamine blockade impairs the exploration-exploitation trade-off in rats
Cinotti, F., Fresno, V., Aklil, N., Coutureau, É., Girard, B., Marchand, A. R., & Khamassi, M. (2019) · Scientific Reports · doi:10.1038/s41598-019-43245-z · Cited by 94 · see citing works
Robot Cognitive Control with a Neurophysiologically Inspired Reinforcement Learning Model
Khamassi, M., Lallée, S., Enel, P., Procyk, E., & Dominey, P. F. (2011) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2011.00001 · Cited by 96 · see citing works
Sex differences in learning from exploration
Chen, C. S., Knep, E., Han, A., Ebitz, R. B., & Grissom, N. M. (2021) · eLife · doi:10.7554/elife.69748 · Cited by 90 · see citing works
Khamassi, M., Quilodran, R., Enel, P., Dominey, P. F., & Procyk, E. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhu114 · Cited by 83 · see citing works
Use It and Improve It or Lose It: Interactions between Arm Function and Use in Humans Post-stroke
Hidaka, Y., Han, C. E., Wolf, S. L., Winstein, C. J., & Schweighofer, N. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002343 · Cited by 88 · see citing works
Self-citationKusy, M., & Zajdel, R. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2376703 · Cited by 86 · see citing works
Efficient reinforcement learning: computational theories, neuroscience and robotics
Kawato, M., & Samejima, K. (2007) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2007.03.004 · Cited by 80 · see citing works
The new robotics—towards human‐centered machines
Stefan Schaal, A. (2007) · HFSP journal · doi:10.2976/1.2748612 · Cited by 83 · see citing works
Humans Can Adopt Optimal Discounting Strategy under Real-Time Constraints
Schweighofer, N., Shishida, K., Han, C. E., Okamoto, Y., Tanaka, S. C., Yamawaki, S., & Doya, K. (2006) · PLoS Comput. Biol. · doi:10.1371/journal.pcbi.0020152 · Cited by 98 · see citing works
Self-citationGautam, M. (2023) · Electricity · doi:10.3390/electricity4040020 · Cited by 82 · see citing works
Neural mechanism for stochastic behaviour during a competitive game
Soltani, A., Lee, D., & Wang, X.-J. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.05.044 · Cited by 77 · see citing works
Meta-seg: A survey of meta-learning for image segmentation
Luo, S., Li, Y., Gao, P., Wang, Y., & Serikawa, S. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108586 · Cited by 79 · see citing works
A multifaceted vision of the human-AI collaboration: a comprehensive review
Puerta-Beldarrain, M., Gómez-Carmona, O., Sánchez-Corcuera, R., Casado-Mansilla, D., López-de-Ipiña, D., & Chen, L. (2025) · IEEE Access · doi:10.1109/access.2025.3536095 · Cited by 73 · see citing works
Najar, A., Bonnet, E., Bahrami, B., & Palminteri, S. (2020) · PLoS Biology · doi:10.1371/journal.pbio.3001028 · Cited by 66 · see citing works
Meta-control: From psychology to computational neuroscience
Eppinger, B., Goschke, T., & Musslick, S. (2021) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-021-00919-4 · Cited by 68 · see citing works
Meta-reinforcement learning via orbitofrontal cortex
Hattori, R., Hedrick, N. G., Jain, A., Chen, S., You, H., Hattori, M., Choi, J., Lim, B. K., Yasuda, R., & Komiyama, T. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01485-3 · Cited by 70 · see citing works
Cortical mechanisms for reinforcement learning in competitive games
Seo, H., & Lee, D. (2008) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2008.0158 · Cited by 65 · see citing works
Berniker, M., & Kording, K. P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002210 · Cited by 64 · see citing works
Serotonin and the Evaluation of Future Rewards
Schweighofer, N., Tanaka, S., & Doya, K. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.011 · Cited by 61 · see citing works
Self-citationReinforcement learning for the traveling salesman problem with refueling
Ottoni, A. L. C., Nepomuceno, E. G., de Oliveira, M. S., & de Oliveira, D. C. R. (2021) · Complex & Intelligent Systems · doi:10.1007/s40747-021-00444-4 · Cited by 60 · see citing works
Medial prefrontal cortex and the adaptive regulation of reinforcement learning parameters
Khamassi, M., Enel, P., Dominey, P. F., & Procyk, E. (2013) · Progress in brain research · doi:10.1016/b978-0-444-62604-2.00022-8 · Cited by 54 · see citing works
An Imperfect Dopaminergic Error Signal Can Drive Temporal-Difference Learning
Potjans, W., Diesmann, M., & Morrison, A. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001133 · Cited by 54 · see citing works
DAPath: Distance-aware knowledge graph reasoning based on deep reinforcement learning
Tiwari, P., Zhu, H., & Pandey, H. M. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.11.012 · Cited by 54 · see citing works
GASTO: A Fast Adaptive Graph Learning Framework for Edge Computing Empowered Task Offloading
Li, Y., Li, J., Lv, Z., Li, H., Wang, Y., & Xu, Z. (2023) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2023.3250395 · Cited by 55 · see citing works
Broekens, J., Kosters, W. A., & Verbeek, F. J. (2007) · Adaptive Behavior · doi:10.1177/1059712307084686 · Cited by 48 · see citing works
Brain-Inspired Learning, Perception, and Cognition: A Comprehensive Review
Jiao, L., Ma, M., He, P., Geng, X., Liu, X., Liu, F., Ma, W., Yang, S., Hou, B., & Tang, X. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3401711 · Cited by 50 · see citing works
Catecholaminergic modulation of meta-learning
Cook, J. L., Swart, J. C., Froböse, M. I., Diaconescu, A. O., Geurts, D. E. M., den Ouden, H. E. M., & Cools, R. (2019) · eLife · doi:10.7554/elife.51439 · Cited by 48 · see citing works
A reinforcement learning model of joy, distress, hope and fear
Broekens, J., Jacobs, E., & Jonker, C. M. (2015) · Connection Science · doi:10.1080/09540091.2015.1031081 · Cited by 42 · see citing works
Khamassi, M., Velentzas, G., Tsitsimis, T., & Tzafestas, C. S. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2018.2843122 · Cited by 43 · see citing works
Kusy, M., & Zajdel, R. (2014) · Applied Intelligence · doi:10.1007/s10489-014-0562-9 · Cited by 41 · see citing works
Federici, L., Scorsoglio, A., Zavoli, A., & Furfaro, R. (2022) · Acta Astronautica · doi:10.1016/j.actaastro.2022.08.047 · Cited by 41 · see citing works
Adaptable automation with modular deep reinforcement learning and policy transfer
Raziei, Z., & Moghaddam, M. (2021) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2021.104296 · Cited by 39 · see citing works
Goals, usefulness and abstraction in value-based choice
Martino, B. D., & Cortese, A. (2022) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.11.001 · Cited by 41 · see citing works
Meta-learning within Projective Simulation
Makmal, A., Melnikov, A. A., Dunjko, V., & Briegel, H. J. (2016) · IEEE Access · doi:10.1109/access.2016.2556579 · Cited by 37 · see citing works
Explainable Artificial Intelligence (xAI) Approaches and Deep Meta-Learning Models
Dağlarlı, E. (2020) · IntechOpen eBooks · doi:10.5772/intechopen.92172 · Cited by 38 · see citing works
Takemoto, A., Miwa, M., Koba, R., Yamaguchi, C., Suzuki, H., & Nakamura, K. (2014) · Neuroscience Research · doi:10.1016/j.neures.2014.10.001 · Cited by 34 · see citing works
Barsce, J. C., Palombarini, J. A., & Martínez, E. (2017) · Latin American Computer Conference (CLEI) · doi:10.1109/clei.2017.8226439 · Cited by 36 · see citing works
Tuning of reinforcement learning parameters applied to SOP using the Scott–Knott method
Ottoni, A. L. C., Nepomuceno, E. G., de Oliveira, M. S., & de Oliveira, D. C. R. (2019) · Soft Computing · doi:10.1007/s00500-019-04206-w · Cited by 34 · see citing works
Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems
Munir, M. S., Tran, N. H. K., Saad, W., & Hong, C. S. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3057960 · Cited by 33 · see citing works
A Comprehensive Overview and Survey of Recent Advances in Meta-LearningPreprint
Peng, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.11149 · Cited by 33 · see citing works
Learning environment-specific learning rates
Simoens, J., Verguts, T., & Braem, S. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011978 · Cited by 31 · see citing works
Li, X., Zhu, R., Ye, H., Jiang, C., & Benslimane, A. (2021) · Safety Science · doi:10.1016/j.ssci.2021.105315 · Cited by 31 · see citing works
Khamassi, M., Velentzas, G., Tsitsimis, T., & Tzafestas, C. S. (2017) · IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc.2017.33 · Cited by 29 · see citing works
Image-based Deep Reinforcement Learning for Autonomous Lunar Landing
Scorsoglio, A., Furfaro, R., Linares, R., & Gaudet, B. (2020) · AIAA Scitech 2020 Forum · doi:10.2514/6.2020-1910 · Cited by 31 · see citing works
Gao, M., Feng, X., Yu, H., & Li, X. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04574-9 · Cited by 28 · see citing works
Sun, Y., Pang, S., Zhao, Z., & Zhang, Y.-A. (2024) · Natural Resources Research · doi:10.1007/s11053-024-10396-4 · Cited by 28 · see citing works
Computational Modeling of Catecholamines Dysfunction in Alzheimer’s Disease at Pre-Plaque Stage
Caligiore, D., Silvetti, M., D’Amelio, M., Puglisi‐Allegra, S., & Baldassarre, G. (2020) · Journal of Alzheimer s Disease · doi:10.3233/jad-200276 · Cited by 28 · see citing works
MetaFollower: Adaptable personalized autonomous car following
Chen, X., Chen, K., Zhu, M., Yang, H., Shen, S., Wang, X., & Wang, Y. (2024) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2024.104872 · Cited by 27 · see citing works
Relative salience signaling within a thalamo-orbitofrontal circuit governs learning rate
Namboodiri, V. M. K., Hobbs, T. G., Trujillo‐Pisanty, I., Simon, R. C., Hjort, M. M., & Stuber, G. D. (2021) · Current Biology · doi:10.1016/j.cub.2021.09.037 · Cited by 25 · see citing works
Sugiyama, T., Schweighofer, N., & Izawa, J. (2023) · Nature Communications · doi:10.1038/s41467-023-39536-9 · Cited by 27 · see citing works
Self-citationStrategies for Affect-Controlled Action-Selection in Soar-RL
Hogewoning, E., Broekens, J., Eggermont, J., & Bovenkamp, E. G. P. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-73055-2_52 · Cited by 26 · see citing works
Intrinsic Motivation and Introspection in Reinforcement Learning
Merrick, K. E. (2012) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2012.2208457 · Cited by 26 · see citing works
Behavioral Game Theory and the Neural Basis of Strategic Choice
Camerer, C. F. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00013-0 · Cited by 26 · see citing works
On Affect and Self-adaptation: Potential Benefits of Valence-Controlled Action-Selection
Broekens, J., Kosters, W. A., & Verbeek, F. J. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-73053-8_36 · Cited by 26 · see citing works
Fast Task Adaptation Based on the Combination of Model-Based and Gradient-Based Meta Learning
Xu, Z., Chen, X., & Cao, L. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3028378 · Cited by 25 · see citing works
Analysis of Amino Acid Mixtures by Voltammetric Electronic Tongues and Artificial Neural Networks
Faura, G., González‐Calabuig, A., & del Valle, M. (2016) · Electroanalysis · doi:10.1002/elan.201600055 · Cited by 25 · see citing works
Best to go with what you know?
Lee, D. (2006) · Nature · doi:10.1038/441822a · Cited by 25 · see citing works
C2S : Class-aware client selection for effective aggregation in federated learning
Cao, M. Q., Zhang, Y., Ma, Z., & Zhao, M. (2022) · High-Confidence Computing · doi:10.1016/j.hcc.2022.100068 · Cited by 24 · see citing works
Dağlarlı, E. (2020) · Advances in systems analysis, software engineering, and high performance computing book series · doi:10.4018/978-1-7998-5101-1.ch003 · Cited by 23 · see citing works
Online meta-learning by parallel algorithm competition
Elfwing, S., Uchibe, E., & Doya, K. (2018) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3205455.3205486 · Cited by 23 · see citing works
Self-citationRobust interplanetary trajectory design under multiple uncertainties via meta-reinforcement learning
Federici, L., & Zavoli, A. (2023) · Acta Astronautica · doi:10.1016/j.actaastro.2023.10.018 · Cited by 21 · see citing works
Ottoni, A. L. C., Nepomuceno, E. G., & de Oliveira, M. S. (2018) · Journal of Control Automation and Electrical Systems · doi:10.1007/s40313-018-0374-y · Cited by 20 · see citing works
Fereoli, G., Schaub, H., & Lizia, P. D. (2024) · Journal of Spacecraft and Rockets · doi:10.2514/1.a36100 · Cited by 19 · see citing works
Robertazzi, F., Vissani, M., Schillaci, G., & Falotico, E. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.06.020 · Cited by 19 · see citing works
An integrative control theory perspective on consciousness.
Mansell, W. (2022) · Psychological Review · doi:10.1037/rev0000384 · Cited by 19 · see citing works
Reinforcement learning with state-dependent discount factor
Yoshida, N., Uchibe, E., & Doya, K. (2013) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2013.6652533 · Cited by 19 · see citing works
Self-citationFernández, F. C., & Caarls, W. (2018) · International Conference on Information Systems and Computer Science (INCISCOS) · doi:10.1109/inciscos.2018.00050 · Cited by 19 · see citing works
Reinforcement learning approaches to hippocampus-dependent flexible spatial navigation
Tessereau, C., O’Dea, R. D., Coombes, S., & Bast, T. (2021) · Brain and Neuroscience Advances · doi:10.1177/2398212820975634 · Cited by 18 · see citing works
Boosting Few-Shot Image Recognition Via Domain Alignment Prototypical Networks
Lu, J., Cao, Z. L., Wu, K., Zhang, G., & Zhang, C. (2018) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai.2018.00048 · Cited by 18 · see citing works
Kan, C., Chen, W., Lin, C., Wang, J., Lu, P.-J., Chan, M.-Y., & Wu, J.-T. (2018) · IEEE Access · doi:10.1109/access.2018.2815599 · Cited by 17 · see citing works
Meta Invariance Defense Towards Generalizable Robustness to Unknown Adversarial Attacks
Zhang, L., Zhou, Y., Yang, Y., & Gao, X. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3385745 · Cited by 17 · see citing works
Online Learning of a Memory for Learning Rates
Meier, F., Kappler, D., & Stefan Schaal, A. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460625 · Cited by 17 · see citing works
Learning continuous chaotic attractors with a reservoir computer
Smith, L. M., Kim, J. Z., Lu, Z., & Bassett, D. S. (2022) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0075572 · Cited by 17 · see citing works
- Authors
- Schweighofer, N., & Doya, K.
- Affiliations
- Advanced Telecommunications Research Institute International (JP)
- Corresponding author
- Schweighofer, Nicolas
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2003-01
- Volume
- 16
- Issue
- 1
- Pages
- 5-9
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W1996579288
- Cited by
- 261
- References
- 17
- Citations by year
- 2026: 7 · 2025: 22 · 2024: 20 · 2023: 22 · 2022: 20 · 2021: 26 · 2020: 17 · 2019: 13 · 2018: 16 · 2017: 16 · 2016: 8 · 2015: 7 · 2014: 11 · 2013: 7 · 2012: 6
- FWCI
- 2.84
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Networks (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Core Research for Evolutional Science and Technology
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Keywords
- meta-learning, meta-reinforcement learning, meta-parameters, dopaminergic neurons, parameter adaptation, Markov decision process, dynamic environments
- MeSH terms
- Algorithms, Artificial Intelligence, Decision Theory, Markov Chains, Reinforcement, Psychology, Stochastic Processes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review