The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index.
Meta-learning in Reinforcement Learning
Abstract
Meta-parameters in reinforcement learning should be tuned to the environmental dynamics and the animal performance. Here, we propose a biologically plausible meta-reinforcement learning algorithm for tuning these meta-parameters in a dynamic, adaptive manner. We tested our algorithm in both a simulation of a Markov decision task and in a non-linear control task. Our results show that the algorithm robustly finds appropriate meta-parameter values, and controls the meta-parameter time course, in both static and dynamic environments. We suggest that the phasic and tonic components of dopamine neuron firing can encode the signal required for meta-learning of reinforcement learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Aston-Jones, G., & Bloom, F. (1981) · The Journal of Neuroscience · doi:10.1523/jneurosci.01-08-00876.1981 · Cited by 1,960
An opponent-process theory of motivation. I. Temporal dynamics of affect
Solomon, R. L., & Corbit, J. D. (1974) · Psychological Review · doi:10.1037/h0036128 · Cited by 1,704
Reinforcement Learning in Continuous Time and Space
Doya, K. (2000) · Neural Computation · doi:10.1162/089976600300015961 · Cited by 995
Self-citationOpponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844
Metalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704
Self-citation5-HT and motor control: a hypothesis
Jacobs, B. L., & Fornal, C. A. (1993) · Trends in Neurosciences · doi:10.1016/0166-2236(93)90090-9 · Cited by 452
On the Complexity of Solving Markov Decision ProblemsPreprint
Littman, M. L., Dean, T., & Kaelbling, L. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.4971 · Cited by 434
A stochastic reinforcement learning algorithm for learning real-valued functions
Gullapalli, V. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90056-q · Cited by 292
The role of exploration in learning control
Thrun, S. (1992) · Cited by 226
Research works printed but not linked: 1
- Tanaka, 2002, Functional MRI study of short-term and long-term prediction of reward, Proceedings of the Eighth International Conference on Functional Mapping of the Human Brain, Sendai, Japan, 1062
Cited by
Sorted by citations
Items 101–200 of 260
Few Is Enough: Task-Augmented Active Meta-learning for Brain Cell Classification
Yuan, P., Mobiny, A., Jahanipour, J., Li, X., Cicalese, P. A., Roysam, B., Patel, V. M., Maric, D., & Nguyen, H. V. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59710-8_36 · Cited by 15
Toward evolutionary and developmental intelligence
Doya, K., & Taniguchi, T. (2019) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2019.04.006 · Cited by 21
Self-citationThe Neural Basis of Strategic Choice
Camerer, C. F., & Hare, T. A. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00025-5 · Cited by 15
Mechanisms for Stochastic Decision Making in the Primate Frontal Cortex
Lee, D., & Wang, X. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00031-2 · Cited by 15
Zheng, W., Yan, L., Gou, C., & Wang, F. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22372 · Cited by 15
“From above and Table 11, we can get the following three points: First, metric‐based methods purpose that samples of the same class are close to each other, and examples of the different classes are far away from each other by simulating the metric distribution among samples.(30,142) Generally speaking, the neural netw…”
Khamassi, M., Wilson, C. R. E., Rothé, M., Quilodran, R., Dominey, P. F., & Procyk, E. (2011) · The MIT Press eBooks · doi:10.7551/mitpress/9780262016438.003.0019 · Cited by 14
“A simple example of implementation of a meta‐learning algorithm was proposed by Schweighofer and Doya (2003) where an agent has to solve a non‐stationary Markov decision task also used in human fMRI experiments 71, 80 .”
MethodFast rule switching and slow rule updating in a perceptual categorization task
Bouchacourt, F., Tafazoli, S., Mattar, M. G., Buschman, T. J., & Daw, N. D. (2022) · eLife · doi:10.7554/elife.82531 · Cited by 14
Emergent Dynamics of Joy, Distress, Hope and Fear in Reinforcement Learning Agents
Jacobs, E., Broekens, J., & Jonker, C. M. (2014) · Cited by 14
Pseudo-Labeling and Meta Reweighting Learning for Image Aesthetic Quality Assessment
Jin, X., Lou, H., Huang, H., Li, X., Li, X., Cui, S., Zhang, X., & Li, X. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3207152 · Cited by 14
“Meta learning mainly includes memory [22], prediction of gradient [23], attention mechanisms [24], LSTM [25], reinforcement learning [26] and others.”
BackgroundReinforcement Learning for Engine Idle Speed Control
Xue, J., Gao, Q., & Ju, W. (2010) · International Conference on Measuring Technology and Mechatronics Automation (ICMTMA) · doi:10.1109/icmtma.2010.249 · Cited by 14
“Evaluation information is called as reinforcement signal from the environment or the state prediction network, which can be estimate of the state and success or failure action, and so on.”
BackgroundA Survey of Algorithmic Methods for Competency Self-Assessments in Human-Autonomy Teaming
Conlon, N., Ahmed, N. R., & Szafir, D. J. (2023) · ACM Computing Surveys · doi:10.1145/3616010 · Cited by 13
“Meta-RL agents can use this process to indirectly improve performance by better tuning parameters during the learning process [ 101 ] or directly improve performance by learning some underlying task structures or patterns [ 68 , 110 ].”
MethodHumans can adopt optimal discounting strategy under real-time constraints
Schweighofer, N., Shishida, K., Cheol, H. E., Okamoto, Y., Tanaka, S., Yamawaki, S., & Doya, K. (2005) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0020152.eor · Cited by 13
Self-citationMulti-objective Reinforcement Learning – Concept, Approaches and Applications
Zhang, L., Qi, Z., & Shi, Y. (2023) · Procedia Computer Science · doi:10.1016/j.procs.2023.08.018 · Cited by 12
Yang, Z., Ye, R., Jiang, L., Long, J., Huang, Y., & Li, C. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.112726 · Cited by 12
The aggregate complexity of decisions in the game of Go
Harré, M., Bossomaier, T. R. J., Gillett, A. C., & Snyder, A. W. (2011) · The European Physical Journal B · doi:10.1140/epjb/e2011-10905-8 · Cited by 12
Lin, C., Kan, C., Wang, J., Chen, W., & Chen, P.-Y. (2018) · IEEE Access · doi:10.1109/access.2018.2870689 · Cited by 12
“Weights, c1 and c2, are time-varying acceleration coefficients [34], [36], [37], described as follows:”
Continual Human-in-the-Loop Optimization
Liao, Y.-C., Streli, P., Li, Z., Christoph, G., & Holz, C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713603 · Cited by 15
Broekens, J. (2021) · ACM eBooks · doi:10.1145/3477322.3477333 · Cited by 12
Salimi-Badr, A., Ebadzadeh, M. M., & Darlot, C. (2017) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2017.11.004 · Cited by 11
A Meta-learning Method Based on Temporal Difference Error
Kobayashi, K., Mizoue, H., Kuremoto, T., & Obayashi, M. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-10677-4_60 · Cited by 11
“proposed the meta-learning method based on mid-term and longterm rewards [4].”
BackgroundMethodInfluentialTo each his own: The caregiver's role in a computational model of gaze following
Teuscher, C., & Triesch, J. (2007) · Neurocomputing · doi:10.1016/j.neucom.2006.02.023 · Cited by 11
“Schweigerhofer and Doya [32], for example , propose a meta-reinforcement learning algorithm to tune a, g, and t.”
BackgroundHoxha, I., Sperber, L., & Palminteri, S. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2422144122 · Cited by 12
Context meta-reinforcement learning via neuromodulation
Ben-Iwhiwhu, E., Dick, J., Ketz, N., Pilly, P. K., & Soltoggio, A. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.04.003 · Cited by 13
“This work builds on the existing meta learning frameworks (Bengio et al., 1992; Schmidhuber et al., 1996; Thrun and Pratt, 1998; Schweighofer and Doya, 2003) in the domain of reinforcement learning.”
31st Annual Computational Neuroscience Meeting: CNS*2022
Corsi, M., Sorrentino, P., Schwartz, D., George, N., Hugueville, L., Kahn, A. E., Dupont, S., Bassett, D. S., Jirsa, V., & Fallani, F. D. V. (2023) · Journal of Computational Neuroscience · doi:10.1007/s10827-022-00841-9 · Cited by 11
Meta-Reinforcement Learning with Transformer Networks for Space Guidance Applications
Federici, L., & Furfaro, R. (2024) · AIAA SciTech Forum · doi:10.2514/6.2024-2061 · Cited by 9
Autonomous and Adaptive Identification of Topics in Unstructured Text
Massey, L. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23863-5_1 · Cited by 9
A Framework for Robot Learning During Child-Robot Interaction with Human Engagement as Reward Signal
Khamassi, M., Chalvatzaki, G., Tsitsimis, T., Velentzas, G., & Tzafestas, C. S. (2018) · IEEE RO-MAN · doi:10.1109/roman.2018.8525598 · Cited by 9
“In order to perform active exploration, we apply a noiseless version of the meta-learning algorithm of [10], which tracks online variations of the agent’s performance measured by short-term r̄t and long-term r̄t reward running averages.”
MethodCognitive bots and algorithmic humans: toward a shared understanding of social intelligence
McDonald, K. R., & Pearson, J. M. (2019) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2019.04.013 · Cited by 10
Chen, Q., Niu, Y., Wan, B., & Peng, X. H. (2023) · Applied Sciences · doi:10.3390/app132312642 · Cited by 9
Xu, Z., & Yu, G. (2024) · Energies · doi:10.3390/en17081947 · Cited by 8
Genetically-regulated Neuromodulation Facilitates Multi-Task Reinforcement Learning
Cussat‐Blanc, S., & Harrington, K. (2015) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2739480.2754730 · Cited by 8
“As was noted in [25], neuromodulation is neither problem nor RL-algorithm specific, thus future work may investigate the application of genetically-regulated neuromodulation to alternative RL algorithms, such as Q-learning, as well as compare the performance and dynamics of GRNM to alter-150 native…”
BackgroundJoy, distress, hope, and fear in reinforcement learning
Jacobs, E., Broekens, J., & Jonker, C. M. (2014) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/2615731.2616089 · Cited by 8
Salimi-Badr, A., Ebadzadeh, M. M., & Darlot, C. (2017) · Journal of Computational Neuroscience · doi:10.1007/s10827-017-0665-5 · Cited by 8
“…models, BG are assumed to perform a temporal difference algorithm, similar to an actor-critic network (Suri and Schultz 1998, 1999; Suri et al. 2001; Contreras-Vidal and Schultz 1999; Schweighofer and Doya 2003; Sridharan et al. 2006; Joseph et al. 2010; Magdoom et al. 2011; Gao and Song 2014).”
BackgroundMethodAda-VSR: Adaptive Video Super-Resolution with Meta-Learning
Gupta, A., Jonnalagedda, P., Bhanu, B., & Roy–Chowdhury, A. K. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475320 · Cited by 8
“Recently, meta-learning algorithms have achieved impressive performance in various applications like few-shot learning [25, 34, 50, 56, 59], reinforcement learning [14, 20, 44, 52] and image super-resolution [33, 57].”
Velentzas, G., Tsitsimis, T., Rañó, I., Tzafestas, C. S., & Khamassi, M. (2018) · Paladyn Journal of Behavioral Robotics · doi:10.1515/pjbr-2018-0016 · Cited by 8
Silvetti, M., Baldassarre, G., & Caligiore, D. (2019) · Springer series in cognitive and neural systems · doi:10.1007/978-3-030-18830-6_12 · Cited by 8
Adaptive first arrival picking model with meta-learning
Yuan, P., Hu, W., Wu, X., Chen, J., & Nguyen, H. V. (2020) · Second International Meeting for Applied Geoscience & Energy · doi:10.1190/segam2020-3420195.1 · Cited by 9
Federici, L., & Furfaro, R. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10520-8 · Cited by 7
“In such cases, meta-reinforcement learning (meta-RL) [35] can significantly enhance the ability of the DNN-based controller to reconstruct the optimal control policy.”
Modulating reinforcement-learning parameters using agent emotions
von Haugwitz, R., Kitamura, Y., & Takashima, K. (2012) · Joint International Conference on Soft Computing and Intelligent Systems and International Symposium on Advanced Intelligent Systems (SCIS&ISIS) · doi:10.1109/scis-isis.2012.6505340 · Cited by 7
“A more general approach for control of any learning parameter is presented in [12], who expand on the Stochastic Real Value Units (SRV) reinforcement learning algorithm by [13] to achieve meta-learning with stochastic gradients.”
MethodOut-of-Distribution Detection with Confidence Deep Reinforcement Learning
Wang, D. (2023) · International Conference on Communications, Computing, Cybersecurity, and Informatics (CCCI) · doi:10.1109/ccci58712.2023.10290768 · Cited by 7
“Moreover, Schweighofer et al. [26] prove that reinforcement learning can robustly find better parameters in static and dynamic environments.”
BackgroundSoft-Root-Sign: A New Bounded Neural Activation Function
Li, D., & Zhou, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60636-7_26 · Cited by 9
Improving the efficiency of reinforcement learning for a spacecraft powered descent with Q-learning
Wilson, C., & Riccardi, A. (2021) · Optimization and Engineering · doi:10.1007/s11081-021-09687-z · Cited by 8
“These automated methods use a variety of intelligent approaches such as evolutionary computation (Schweighofer and Doya 2003; Young et al. 2015) and Bayesian optimisation methods (Barsce et al. 2017; Bergstra et al. 2011).”
MethodCell selection for open-access femtocell networks: Learning in changing environment
Dhahri, C., & Ohtsuki, T. O. (2014) · Physical Communication · doi:10.1016/j.phycom.2014.04.008 · Cited by 7
“In [18], the authors propose an algorithm that consists of adding a second learning layer to tune the reinforcement learning parameters to adapt non-stationary environments.”
BackgroundMeta-Learning of Exploration and Exploitation Parameters with Replacing Eligibility Traces
Tokic, M., Schwenker, F., & Palm, G. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40705-5_7 · Cited by 7
“In contrast, neurobiologically inspired models utilize the immediate reward [5] or the temporal-difference error [6] to adapt the amount of exploration, e.g. by the meta-parameter τ of Softmax action selection.”
Evolving inborn knowledge for fast adaptation in dynamic POMDP problems
Ben-Iwhiwhu, E., Ładosz, P., Dick, J., Chen, W., Pilly, P. K., & Soltoggio, A. (2020) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3377930.3390214 · Cited by 7
“dlpbc/penn-a. 2 RELATED WORK In reinforcement learning (RL) literature, meta-RL methods seek to develop agents that adapt to changing tasks in an environment or a set of related environments. Meta-RL [23, 24] is based on the general idea of meta-learning [2, 10, 30] applied to the RL domain. Recently, deep meta-RL has…”
BackgroundMurakoshi, K., & Mizuno, J. (2004) · Biosystems · doi:10.1016/j.biosystems.2004.05.001 · Cited by 7
“Schweighofer and Doya (2003) proposed a learning algorithm based on a conceptual theory (Doya, 2002) in which parameters in reinforcement learning are adjusted by neuromodulators.”
InfluentialHuang, Q., Luo, Y., Liu, Z., Xia, J., Chang, M., & Li, J. (2025) · Journal of Marine Science and Engineering · doi:10.3390/jmse13071271 · Cited by 6
A Meta-Learning-Based Train Dynamic Modeling Method for Accurately Predicting Speed and Position
Cao, Y., Wang, X., Zhu, L., Wang, H., & Wang, X. (2023) · Sustainability · doi:10.3390/su15118731 · Cited by 6
A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions
Liao, Y.-C., Desai, R., Pierce, A. M., Taylor, K. E., Benko, H., Jonker, T. R., & Gupta, A. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642071 · Cited by 6
Personalized Web-page Rendering System
Raj, S. R. P., & Ravindran, B. (2008) · Conference on Management of Data · Cited by 6
Neuronal Circuit Computation of Choice
Wang, X. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00023-1 · Cited by 6
DUSC-DQN:An Improved Deep Q-Network for Intelligent Penetration Testing Path Design
Wang, P., Liu, J., Zhong, X., Yang, G., Zhou, S., & Zhang, Y. (2022) · International Conference on Computer and Communication Systems (ICCCS) · doi:10.1109/icccs55155.2022.9846482 · Cited by 7
Meta-OLE: Meta-learned Orthogonal Low-Rank Embedding
Wang, Z., Lu, Y., & Qiu, Q. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00527 · Cited by 6
“Finally, in addition to its successful application in few-shot learning, the idea of metalearning has also been proved to be effective at diverse tasks such as memory [3] and reinforcement learning [36, 14, 34].”
BackgroundMeta Inverse Reinforcement Learning via Maximum Reward Sharing for Human Motion AnalysisPreprint
Li, K., & Burdick, J. W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.03592 · Cited by 6
“The method in [26] tunes meta parameters for reinforcement learning, learning rate for TD learning, action selection trade-off, and discount factor.”
MethodSharma, A. K., Tokekar, S., & Varma, S. (2022) · Automated Software Engineering · doi:10.1007/s10515-022-00362-w · Cited by 5
“Also, meta-RL assumed that meta-training time should match meta-test time to achieve the best training performance, which is difficult in real-time scenarios (Schweighofera and Doya 2003).”
BackgroundSharma, A. K., Tokekar, S., & Varma, S. (2023) · IEEE International Conference on Technology, Engineering, Management for Societal Impact using Marketing, Entrepreneurship and Talent (TEMSMET) · doi:10.1109/temsmet56707.2023.10150107 · Cited by 5
“Usually, prior expertise is required to learn new methods of working that specifically optimize rapid adaption [8].”
BackgroundArtificial Emotion Model Based on Neuromodulators and Q-learning
Shi, X., Wang, Z., & Zhang, Q. (2012) · Lecture notes in electrical engineering · doi:10.1007/978-3-642-31006-5_35 · Cited by 5
Dou, W., Liu, X., Chen, G., Cheung, S.-C., & Cai, S. (2007) · International Journal of Knowledge and Learning · doi:10.1504/ijkl.2007.012602 · Cited by 5
“Similarly, the local learning process for promoting the metacognition would be treated as a meta-learning process and the global learning process for promoting the group cognition would be treated as a group learning process (Schweighofer and Doya, 2003).”
BackgroundAlgorithm selection of off-policy reinforcement learning algorithm.Preprint
Laroche, R., & Féraud, R. (2017) · arXiv (Cornell University) · Cited by 5
Meta Learning in the Continuous Time LimitPreprint
Xu, R., Chen, L., & Karbasi, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10921 · Cited by 5
“This idea has been successfully applied to different learning scenarios including reinforcement learning [39, 4, 32], deep probabilistic models [6, 16, 10], language models [25], imitation learning [5], and unsupervised learning [14], to name a few.”
BackgroundMethodHao, L., Cheng, Q., Liu, H., & Zhang, Y. (2024) · Applied Sciences · doi:10.3390/app142310821 · Cited by 4
One-Shot Chinese Character Recognition Based on Deep Siamese Networks
Li, H., Ren, X., & Lv, Y. (2019) · Lecture notes in electrical engineering · doi:10.1007/978-981-32-9682-4_78 · Cited by 4
García‐Peñalvo, F. J., Vázquez‐Ingelmo, A., & Garc韆-Holgado, A. (2023) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2023.023897 · Cited by 4
Coupling Effect of Exploration Rate and Learning Rate for Optimized Scaled Reinforcement Learning
Gupta, S., Pal, S., Kumar, K., & Ghosh, K. (2023) · SN Computer Science · doi:10.1007/s42979-023-02114-3 · Cited by 4
“In addition, the complexity of learning a value function is directly proportional to 1 1− [23].”
BackgroundDeep reinforcement learning in medical imaging
Zhou, S. K., & Wang, Q. (2024) · Elsevier eBooks · doi:10.1016/b978-0-32-385124-4.00010-6 · Cited by 4
Meta-Learning based efficient framework for diagnosing rare disorders: A comprehensive survey
Singh, K., & Malhotra, D. (2024) · AIP conference proceedings · doi:10.1063/5.0199881 · Cited by 4
Robust Reinforcement Learning for Autonomous Driving in Uncertain Environments
Wang, D.-J., & Ghoreishi, S. F. (2025) · 2025 IEEE 21st International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case58245.2025.11164129 · Cited by 6
Joy, Distress, Hope, and Fear in Reinforcement Learning (Extended Abstract)
Jacobs, E., Broekens, J., & Jonker, C. M. (2014) · Research Repository (Delft University of Technology) · Cited by 4
Ottoni, A. L. C., Nepomuceno, E. G., de Oliveira, M. S., Ottoni, L. T. C., & Lamperti, R. D. (2016) · Revista Brasileira de Computação Aplicada · doi:10.5335/rbca.v8i2.5249 · Cited by 4
Bio-inspired meta-learning for active exploration during non-stationary multi-armed bandit tasks
Velentzas, G., Tzafestas, C. S., & Khamassi, M. (2017) · Intelligent Systems Conference (IntelliSys) · doi:10.1109/intellisys.2017.8324365 · Cited by 4
“We evaluated the performance of some of the best stateof-the-art algorithms, namely: KF-MANB [8] , D-UCB [7], SW-UCB [6] and Adapt-Eve [20], with two new algorithms proposed in this paper: a modification of meta-learning for reinforcement learning proposed in [19] termed MLB (metalearning for bandits) and a new hybrid…”
MethodInfluentialAdaptive Tuning Curve Widths Improve Sample Efficient Learning
Meier, F., Dang-Nhu, R., & Steger, A. (2020) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2020.00012 · Cited by 5
“” In the meta learning setting, an outer learning system adjusts parameters or learning mechanisms of an inner learning system in order to improve the performance and efficiency of the later (Schmidhuber et al., 1996; Baxter, 1998; Thrun and Pratt, 1998; Hochreiter et al., 2001; Schweighofer and Doya, 2003).”
BackgroundTowards on-line tuning of adaptive-agent’s multivariate meta-parameter
Kárný, M. (2021) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-021-01358-w · Cited by 4
Risk-aware Meta-level Decision Making for Exploration Under Uncertainty
Ott, J., Kim, S. K., Bouman, A., Peltzer, O., Sobue, M., Delecki, H., Kochenderfer, M. J., Burdick, J. W., & Agha–mohammadi, A. (2024) · International Conference on Control, Decision and Information Technologies (CoDIT) · doi:10.1109/codit62066.2024.10708134 · Cited by 4
“Meta-learning, on the other hand, focuses on learning to learn [25, 26, 27, 28].”
BackgroundNeural Circuit Mechanisms of Value-Based Decision-Making and Reinforcement Learning
Soltani, A., Chaisangmongkon, W., & Wang, X.-J. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-805308-9.00013-0 · Cited by 5
Schneider, M. (2018) · Cognitive Systems Research · doi:10.1016/j.cogsys.2018.07.005 · Cited by 4
Opportunistic Spectrum Access: Online Search of Optimality
Alaya-Feki, A. B. H., Sayrac, B., Moulines, É., & Cornec, A. L. (2008) · Globecom · doi:10.1109/glocom.2008.ecp.594 · Cited by 4
“Meta learning (ML) corresponds to the utilization of a second layer of learning to enhance the performance of the base learning with dynamic tuning of the base learning parameters [ 8 ].”
MethodComputational Intelligence for Meta-Learning: A Promising Avenue of Research
Castiello, C., & Fanelli, A. M. (2011) · Studies in computational intelligence · doi:10.1007/978-3-642-20980-2_5 · Cited by 4
“Various meanings can be found in literature coupled with the metalearning catchword [14, 15, 16, 17].”
BackgroundDopamine regulates the exploration-exploitation trade-off in ratsPreprint
Cinotti, F., Fresno, V., Aklil, N., Coutureau, É., Girard, B., Marchand, A. R., & Khamassi, M. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/482802 · Cited by 4
“Dopaminergic regulation of exploration appears to occur at a longer time scale than that of a few trials, which would constitute a form of meta-learning[3], [32] adapting behavior to the general characteristics of the task rather than to immediate events.”
BackgroundImitation as a model-free process in human reinforcement learningPreprint
Najar, A., Bonnet, E., Bahrami, B., & Palminteri, S. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/797407 · Cited by 4
“At the computational level, this effect can be formalized as a meta-learning process, where relevant variables are used to optimally tune the learning parameters (27, 28).”
How Does an Approximate Model Help in Reinforcement Learning?Preprint
Feng, F., Yin, W., & Yang, L. F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02986 · Cited by 16
Analysis of Model-Agnostic Meta-Reinforcement Learning on Automated HVAC Control
Filiz, U., Hekimoğlu, M. B., Baghaee, S., & Ulusoy, İ. (2025) · Signal Processing and Communications Applications Conference (SIU) · doi:10.1109/siu66497.2025.11112293 · Cited by 3
“Meta-learning, or "learning to learn," enhances the adapt-ability of models by enabling them to generalize across different tasks with minimal data [15].”
Direct and indirect striatal projecting neurons exert strategy-dependent effects on decision-making
Rodriguez, E. C., Naudé, J., Rial, D., & d’Exaerde, A. D. K. (2025) · Science Advances · doi:10.1126/sciadv.adq0484 · Cited by 3
CausalCOMRL: Context-based offline meta-reinforcement learning with causal representation
Zhang, Z., Meng, W., Sun, H., & Pan, G. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107955 · Cited by 5
A neuro-computational model of sequence learning in macaques: the Simultaneous Chaining Paradigm
Gasser, B., & Arbib, M. A. (2017) · Adaptive Behavior · doi:10.1177/1059712317715324 · Cited by 3
Yuan, P., & Nguyen, H. V. (2022) · Elsevier eBooks · doi:10.1016/b978-0-32-399851-2.00009-0 · Cited by 3
Regulation of reinforcement learning parameters captures long‐term changes in rat behaviour
Cinotti, F., Coutureau, É., Khamassi, M., Marchand, A. R., & Girard, B. (2024) · European Journal of Neuroscience · doi:10.1111/ejn.16449 · Cited by 4
“…how animals adapted their behaviour while progressively learning the task, and whether such an adaptation resulted from a form of meta-learning (Schweighofer & Doya, 2003) in which either the exploration-exploitation trade-off or the learning rate was being regulated by the current performance…”
Meta Reinforcement Learning for Heuristic Planing
Gutiérrez, R. L., & Leonetti, M. (2021) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v31i1.16003 · Cited by 3
Computational Account of Emotion, an Oxymoron?
Dodig-Crnković, G., von Haugwitz, R., & Almér, A. (2015) · ISIS Summit: The Information Society at the Crossroads · doi:10.3390/isis-summit-vienna-2015-t9.2004 · Cited by 3
Tonic dopamine effects in a bio-inspired robot controller enhance expected lifetime
Nettle, C. J., Escobar, M.-J., & Leblois, A. (2016) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2016.7846824 · Cited by 3
“[42] and be used as an alternative to regulate the β parameter.”
Bayesian Network based Optimal Load Balancing in Software Defined Networks
Shaikh, M. R. R. (2023) · International Conference on Emerging Smart Computing and Informatics (ESCI) · doi:10.1109/esci56872.2023.10099730 · Cited by 3
“DRL has received a lot of interest as a solution for the load balancing problem in the distributed controller system [3-5].”
BackgroundAnálise do desempenho do aprendizado por reforço na solução do problema da mochila multidimensional
Ottoni, A. L. C., Nepomuceno, E. G., & de Oliveira, M. S. (2017) · Revista Brasileira de Computação Aplicada · doi:10.5335/rbca.v9i3.6601 · Cited by 3
Message Action Adapter Framework in Multi-Agent Reinforcement Learning
Park, B., & Choi, J. (2024) · Applied Sciences · doi:10.3390/app142110079 · Cited by 2
Robust priors for regularized regression
Bobadilla-Suarez, S., Jones, M., & Love, B. C. (2021) · Cognitive Psychology · doi:10.1016/j.cogpsych.2021.101444 · Cited by 2
Sketch face recognition based on domain adaptation scaled entropy meta-network
Cao, L., Chen, C., Guo, Y., & Du, K. (2022) · Measurement and Control · doi:10.1177/00202940221097979 · Cited by 2
Improving the Robustness of Instance-Based Reinforcement Learning Robots by Metalearning
Yasuda, T., Araki, K., & Ohkura, K. (2011) · Journal of Advanced Computational Intelligence and Intelligent Informatics · doi:10.20965/jaciii.2011.p1065 · Cited by 2
Brain Circuitry for Social Decision Making in Non-Human Primates
Lee, D., & Dorris, M. C. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00026-7 · Cited by 2
Ottoni, A. L. C., Nepomuceno, E. G., & de Oliveira, M. S. (2019) · Anais do 14º Simpósio Brasileiro de Automação Inteligente · doi:10.17648/sbai-2019-111113 · Cited by 2
Bridging Computational Neuroscience and Machine Learning on Non-Stationary Multi-Armed BanditsPreprint
Velentzas, G., Tzafestas, C. S., & Khamassi, M. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/117598 · Cited by 2
“Furthermore, following [8], a similar update rule applies to estimate mid-term and long-term reward running averages according to:”
BackgroundMethodUnified dynamic compensation method for heterogeneous sensors under few-shot conditions
Xu, B., Xu, D., Liu, L., Zhang, X., Tang, H., Wang, X., & Guo, R. (2026) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2026.114391 · Cited by 1
Methamphetamine-induced adaptation of learning rate dynamics depend on baseline performance
Kirschner, H., Molla, H. M., Nassar, M. R., de Wit, H., & Ullsperger, M. (2025) · eLife · doi:10.7554/elife.101413.3 · Cited by 1
Metadata
- Authors
- Schweighofer, N., & Doya, K.
- Affiliations
- Advanced Telecommunications Research Institute International (JP)
- Corresponding author
- Schweighofer, Nicolas
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2003-01
- Volume
- 16
- Issue
- 1
- Pages
- 5-9
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W1996579288
- Cited by
- 261
- References
- 17
- Citations by year
- 2026: 7 · 2025: 22 · 2024: 20 · 2023: 22 · 2022: 20 · 2021: 26 · 2020: 17 · 2019: 13 · 2018: 16 · 2017: 16 · 2016: 8 · 2015: 7 · 2014: 11 · 2013: 7 · 2012: 6
- FWCI
- 2.84
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Networks (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Core Research for Evolutional Science and Technology
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Keywords
- meta-learning, meta-reinforcement learning, meta-parameters, dopaminergic neurons, parameter adaptation, Markov decision process, dynamic environments
- MeSH terms
- Algorithms, Artificial Intelligence, Decision Theory, Markov Chains, Reinforcement, Psychology, Stochastic Processes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
