Learning to learn by gradient descent by gradient descent
Open accessAbstract
The move from hand-designed features to learned features in machine learning has been wildly successful. In spite of this, optimization algorithms are still designed by hand. In this paper we show how the design of an optimization algorithm can be cast as a learning problem, allowing the algorithm to learn to exploit structure in the problems of interest in an automatic way. Our learned algorithms, implemented by LSTMs, outperform generic, hand-designed competitors on the tasks for which they are trained, and also generalize well to new tasks with similar structure. We demonstrate this on a number of tasks, including simple convex problems, training neural networks, and styling images with neural art.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
L2O black-box optimizers, typically trained on a limited number of small tasks, struggle to generalize to state-of-the-art settings where much larger models are trained with significantly more training steps.
“However, those black-box optimizers, typically trained on a limited number of small tasks, struggle to generalize to state-of-the-art settings where much larger models are trained with significantly more training steps.”
From Symbolic Discovery of Optimization Algorithms (2023) · cites this paper as Andrychowicz et al., 2016
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
No free lunch theorems for optimization
Wolpert, D. H., & Macready, W. G. (1997) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.585893 · Cited by 14,237 · see citing works
Learning to Control Fast-Weight Memories: An Alternative to Dynamic Recurrent Networks
Schmidhuber, J. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.131 · Cited by 357 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Memory Aware Synapses: Learning What (not) to Forget
Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., & Tuytelaars, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_9 · Cited by 1,590 · see citing works
Cheng, Y., Wang, D., Zhou, P., & Zhang, T. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2765695 · Cited by 817 · see citing works
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762 · see citing works
Reinforcement learning, fast and slow
Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.006 · Cited by 667 · see citing works
Learning to reinforcement learn
Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., & Botvinick, M. (2016) · arXiv · doi:10.48550/arxiv.1611.05763 · Cited by 579 · see citing works
Task Agnostic Meta-Learning for Few-Shot Learning
Jamal, M. A., & Qi, G.-J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01199 · Cited by 519 · see citing works
Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning
Nagabandi, A., Clavera, I., Liu, S., Fearing, R. S., Abbeel, P., Levine, S., & Finn, C. (2018) · arXiv · doi:10.48550/arxiv.1803.11347 · Cited by 370 · see citing works
Federated Learning and Differential Privacy for Medical Image Analysis
Adnan, M., Kalra, S., Cresswell, J. C., Taylor, G. W., & Tizhoosh, H. (2021) · doi:10.21203/rs.3.rs-1005694/v1 · Cited by 464 · see citing works
A high-level overview of AI ethics
Kazim, E., & Koshiyama, A. S. (2021) · Patterns · doi:10.1016/j.patter.2021.100314 · Cited by 269 · see citing works
Deep learning for small and big data in psychiatry
Koppe, G., Meyer-Lindenberg, A., & Durstewitz, D. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-020-0767-z · Cited by 189 · see citing works
Meta-learning in natural and artificial intelligence
Wang, J. X. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.01.002 · Cited by 171 · see citing works
Symbolic Discovery of Optimization Algorithms
Chen, X., Liang, C., Huang, D., Real, E., Wang, K., Liu, Y., Pham, H., Dong, X., Luong, T., Hsieh, C.-J., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.06675 · Cited by 303 · see citing works
“Our work lies in the area of AutoML and meta-learning that includes learning to learn [1, 5, 63, 64, 78, 96, 97], neural architecture search [14, 15, 57, 71, 79, 87, 93, 94, 98, 106] and hyperparameter optimization [25, 43, 44, 55], etc.”
MethodA review on the self and dual interactions between machine learning and optimisation
Song, H., Triguero, I., & Özcan, E. (2019) · Progress in Artificial Intelligence · doi:10.1007/s13748-019-00185-z · Cited by 101 · see citing works
Hao, Y., & Gao, Q. (2020) · Applied Sciences · doi:10.3390/app10113961 · Cited by 95 · see citing works
Learning to Optimize Variational Quantum Circuits to Solve Combinatorial Problems
Khairy, S., Shaydulin, R., Cincio, L., Alexeev, Y., & Balaprakash, P. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i03.5616 · Cited by 117 · see citing works
Clune, J. (2019) · arXiv · doi:10.48550/arxiv.1905.10985 · Cited by 82 · see citing works
Martínez-Pascual, D., Catalán, J. M., Blanco-Ivorra, A., Sanchís, M., Arán-Ais, F., & García-Aracil, N. (2023) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2023.1199459 · Cited by 21 · see citing works
Human-Timescale Adaptation in an Open-Ended Task Space
Adaptive Agent Team, Bauer, J., Baumli, K., Baveja, S., Behbahani, F., Bhoopchand, A., Bradley-Schmieg, N., Chang, M., Clay, N., Collister, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.07608 · Cited by 19 · see citing works
“Other approaches for meta-RL include meta-gradients (Andrychowicz et al., 2016; Finn et al., 2017; Flennerhag et al., 2022; Xu et al., 2018), which can be efficient but often suffer from instability and myopia (Flennerhag et al.”
MethodMeta Learning Backpropagation And Improving It
Kirsch, L., & Schmidhuber, J. (2020) · arXiv · doi:10.48550/arxiv.2012.14905 · Cited by 19 · see citing works
Titans: Learning to Memorize at Test Time
Behrouz, A., Zhong, P., & Mirrokni, V. (2025) · arXiv · doi:10.48550/arxiv.2501.00663 · Cited by 24 · see citing works
“The key ideas of learning at test time or learning to learn (i.e., [115]) backs to very early studies on local learning [116], in which each test data sample is trained on its neighbors before making a prediction [117, 22].”
Chang, O., & Lipson, H. (2018) · The 2018 Conference on Artificial Life · doi:10.1162/isal_a_00049 · Cited by 16 · see citing works
“, 2017), which have been shown to be useful in meta-learning an optimizer (Ravi and Larochelle, 2016; Andrychowicz et al., 2016).”
BackgroundLearning to (Learn at Test Time): RNNs with Expressive Hidden States
Sun, Y., Li, X., Dalal, K., Xu, J., Vikram, A., Zhang, G., Dubois, Y., Chen, X., Wang, X., Koyejo, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.04620 · Cited by 18 · see citing works
“In prior work such as [2], [21] and [55], the inner loop learns from an entire dataset at a time instead of a sequence, so the outer loop needs a collection of datasets or tasks.”
Hidden Incentives for Auto-Induced Distributional Shift
Krueger, D., Maharaj, T., & Leike, J. (2020) · arXiv · doi:10.48550/arxiv.2009.09153 · Cited by 14 · see citing works
“Many recent works focus on multi-task meta-learning, where the OL seeks to find learning rules that generalize to unseen tasks by training the IL on a distribution of tasks (Finn et al., 2017; Ren et al., 2018; Andrychowicz et al., 2016).”
BackgroundA memristive computational neural network model for time-series processing
Pistolesi, V., Ceni, A., Milano, G., Ricciardi, C., & Gallicchio, C. (2025) · APL Machine Learning · doi:10.1063/5.0255168 · Cited by 9 · see citing works
Meta-Harness: End-to-End Optimization of Model Harnesses
Lee, Y., Nair, R., Zhang, Q., Lee, K., Khattab, O., & Finn, C. (2026) · arXiv · doi:10.48550/arxiv.2603.28052 · Cited by 3 · see citing works
“At a high level, Meta-Harness brings ideas from the broader literature on credit assignment and meta-learning [40; 46; 3; 17; 44; 2] in a new regime enabled by recent advances in coding agents.”
Learning Not to Learn: Nature versus Nurture in Silico
Lange, R. T., & Sprekeler, H. (2020) · arXiv · doi:10.48550/arxiv.2010.04466 · Cited by 6 · see citing works
EvoX: Meta-Evolution for Automated Discovery
Liu, S., Agarwal, S., Maheswaran, M., Cemri, M., Li, Z., Mang, Q., Naren, A., Boneh, E., Cheng, A., Pan, M. Z., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.23413 · Cited by 1 · see citing works
“Prior work explores learned optimizers, symbolic discovery of update rules [6], gradient-based meta-learning [2], and reinforcement learning-based optimizers [26].”
End-to-End Test-Time Training for Long Context
Tandon, A., Dalal, K., Li, X., Koceja, D., Rød, M., Buchanan, S., Wang, X., Leskovec, J., Koyejo, S., Hashimoto, T., et al. (2025) · arXiv
- Authors
- Andrychowicz, M., Denil, M., Gomez, S., Hoffman, M. W., Pfau, D., Schaul, T., Shillingford, B., & de Freitas, N.
- Affiliations
- University of Warsaw (PL)
- Google (United States) (US)
- University of Oxford (GB)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1606.04474
- OpenAlex
- W2427497464
- Cited by
- 1,486
- References
- 3
- Citations by year
- 2025: 81 · 2024: 140 · 2023: 166 · 2022: 162 · 2021: 265 · 2020: 266 · 2019: 213 · 2018: 138 · 2017: 46 · 2016: 8 · 2015: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- gradient descent, automatic algorithm design, learned features, image stylization, neural network training, convex optimization, task generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
