CoChat
Sign Up
PreprintCited by 7Domain Adaptation and Few-Shot LearningMachine Learning and AlgorithmsReinforcement Learning in Robotics

Meta-learners' learning dynamics are unlike learners'

Open access
Rabinowitz, Neil C.·2019·arXiv
DOI 10.48550/arxiv.1905.01320arXiv 1905.01320
Open this paper in CoChat — free accountOpen PDF

Abstract

Meta-learning is a tool that allows us to build sample-efficient learning systems. Here we show that, once meta-trained, LSTM Meta-Learners aren't just faster learners than their sample-inefficient deep learning (DL) and reinforcement learning (RL) brethren, but that they actually pursue fundamentally different learning trajectories. We study their learning dynamics on three sets of structured tasks for which the corresponding learning dynamics of DL and RL systems have been previously described: linear regression (Saxe et al., 2013), nonlinear regression (Rahaman et al., 2018; Xu et al., 2018), and contextual bandits (Schaul et al., 2019). In each case, while sample-inefficient DL and RL Learners uncover the task structure in a staggered manner, meta-trained LSTM Meta-Learners uncover almost all task structure concurrently, congruent with the patterns expected from Bayes-optimal inference algorithms. This has implications for research areas wherever the learning behaviour itself is of interest, such as safety, curriculum design, and human-in-the-loop machine learning.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1905.01320

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Meta-learning can be used to construct Bayes-optimal learning algorithms.

    “We accomplish this by pointing out that meta-learning can be used to construct Bayes-optimal learning algorithms, allowing us to draw strong connections to the rational analysis of cognition.”

    From Meta-learned models of cognition (2024) · cites this paper as Rabinowitz, 2019

    Section 3.2.6 (p. 7) says meta-learning’s inner loop should come to approximate amortised Bayes-optimal inference, and reports similar qualitative learning dynamics on in-distribution regression problems. That supports a connection between meta-learning and Bayes-optimal inference, but not the unqualified claim that it constructs Bayes-optimal learning algorithms. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Meta-learning procedures similar to the one discussed often lead to networks that closely approximate Bayesian inference.

    “in practice, it has been observed that meta-learning procedures similar to the one discussed here often lead to networks that closely approximate Bayesian inference (Mikulik, Delétang, et al., 2020; Rabinowitz, 2019).”

    From Meta-learned models of cognition (2024) · cites this paper as Rabinowitz, 2019

    Section 3.2.6 (p. 7) says meta-learning's inner loop should come to approximate amortised Bayes-optimal inference, and reports similar qualitative learning dynamics for an in-distribution regression task. Section 4.2.5 (p. 11) reports a similar qualitative match on another task. These passages support a connection, but do not establish that such procedures often produce networks that closely approximate Bayesian inference. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 110

  1. Pieter Abbeel and Andrew Y Ng. Apprenticeship learning via inverse reinforcement learning. In Proceedings of the twenty-first international conference on Machine learning , page 1. ACM, 2004.
  2. Alessandro Achille and Stefano Soatto. Emergence of invariance and disentanglement in deep representations. Journal of Machine Learning Research , 19(50):1-34, 2018.
  3. Alessandro Achille, Matteo Rovere, and Stefano Soatto. Critical learning periods in deep networks. In International Conference on Learning Representations , 2019. URL https://openreview.net/forum? id=BkeStsCcKQ .
  4. Madhu S Advani and Andrew M Saxe. High-dimensional dynamics of generalization error in neural networks. arXiv preprint arXiv:1710.03667 , 2017. · arXiv
  5. Ron Amit and Ron Meir. Meta-learning by adjusting priors based on extended pac-bayes theory. In International Conference on Machine Learning , pages 205-214, 2018.
  6. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Man´ e. Concrete problems in ai safety. arXiv preprint arXiv:1606.06565 , 2016. · arXiv
  7. Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas. Learning to learn by gradient descent by gradient descent. In Advances in Neural Information Processing Systems , pages 3981-3989, 2016.
  8. Devansh Arpit, Stanisław Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. arXiv preprint arXiv:1706.05394 , 2017. · arXiv
  9. Marco Baity-Jesi, Levent Sagun, Mario Geiger, Stefano Spigler, G Ben Arous, Chiara Cammarota, Yann LeCun, Matthieu Wyart, and Giulio Biroli. Comparing dynamics: Deep neural networks versus glassy systems. arXiv preprint arXiv:1803.06969 , 2018. · arXiv
  10. Horace B Barlow. Unsupervised learning. Neural computation , 1(3):295-311, 1989.
  11. Jonathan Baxter. Theoretical models of learning to learn. In Learning to learn , pages 71-94. Springer, 1998.
  12. Jonathan Baxter. A model of inductive bias learning. Journal of Artificial Intelligence Research , 12:149-198, 2000.
  13. Yoshua Bengio, J´ erˆ ome Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning , pages 41-48. ACM, 2009.
  14. Alberto Bernacchia, Mate Lengyel, and Guillaume Hennequin. Exact natural gradient in deep linear networks and its application to the nonlinear case. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. CesaBianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31 , pages 5945-5954. Curran Associates, Inc., 2018.
  15. L´ eon Bottou. Large-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT'2010 , pages 177-186. Springer, 2010.
  16. Pratik Chaudhari and Stefano Soatto. Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks. In 2018 Information Theory and Applications Workshop (ITA) , pages 1-10. IEEE, 2018.
  17. Yutian Chen, Matthew W Hoffman, Sergio G´ omez Colmenarejo, Misha Denil, Timothy P Lillicrap, Matt Botvinick, and Nando de Freitas. Learning to learn without gradient descent by gradient descent. In Proceedings of the 34th International Conference on Machine Learning-Volume 70 , pages 748-756. JMLR. org, 2017.
  18. Anna Choromanska, Mikael Henaff, Michael Mathieu, G´ erard Ben Arous, and Yann LeCun. The loss surfaces of multilayer networks. In Artificial Intelligence and Statistics , pages 192-204, 2015.
  19. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems , pages 4299-4307, 2017.
  20. David A Cohn, Zoubin Ghahramani, and Michael I Jordan. Active learning with statistical models. Journal of artificial intelligence research , 4:129-145, 1996.
  21. Francis Crick. The recent excitement about neural networks. Nature , 337(6203):129-132, 1989.
  22. Yann N Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, and Yoshua Bengio. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. In Advances in neural information processing systems , pages 29332941, 2014.
  23. J Michael Davis. Imitation: A review and critique. In Perspectives in ethology , pages 43-72. Springer, 1973.
  24. Vikas Dhiman, Shurjo Banerjee, Brent Griffin, Jeffrey M Siskind, and Jason J Corso. A critical investigation of deep reinforcement learning for navigation. arXiv preprint arXiv:1802.02274 , 2018. · arXiv
  25. Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, and Pieter Abbeel. Rl$ˆ2$: Fast reinforcement learning via slow reinforcement learning. CoRR , abs/1611.02779, 2016. URL http://arxiv.org/ abs/1611.02779 . · arXiv
  26. Jeffrey L Elman. Learning and development in neural networks: The importance of starting small. Cognition , 48 (1):71-99, 1993.
  27. SM Ali Eslami, Danilo Jimenez Rezende, Frederic Besse, Fabio Viola, Ari S Morcos, Marta Garnelo, Avraham Ruderman, Andrei A Rusu, Ivo Danihelka, Karol Gregor, et al. Neural scene representation and rendering. Science , 360(6394):1204-1210, 2018.
  28. Chrisantha Thomas Fernando, Jakub Sygnowski, Simon Osindero, Jane Wang, Tom Schaul, Denis Teplyashin, Pablo Sprechmann, Alexander Pritzel, and Andrei A Rusu. Meta learning by the baldwin effect. arXiv preprint arXiv:1806.07917 , 2018. · arXiv
  29. Chelsea Finn, Pieter Abbeel, and Sergey Levine. Modelagnostic meta-learning for fast adaptation of deep networks. arXiv preprint arXiv:1703.03400 , 2017. · arXiv
  30. Jaime F Fisac, Monica A Gates, Jessica B Hamrick, Chang Liu, Dylan Hadfield-Menell, Malayandi Palaniappan, Dhruv Malik, S Shankar Sastry, Thomas L Griffiths, and Anca D Dragan. Pragmatic-pedagogic value alignment. arXiv preprint arXiv:1707.06354 , 2017. · arXiv
  31. Meire Fortunato, Mohammad Gheshlaghi Azar, Bilal Piot, Jacob Menick, Ian Osband, Alex Graves, Vlad Mnih, Remi Munos, Demis Hassabis, Olivier Pietquin, et al. Noisy networks for exploration. arXiv preprint arXiv:1706.10295 , 2017. · arXiv
  32. Kunihiko Fukushima and Sei Miyake. Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition. In Competition and cooperation in neural nets , pages 267-285. Springer, 1982.
  33. Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J Rezende, SM Eslami, and Yee Whye Teh. Neural processes. arXiv preprint arXiv:1807.01622 , 2018. · arXiv
  34. Christophe Giraud-Carrier, Ricardo Vilalta, and Pavel Brazdil. Introduction to the special issue on metalearning. Machine learning , 54(3):187-193, 2004.
  35. Ian J Goodfellow, Oriol Vinyals, and Andrew M Saxe. Qualitatively characterizing neural network optimization problems. arXiv preprint arXiv:1412.6544 , 2014. · arXiv
  36. Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths. Recasting gradient-based meta-learning as hierarchical bayes. arXiv preprint arXiv:1801.08930 , 2018. · arXiv
  37. Umut G¨ uc ¸l¨ u and Marcel AJ van Gerven. Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. Journal of Neuroscience , 35(27):10005-10014, 2015.
  38. Abhishek Gupta, Russell Mendonca, YuXuan Liu, Pieter Abbeel, and Sergey Levine. Meta-reinforcement learning of structured exploration strategies. arXiv preprint arXiv:1802.07245 , 2018. · arXiv
  39. Dylan Hadfield-Menell, Stuart J Russell, Pieter Abbeel, and Anca Dragan. Cooperative inverse reinforcement learning. In Advances in neural information processing systems , pages 3909-3917, 2016.
  40. James Harrison, Apoorva Sharma, and Marco Pavone. Meta-learning priors for efficient online bayesian regression. arXiv preprint arXiv:1807.08912 , 2018. · arXiv
  41. Geoffrey E Hinton and Steven J Nowlan. How learning can guide evolution. Complex systems , 1(3):495-502, 1987.
  42. Sepp Hochreiter, A Steven Younger, and Peter R Conwell. Learning to learn using gradient descent. In International Conference on Artificial Neural Networks , pages 87-94. Springer, 2001.
  43. Rein Houthooft, Xi Chen, Yan Duan, John Schulman, Filip De Turck, and Pieter Abbeel. Vime: Variational information maximizing exploration. In Advances in Neural Information Processing Systems , pages 1109-1117, 2016.
  44. Francis P Hunkins and Allan C Ornstein. Curriculum: Foundations, principles, and issues . Pearson Education, 2016.
  45. Thomas Jaksch, Ronald Ortner, and Peter Auer. Nearoptimal regret bounds for reinforcement learning. Journal of Machine Learning Research , 11(Apr):1563-1600, 2010.
  46. Ingmar Kanitscheider and Ila Fiete. Training recurrent networks to generate hypotheses about how the brain solves hard navigation problems. In Advances in Neural Information Processing Systems , pages 4529-4538, 2017.
  47. Seyed-Mahdi Khaligh-Razavi and Nikolaus Kriegeskorte. Deep supervised, but not unsupervised, models may explain it cortical representation. PLoS computational biology , 10(11):e1003915, 2014.
  48. Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014. · arXiv
  49. Kai A Krueger and Peter Dayan. Flexible shaping: How learning in small steps helps. Cognition , 110(3):380394, 2009.
  50. M Pawan Kumar, Benjamin Packer, and Daphne Koller. Self-paced learning for latent variable models. In Advances in Neural Information Processing Systems , pages 1189-1197, 2010.
  51. Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induction. Science , 350(6266): 1332-1338, 2015.
  52. Andrew K Lampinen and Surya Ganguli. An analytic theory of generalization dynamics and transfer learning in deep linear networks. arXiv preprint arXiv:1809.10374 , 2018. · arXiv
  53. Yong Jae Lee and Kristen Grauman. Learning the easy things first: Self-paced visual category discovery. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on , pages 1721-1728. IEEE, 2011.
  54. Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, and Shane Legg. Scalable agent alignment via reward modeling: a research direction. arXiv preprint arXiv:1811.07871 , 2018. · arXiv
  55. David D Lewis and William A Gale. A sequential algorithm for training text classifiers. In Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval , pages 3-12. Springer-Verlag New York, Inc., 1994.
  56. Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. In Advances in Neural Information Processing Systems , pages 6391-6401, 2018.
  57. Ke Li and Jitendra Malik. Learning to optimize. arXiv preprint arXiv:1606.01885 , 2016. · arXiv
  58. Zhiyu Lin, Brent Harrison, Aaron Keech, and Mark O Riedl. Explore, exploit or listen: Combining human feedback and policy model to speed up deep reinforcement learning in 3d worlds. arXiv preprint arXiv:1709.03969 , 2017. · arXiv
  59. Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. Meta-learning with temporal convolutions. CoRR , abs/1707.03141, 2017. URL http://arxiv. org/abs/1707.03141 . · arXiv
  60. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. Nature , 518(7540):529, 2015.
  61. Ian SP Nation and John Macalister. Language curriculum design . Routledge, 2009.
  62. Aran Nayebi, Daniel Bear, Jonas Kubilius, Kohitij Kar, Surya Ganguli, David Sussillo, James J DiCarlo, and Daniel L Yamins. Task-driven convolutional recurrent models of the visual system. In Advances in Neural Information Processing Systems , pages 5295-5306, 2018.
  63. Andrew Y Ng, Daishi Harada, and Stuart Russell. Policy invariance under reward transformations: Theory and application to reward shaping. In ICML , volume 99, pages 278-287, 1999.
  64. Bruno A Olshausen and David J Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature , 381(6583):607, 1996.
  65. Ian Osband, Charles Blundell, Alexander Pritzel, and Benjamin Van Roy. Deep exploration via bootstrapped dqn. In Advances in neural information processing systems , pages 4026-4034, 2016.
  66. Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International Conference on Machine Learning (ICML) , volume 2017, 2017.
  67. Jeffrey Pennington, Samuel S Schoenholz, and Surya Ganguli. The emergence of spectral universality in deep networks. arXiv preprint arXiv:1802.09979 , 2018. · arXiv
  68. Gail B Peterson. A day of great illumination: Bf skinner's discovery of shaping. Journal of the Experimental Analysis of Behavior , 82(3):317-328, 2004.
  69. Kim Plunkett and Chris Sinha. Connectionism and developmental theory. British Journal of Developmental Psychology , 10(3):209-254, 1992.
  70. Lutz Prechelt. Early stopping-but when? In Neural Networks: Tricks of the trade , pages 55-69. Springer, 1998.
  71. Neil Rabinowitz, Frank Perbet, Francis Song, Chiyuan Zhang, S. M. Ali Eslami, and Matthew Botvinick. Machine theory of mind. 80:4218-4227, 10-15 Jul 2018. URL http://proceedings.mlr.press/ v80/rabinowitz18a.html .
  72. Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, and Jascha Sohl Dickstein. On the expressive power of deep neural networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70 , pages 2847-2854. JMLR. org, 2017.
  73. Nasim Rahaman, Devansh Arpit, Aristide Baratin, Felix Draxler, Min Lin, Fred A Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of deep neural networks. arXiv preprint arXiv:1806.08734 , 2018. · arXiv
  74. Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. ICLR , 2017.
  75. Larry A Rendell, Raj Sheshu, and David K Tcheng. Layered concept-learning and dynamically variable bias management. In IJCAI , pages 308-314, 1987.
  76. Timothy T Rogers and James L McClelland. Semantic cognition: A parallel distributed processing approach . MIT press, 2004.
  77. Frank Rosenblatt. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review , 65(6):386, 1958.
  78. Stuart Russell, Daniel Dewey, and Max Tegmark. Research priorities for robust and beneficial artificial intelligence. Ai Magazine , 36(4):105-114, 2015.
  79. Andrei A Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. Meta-learning with latent embedding optimization. arXiv preprint arXiv:1807.05960 , 2018. · arXiv
  80. Terence D Sanger. Neural network learning control of robot manipulators using gradually increasing task difficulty. IEEE transactions on Robotics and Automation , 10(3): 323-333, 1994.
  81. AdamSantoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta-learning with memory-augmented neural networks. In International conference on machine learning , pages 1842-1850, 2016.
  82. Andrew M Saxe, James L McClelland, and Surya Ganguli. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint arXiv:1312.6120 , 2013. · arXiv
  83. Andrew M Saxe, James L McClelland, and Surya Ganguli. A mathematical theory of semantic development in deep neural networks. arXiv preprint arXiv:1810.10531 , 2018a. · arXiv
  84. Stefan Schaal. Is imitation learning the route to humanoid robots? Trends in cognitive sciences , 3(6):233-242, 1999.
  85. Stefan Schaal, Auke Ijspeert, and Aude Billard. Computational approaches to motor learning by imitation. Philosophical Transactions of the Royal Society of London B: Biological Sciences , 358(1431):537-547, 2003.
  86. Roger C Schank. Conceptual dependency: A theory of natural language understanding. Cognitive psychology , 3 (4):552-631, 1972.
  87. Tom Schaul, Diana Borsa, Joseph Modayil, and Razvan Pascanu. Ray interference: a source of plateaus in deep reinforcement learning. arXiv preprint arXiv:1904.11455 , 2019. · arXiv
  88. Samuel S Schoenholz, Justin Gilmer, Surya Ganguli, and Jascha Sohl-Dickstein. Deep information propagation. arXiv preprint arXiv:1611.01232 , 2016. · arXiv
  89. Burr Settles. Active learning. Synthesis Lectures on Artificial Intelligence and Machine Learning , 6(1):1-114, 2012.
  90. Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard example mining. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 761-769, 2016.
  91. Ravid Shwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via information. arXiv preprint arXiv:1703.00810 , 2017. · arXiv
  92. Burrhus F Skinner. Reinforcement today. American Psychologist , 13(3):94, 1958.
  93. Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, and Nathan Srebro. The implicit bias of gradient descent on separable data. Journal of Machine Learning Research , 19(70), 2018.
  94. Christopher H Stock, Alex H Williams, Madhu S Advani, Andrew M Saxe, and Surya Ganguli. Learning dynamics of deep networks admit low-rank tensor descriptions. ICLR 2018 workshop submission , 2018.
  95. David G Stork. Is backpropagation biologically plausible. In International Joint Conference on Neural Networks , volume 2, pages 241-246. IEEE Washington, DC, 1989.
  96. Weijie Su, Stephen Boyd, and Emmanuel Candes. A differential equation for modeling nesterovs accelerated gradient method: Theory and insights. In Advances in Neural Information Processing Systems , pages 2510-2518, 2014.
  97. Haoran Tang, Rein Houthooft, Davis Foote, Adam Stooke, OpenAI Xi Chen, Yan Duan, John Schulman, Filip DeTurck, and Pieter Abbeel. # exploration: A study of count-based exploration for deep reinforcement learning. In Advances in Neural Information Processing Systems , pages 2753-2762, 2017.
  98. Sebastian Thrun. Lifelong learning algorithms. In Learning to learn , pages 181-209. Springer, 1998.
  99. Sebastian Thrun and Lorien Pratt. Learning to learn: Introduction and overview. In Learning to learn , pages 3-17. Springer, 1998.
  100. Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J Gordon. An empirical study of example forgetting during deep neural network learning. arXiv preprint arXiv:1812.05159 , 2018. · arXiv
  101. Oriol Vinyals, Charles Blundell, Tim Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In Advances in Neural Information Processing Systems , pages 3630-3638, 2016.
  102. Jane X Wang, Zeb Kurth-Nelson, Dhruva Tirumala, Hubert Soyer, Joel Z Leibo, Remi Munos, Charles Blundell, Dharshan Kumaran, and Matt Botvinick. Learning to reinforcement learn. arXiv preprint arXiv:1611.05763 , 2016. · arXiv
  103. Daphna Weinshall and Dan Amir. Theory of curriculum learning, with convex loss functions. arXiv preprint arXiv:1812.03472 , 2018. · arXiv
  104. Andre Wibisono, Ashia C Wilson, and Michael I Jordan. A variational perspective on accelerated methods in optimization. Proceedings of the National Academy of Sciences , 113(47):E7351-E7358, 2016.
  105. Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning , 8(3-4):229-256, 1992.
  106. Aaron Wilson, Alan Fern, and Prasad Tadepalli. A bayesian approach for policy learning from trajectory preference queries. In Advances in neural information processing systems , pages 1133-1141, 2012.
  107. Zhi-Qin J Xu, Yaoyu Zhang, and Yanyang Xiao. Training behavior of deep neural network in frequency domain. arXiv preprint arXiv:1807.01251 , 2018. · arXiv
  108. Zhiqin John Xu. Understanding training and generalization in deep learning by fourier analysis. arXiv preprint arXiv:1808.04295 , 2018. · arXiv
  109. Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James J DiCarlo. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences , 111(23):8619-8624, 2014.
  110. Lin Yang, Raman Arora, Tuo Zhao, et al. The physical systems behind optimization algorithms. In Advances in Neural Information Processing Systems , pages 43774386, 2018.

Cited by

Sorted by citations

Authors
Rabinowitz, N. C.
Corresponding author
Rabinowitz, Neil C.
ORCID
Rabinowitz, Neil C.
Venue
arXiv
Publisher
arXiv
Published
2019
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1905.01320
arXiv
1905.01320
OpenAlex
W2944609053
Cited by
7
References
0
Citations by year
2024: 1 · 2022: 2 · 2021: 1 · 2020: 1 · 2019: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
meta-learning, learning dynamics, sample efficiency, learning trajectories, linear regression, nonlinear regression, contextual bandits, reinforcement learning, deep learning, curriculum development, human-in-the-loop learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review