The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
A survey of deep meta-learning
Open accessAbstract
Deep neural networks can achieve great successes when presented with large data sets and sufficient computational resources. However, their ability to learn new concepts quickly is limited. Meta-learning is one approach to address this issue, by enabling the network to learn how to learn. The field of Deep Meta-Learning advances at great speed, but lacks a unified, in-depth overview of current techniques. With this work, we aim to bridge this gap. After providing the reader with a theoretical foundation, we investigate and summarize key methods, which are categorized into (i) metric-, (ii) model-, and (iii) optimization-based techniques. In addition, we identify the main open challenges, such as performance evaluations on heterogeneous benchmarks, and reduction of the computational costs of meta-learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
International Conference on Learning Representations (ICLR 2013)
松尾, 豊. (2013) · 人工知能学会誌 = Journal of Japanese Society for Artificial Intelligence · Cited by 6,286
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550
Building Machines That Learn and Think Like People
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2016) · arXiv · doi:10.48550/arxiv.1604.00289 · Cited by 2,550
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484
The Theory and Practice of Online Learning
(2008) · Athabasca University Press eBooks · doi:10.15215/aupress/9781897425084.01 · Cited by 1,841
Meta-Learning With Differentiable Convex Optimization
Lee, K., Maji, S., Ravichandran, A., & Soatto, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01091 · Cited by 1,260
OpenML: networked science in machine learning
Vanschoren, J., van Rijn, J. N., Bischl, B., & Torgo, L. (2014) · ACM SIGKDD Explorations Newsletter · doi:10.1145/2641190.2641198 · Cited by 1,141
Self-citationMeta-Learning in Neural Networks: A Survey
Hospedales, T., Antoniou, A., Micaelli, P., & Storkey, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05439 · Cited by 886
Efficient algorithms for online decision problems
Kalai, A., & Vempala, S. (2005) · Journal of Computer and System Sciences · doi:10.1016/j.jcss.2004.10.016 · Cited by 591
Learning to Learn Using Gradient Descent
Hochreiter, S., Younger, A. S., & Conwell, P. R. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44668-0_13 · Cited by 553
Thrun, S. (1998) · Learning to Learn · doi:10.1007/978-1-4615-5529-2_8 · Cited by 413
Learning a synaptic learning rule
Bengio, Y., Bengio, S., & Cloutier, J. (1991) · IJCNN-91-Seattle International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1991.155621 · Cited by 231
Schmidhuber, J., Zhao, J., & Wiering, M. (1997) · Machine Learning · doi:10.1023/a:1007383707642 · Cited by 169
Learning Compositional Representations for Few-Shot Recognition
Tokmakov, P., Wang, Y.-X., & Hebert, M. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00647 · Cited by 140
CapsNet topology to classify tumours from brain images and comparative evaluation
Göçeri, E. (2019) · IET Image Processing · doi:10.1049/iet-ipr.2019.0312 · Cited by 105
Meta-Learning in Computational Intelligence
Jankowski, N., Gra̧bczewski, K., & Duch, W. (2011) · Studies in computational intelligence · doi:10.1007/978-3-642-20980-2 · Cited by 97
A neural network that embeds its own meta-levels
Schmidhuber, J. (2002) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/icnn.1993.298591 · Cited by 95
Challenges and Recent Solutions for Image Segmentation in the Era of Deep Learning
Göçeri, E. (2019) · International Conference on Image Processing Theory, Tools and Applications (IPTA) · doi:10.1109/ipta.2019.8936087 · Cited by 89
Meta-learning with backpropagation
Younger, A. S., Hochreiter, S., & Conwell, P. R. (2002) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2001.938471 · Cited by 83
Convolutional Neural Network Based Desktop Applications to Classify Dermatological Diseases
Göçeri, E. (2020) · IEEE International Conference on Image Processing, Applications and Systems (IPAS) · doi:10.1109/ipas50080.2020.9334956 · Cited by 61
Deep learning recognition of diseased and normal cell representation
Iqbal, M. S., Ahmad, I. A., Bin, L., Khan, S., & Rodrigues, J. J. P. C. (2020) · Transactions on Emerging Telecommunications Technologies · doi:10.1002/ett.4017 · Cited by 48
Efficient cell classification of mitochondrial images by using deep learning
Iqbal, M. S., El-Ashram, S. A., Hussain, S., Khan, T., Huang, S., Mehmood, R., & Luo, B. (2019) · Journal of Optics · doi:10.1007/s12596-018-0508-4 · Cited by 41
Iqbal, M. S., Luo, B., Mehmood, R., Alrige, M., & Alharbey, R. (2019) · IEEE Access · doi:10.1109/access.2019.2925041 · Cited by 28
Heterogeneous transfer learning techniques for machine learning
Iqbal, M. S., Luo, B., Khan, T., Mehmood, R., & Sadiq, M. (2018) · Iran Journal of Computer Science · doi:10.1007/s42044-017-0004-z · Cited by 16
Cited by
Explainable artificial intelligence: a comprehensive review
Minh, D., Wang, H. X., Li, Y. F., & Nguyen, T. N. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10088-y · Cited by 983
Alzubaidi, L., Bai, J., Al-Sabaawi, A., Santamaría, J., Albahri, A. S., Al-dabbagh, B. S. N., Fadhel, M. A., Manoufali, M., Zhang, J., Al-Timemy, A. H., et al. (2023) · Journal of Big Data · doi:10.1186/s40537-023-00727-2 · Cited by 907
Advances and Challenges in Meta-Learning: A Technical Review
Vettoruzzo, A., Bouguelia, M.-R., Vanschoren, J., Rögnvaldsson, T., & Santosh, K. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3357847 · Cited by 314
Feng, Y., Chen, J., Xie, J., Zhang, T., Lv, H., & Pan, T. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107646 · Cited by 262
Face2Exp: Combating Data Biases for Facial Expression Recognition
Zeng, D., Lin, Z., Yan, X., Liu, Y., Wang, F., & Tang, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01965 · Cited by 141
“Specifically, it can be regarded as a bilevel optimization problem [15], which is primarily concerned with networks learned at different levels: an inner- and an outer-level optimization.”
Background
Metadata
- Authors
- Huisman, M., van Rijn, J. N., & Plaat, A.
- Venue
- Artificial Intelligence Review
- Published
- 2021-04-19
- Volume
- 54
- Issue
- 6
- Pages
- 4483-4541
- ISSN
- 0269-2821 (ISSN-L)
- 1573-7462
- OpenAlex
- W3092053846
- Cited by
- 376
- References
- 30
- FWCI
- 21.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.