Meta-Learning
Open accessThe DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index.
Abstract
Meta-learning, or learning to learn , is the science of systematically observing how different machine learning approaches perform on a wide range of learning tasks, and then learning from this experience, or meta-data , to learn new tasks much faster than otherwise possible. Not only does this dramatically speed up and improve the design of machine learning pipelines or neural architectures, it also allows us to replace hand-engineered algorithms with novel approaches learned in a data-driven way. In this chapter, we provide an overview of the state of the art in this fascinating and continuously evolving field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Lecture Notes in Computer Science 1205
(1999) · Industrial Robot: An International Journal · doi:10.1108/ir.1999.04926fae.001 · Cited by 38,662
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
Efficient Global Optimization of Expensive Black-Box Functions
Jones, D. R., Schonlau, M., & Welch, W. J. (1998) · Journal of Global Optimization · doi:10.1023/a:1008306431147 · Cited by 8,070
Lecture Notes in Artificial Intelligence
Brézillon, P., & Bouquet, P. (1999) · Cited by 7,473
A NEW MEASURE OF RANK CORRELATION
KENDALL, M. G. (1938) · Biometrika · doi:10.1093/biomet/30.1-2.81 · Cited by 7,269
Agresti, A. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/0471249688 · Cited by 6,746
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451
Nadaraya, E. A. (1964) · Theory of Probability & Its Applications · doi:10.1137/1109020 · Cited by 3,462
Rendle, S. (2010) · 2010 IEEE International Conference on Data Mining · doi:10.1109/icdm.2010.127 · Cited by 3,189
One-shot learning of object categories
Fei-Fei, L., Fergus, R., & Perona, P. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.79 · Cited by 3,029
THOMPSON, W. R. (1933) · Biometrika · doi:10.1093/biomet/25.3-4.285 · Cited by 2,712
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550
Building Machines That Learn and Think Like People
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2016) · arXiv · doi:10.48550/arxiv.1604.00289 · Cited by 2,550
Advances in Neural Information Processing Systems 32 (NeurIPS 2019)
Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché–Buc, F., Fox, E. B., & Garnett, R. (2020) · HAL · Cited by 2,451
Lecture Notes in Computer Science 1867
Baldwin, J., Martin, T., & Tzanavari, A. (2000) · Cited by 1,643
Regularized multi--task learning
Evgeniou, T., & Pontil, M. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014067 · Cited by 1,433
OpenML: networked science in machine learning
Vanschoren, J., van Rijn, J. N., Bischl, B., & Torgo, L. (2014) · ACM SIGKDD Explorations Newsletter · doi:10.1145/2641190.2641198 · Cited by 1,141
Self-citationWiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery
(2011) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/(issn)1942-4795 · Cited by 1,007
Complexity measures of supervised classification problems
Ho, T. K., & Basu, M. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.990132 · Cited by 734
Evaluation of a Tree-based Pipeline Optimization Tool for Automating Data Science
Olson, R. S., Bartley, N., Urbanowicz, R. J., & Moore, J. H. (2016) · Proceedings of the Genetic and Evolutionary Computation Conference 2016 · doi:10.1145/2908812.2908918 · Cited by 595
Google Vizier: A Service for Black-Box Optimization
Golovin, D., Solnik, B., Moitra, S., Kochanski, G., Karro, J., & Sculley, D. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098043 · Cited by 571
Learning to Learn Using Gradient Descent
Hochreiter, S., Younger, A. S., & Conwell, P. R. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44668-0_13 · Cited by 553
Cross-disciplinary perspectives on meta-learning for algorithm selection
Smith-Miles, K. A. (2009) · ACM Computing Surveys · doi:10.1145/1456650.1456656 · Cited by 552
Learning to Learn: Introduction and Overview
Thrun, S., & Pratt, L. (1998) · Learning to Learn · doi:10.1007/978-1-4615-5529-2_1 · Cited by 521
Thrun, S. (1998) · Learning to Learn · doi:10.1007/978-1-4615-5529-2_8 · Cited by 413
Ranking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results
Brazdil, P. B., Soares, C., & da Costa, J. P. (2003) · Machine Learning · doi:10.1023/a:1021713901879 · Cited by 406
Algorithm runtime prediction: Methods & evaluation
Hutter, F., Xu, L., Hoos, H. H., & Leyton-Brown, K. (2014) · Artificial Intelligence · doi:10.1016/j.artint.2013.10.003 · Cited by 395
On learning algorithm selection for classification
Ali, S., & Smith, K. A. (2006) · Applied Soft Computing · doi:10.1016/j.asoc.2004.12.002 · Cited by 374
Metalearning: A survey of trends and technologies
Lemke, C., Budka, M., & Gabrys, B. (2015) · Artificial Intelligence Review · doi:10.1007/s10462-013-9406-y · Cited by 365
Learning to Control Fast-Weight Memories: An Alternative to Dynamic Recurrent Networks
Schmidhuber, J. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.131 · Cited by 357
A review of automatic selection methods for machine learning algorithms and hyper-parameter values
Luo, G. (2016) · Network Modeling Analysis in Health Informatics and Bioinformatics · doi:10.1007/s13721-016-0125-6 · Cited by 357
Constructing informative priors using transfer learning
Raina, R., Ng, A. Y., & Koller, D. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143934 · Cited by 299
Metalearning: Applications to Data Mining
Brazdil, P., Giraud-Carrier, C., Soares, C., & Vilalta, R. (2009) · Cognitive Technologies · doi:10.1007/978-3-540-73263-1 · Cited by 264
(2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9 · Cited by 236
Advances in intelligent data analysis
Hand, D. J., Fisher, D. H., & Berthold, M. R. (2000) · Intelligent Data Analysis · doi:10.3233/ida-2000-4201 · Cited by 222
Lin, S. (2010) · WIREs Computational Statistics · doi:10.1002/wics.111 · Cited by 215
ASlib: A benchmark library for algorithm selection
Bischl, B., Kerschke, P., Kotthoff, L., Lindauer, M., Malitsky, Y., Fréchette, A., Hoos, H., Hutter, F., Leyton-Brown, K., Tierney, K., et al. (2016) · Artificial Intelligence · doi:10.1016/j.artint.2016.04.003 · Cited by 214
Self-citationHyperparameter Importance Across Datasets
van Rijn, J. N., & Hutter, F. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220058 · Cited by 210
The Evolution of Learning: An Experiment in Genetic Connectionism
Chalmers, D. J. (1991) · Connectionist Models · doi:10.1016/b978-1-4832-1448-1.50014-7 · Cited by 193
Meta-learning for evolutionary parameter optimization of classifiers
Reif, M., Faisal Shafait, B., & Dengel, A. R. (2012) · Machine Learning · doi:10.1007/s10994-012-5286-7 · Cited by 193
A Meta-Learning Method to Select the Kernel Width in Support Vector Regression
Soares, C., Brazdil, P. B., & Kuba, P. (2004) · Machine Learning · doi:10.1023/b:mach.0000015879.28004.9b · Cited by 187
Combining meta-learning and search techniques to select parameters for support vector machines
Gomes, T. A. F., Prudêncio, R. B. C., Soares, C., Rossi, A. L. D., & de Carvalho, A. C. P. L. F. (2011) · Neurocomputing · doi:10.1016/j.neucom.2011.07.005 · Cited by 178
Automatic classifier selection for non-experts
Reif, M., Faisal Shafait, B., Goldstein, M., Breuel, T. M., & Dengel, A. R. (2012) · Pattern Analysis and Applications · doi:10.1007/s10044-012-0280-z · Cited by 177
Schmidhuber, J., Zhao, J., & Wiering, M. (1997) · Machine Learning · doi:10.1023/a:1007383707642 · Cited by 169
Some Aspects of the Sequential Design of Experiments
Robbins, H. E. (1985) · Springer eBooks · doi:10.1007/978-1-4612-5110-1_13 · Cited by 168
Meta-learning approaches to selecting time series models
Prudêncio, R. B. C., & Ludermir, T. B. (2004) · Neurocomputing · doi:10.1016/j.neucom.2004.03.008 · Cited by 154
A meta-learning approach to automatic kernel selection for support vector machines
Ali, S., & Smith‐Miles, K. (2006) · Neurocomputing · doi:10.1016/j.neucom.2006.03.004 · Cited by 148
Thrun, S., & Mitchell, T. M. (1994) · Defense Technical Information Center (DTIC) · doi:10.21236/ada285342 · Cited by 145
Decision tree and instance-based learning for label ranking
Cheng, W., Hühn, J., & Hüllermeier, E. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553395 · Cited by 139
OBOE: Collaborative Filtering for AutoML Model Selection
Yang, C., Akimoto, Y., Kim, D. W., & Udell, M. R. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330909 · Cited by 125
Experiment databases: A new way to share, organize and learn from experiments
Vanschoren, J., Blockeel, H., Pfahringer, B., & Holmes, G. (2012) · Machine Learning · doi:10.1007/s10994-011-5277-0 · Cited by 117
Self-citationRECIPE: A Grammar-Based Framework for Automatically Evolving Classification Pipelines
de Sá, A. G. C., Pinto, W. J. G. S., Oliveira, L. O. V. B., & Pappa, G. (2017) · European Conference on Genetic Programming · doi:10.1007/978-3-319-55696-3_16 · Cited by 115
A survey of intelligent assistants for data analysis
Serban, F., Vanschoren, J., Kietz, J.-U., & Bernstein, A. (2013) · ACM Computing Surveys · doi:10.1145/2480741.2480748 · Cited by 111
Self-citationLearning hyperparameter optimization initializations
Wistuba, M., Schilling, N., & Schmidt-Thieme, L. (2015) · International Conference on Data Science and Advanced Analytics · doi:10.1109/dsaa.2015.7344817 · Cited by 111
Hyper-Parameter Tuning of a Decision Tree Induction Algorithm
Mantovani, R. G., Horváth, T., Cerri, R., Vanschoren, J., & de Carvalho, A. D. F. (2016) · Brazilian Conference on Intelligent Systems · doi:10.1109/bracis.2016.018 · Cited by 101
Self-citationEstimating the Predictive Accuracy of a Classifier
Bensusan, H. N., & Kalousis, A. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44795-4_3 · Cited by 100
Pairwise meta-rules for better meta-learning-based algorithm ranking
Sun, Q., & Pfahringer, B. (2013) · Machine Learning · doi:10.1007/s10994-013-5387-y · Cited by 99
Meta-data: Characterization of Input Features for Meta-learning
Castiello, C., Castellano, G., & Fanelli, A. M. (2005) · Lecture notes in computer science · doi:10.1007/11526018_45 · Cited by 95
Meta-QSAR: a large-scale application of meta-learning to drug design and discovery
Olier, I., Sadawi, N. M., Bickerton, G. R., Vanschoren, J., Groşan, C., Soldatova, L., & King, R. (2017) · Machine-mediated learning · doi:10.1007/s10994-017-5685-x · Cited by 92
Self-citationAlors: An algorithm recommender system
Mısır, M., & Sebag, M. (2017) · Artificial Intelligence · doi:10.1016/j.artint.2016.12.001 · Cited by 91
Learning Step Size Controllers for Robust Neural Network Training
Daniel, C., Taylor, J., & Nowozin, S. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10187 · Cited by 83
Predicting relative performance of classifiers from samples
Leite, R., & Brazdil, P. (2005) · International Conference on Machine Learning · doi:10.1145/1102351.1102414 · Cited by 82
Wistuba, M., Schilling, N., & Schmidt-Thieme, L. (2015) · ECML/PKDD · doi:10.1007/978-3-319-23525-7_7 · Cited by 78
Lecture Notes in Artificial Intelligence
Pontier, Widdershoven, G. A. M., & Hoorn, J. F. (2012) · Pure Amsterdam UMC · Cited by 73
Fast Algorithm Selection Using Learning Curves
van Rijn, J. N., Abdulrahman, S. M., Brazdil, P., & Vanschoren, J. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24465-5_26 · Cited by 71
Self-citationTo tune or not to tune: Recommending when to adjust SVM hyper-parameters via meta-learning
Mantovani, R. G., Rossi, A., Vanschoren, J., Bischl, B., & de Carvalho, A. (2015) · IEEE International Joint Conference on Neural Network · doi:10.1109/ijcnn.2015.7280644 · Cited by 64
Self-citationFeature Selection for Meta-learning
Kalousis, A., & Hilario, M. (2001) · Lecture notes in computer science · doi:10.1007/3-540-45357-1_26 · Cited by 59
Datasets meta-feature description for recommending feature selection algorithm
Filchenkov, A. A., & Pendryak, A. (2015) · Artificial Intelligence and Natural Language · doi:10.1109/ainl-ismw-fruct.2015.7382962 · Cited by 59
Towards Automatic Generation of Metafeatures
Pinto, F., Soares, C., & Mendes‐Moreira, J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-31753-3_18 · Cited by 59
Algorithm Selection on Data Streams
van Rijn, J. N., Holmes, G., Pfahringer, B., & Vanschoren, J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11812-3_28 · Cited by 58
Self-citationIntelligent assistance for data pre-processing
Bilalli, B., Abelló, A., Aluja-Banet, T., & Wrembel, R. (2018) · Comput. Stand. Interfaces · doi:10.1016/j.csi.2017.05.004 · Cited by 58
Efficient benchmarking of algorithm configurators via model-based surrogates
Eggensperger, K., Lindauer, M., Hoos, H., Hutter, F., & Leyton-Brown, K. (2017) · Machine-mediated learning · doi:10.1007/s10994-017-5683-z · Cited by 57
On the predictive power of meta-features in OpenML
Bilalli, B., Abelló, A., & Aluja‐Banet, T. (2017) · International Journal of Applied Mathematics and Computer Science · doi:10.1515/amcs-2017-0048 · Cited by 53
Speeding up algorithm selection using average ranking and active testing by introducing runtime
Abdulrahman, S. M., Brazdil, P., van Rijn, J. N., & Vanschoren, J. (2017) · Machine Learning · doi:10.1007/s10994-017-5687-8 · Cited by 52
Self-citationUsing Meta-mining to Support Data Mining Workflow Planning and Optimization
Nguyen, P., Hilario, M., & Kalousis, A. (2014) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4377 · Cited by 51
Hyperparameter Optimization with Factorized Multilayer Perceptrons
Schilling, N., Wistuba, M., Drumond, L., & Schmidt-Thieme, L. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-23525-7_6 · Cited by 48
Collaborative Expert Portfolio Management
Stern, D., Samulowitz, H., Herbrich, R., Graepel, T., Pulina, L., & Tacchella, A. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7561 · Cited by 45
Experiments in Meta-level Learning with ILP
Todorovski, L., & Džeroski, S. (1999) · Lecture notes in computer science · doi:10.1007/978-3-540-48247-5_11 · Cited by 43
Informing the Use of Hyperparameter Optimization Through Metalearning
Sanders, S., & Giraud-Carrier, C. (2017) · Industrial Conference on Data Mining · doi:10.1109/icdm.2017.137 · Cited by 40
On the search for new learning rules for ANNs
Samy Bengio, A., Bengio, Y., & Cloutier, J. (1995) · Neural Processing Letters · doi:10.1007/bf02279935 · Cited by 35
Predicting the Performance of Learning Algorithms Using Support Vector Machines as Meta-regressors
Guerra, S. B., Prudêncio, R. B. C., & Ludermir, T. B. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87536-9_54 · Cited by 34
Fusion of Meta-knowledge and Meta-data for Case-Based Model Selection
Hilario, M., & Kalousis, A. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44794-6_15 · Cited by 28
Towards a Framework for Designing Full Model Selection and Optimization Systems
Sun, Q., Pfahringer, B., & Mayo, M. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-38067-9_23 · Cited by 24
SHARKEY, N. E., & SHARKEY, A. J. C. (1993) · Artificial Intelligence Review · doi:10.1007/bf00849058 · Cited by 21
Information-Theoretic Transfer Learning Framework for Bayesian Optimisation
Ramachandran, A., Gupta, S., Rana, S., & Venkatesh, S. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-10928-8_49 · Cited by 20
Employment of neural network and rough set in meta-learning
Salama, M. A., Hassanien, A. E., & Revett, K. (2013) · Memetic Computing · doi:10.1007/s12293-013-0114-6 · Cited by 13
Vanschoren, J., & Blockeel, H. (2010) · Springer eBooks · doi:10.1007/978-1-4419-7738-0_14 · Cited by 7
Self-citation
Cited by
Medical image segmentation using deep learning: A survey
Wang, R., Lei, T., Cui, R., Zhang, B., Meng, H., & Nandi, A. K. (2022) · IET Image Processing · doi:10.1049/ipr2.12419 · Cited by 892
Guo, J., Wen, C.-K., Jin, S., & Li, G. Y. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2968430 · Cited by 388
Self-Supervised Learning in Remote Sensing: A review
Wang, Y., Albrecht, C. M., Braham, N. A. A., Mou, L., & Zhu, X. X. (2022) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2022.3198244 · Cited by 364
Advances and Challenges in Meta-Learning: A Technical Review
Vettoruzzo, A., Bouguelia, M.-R., Vanschoren, J., Rögnvaldsson, T., & Santosh, K. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3357847 · Cited by 314
Self-citationMore Than Privacy: Applying Differential Privacy in Key Areas of Artificial Intelligence
Zhu, T., Ye, D., Wang, W., Zhou, W., & Yu, P. S. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.3014246 · Cited by 219
Metadata
- Authors
- Vanschoren, J.
- Venue
- The Springer Series on Challenges in Machine Learning
- Publisher
- Springer International Publishing
- Published
- 2019
- Pages
- 35-61
- ISSN
- 2520-131X (ISSN-L)
- 2520-1328
- OpenAlex
- W4211208325
- Cited by
- 328
- References
- 91
- FWCI
- 40.56
- Type
- book chapter
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.