Cited by 3,165Domain Adaptation and Few-Shot LearningMachine Learning and AlgorithmsMachine Learning and Data Classification

Generalizing from a Few Examples: A Survey on Few-shot Learning

Wang, Yaqing, Yao, Quanming, Kwok, James T., Ni, Lionel M.·2021·ACM Computing Surveys

Abstract

Machine learning has been highly successful in data-intensive applications but is often hampered when the data set is small. Recently, Few-shot Learning (FSL) is proposed to tackle this problem. Using prior knowledge, FSL can rapidly generalize to new tasks containing only a few samples with supervised information. In this article, we conduct a thorough survey to fully understand FSL. Starting from a formal definition of FSL, we distinguish FSL from several relevant machine learning problems. We then point out that the core issue in FSL is that the empirical risk minimizer is unreliable. Based on how prior knowledge can be used to handle this core issue, we categorize FSL methods from three perspectives: (i) data, which uses prior knowledge to augment the supervised experience; (ii) model, which uses prior knowledge to reduce the size of the hypothesis space; and (iii) algorithm, which uses prior knowledge to alter the search for the best hypothesis in the given hypothesis space. With this taxonomy, we review and discuss the pros and cons of each category. Promising directions, in the aspects of the FSL problem setups, techniques, applications, and theories, are also proposed to provide insights for future research. 1

Integrity signals

Registry

Crossref · 10.1145/3386252

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In other research domains, few-shot learning has been shown to tune a pre-trained neural network and improve its generalization.

    “However, in other research domains, it was shown [39] that a few-shot learning can be used to tune a pre-trained neural network and it will improve its generalization.”
  • Wei et al. showed that soft-prompt tuning can relax the non-degeneracy assumptions needed for downstream recovery, making it easier to extract task-specific information.

    “Wei et al. [141] showed that soft-prompt tuning can relax the non-degeneracy assumptions (the generation probability of each token is linearly independent) needed for downstream recovery (i.e., recover the ground-truth labels of the downstream task.).making it easier to extract task-specific information.”
  • This gap can result in overfitting, where the model excels on training data but underperforms on testing data, compromising its generalization ability.

    “This gap can result in overfitting, where the model excels on training data but underperforms on testing data, compromising its generalization ability [27, 39].”
  • When supervised information in D_train is sufficient, empirical risk minimization function in F can approximate the best-expected risk minimization function in F well.

    “As illustrated in Fig. 4(a), when the supervised information in D train is sufficient, i.e., I in D train is large enough, the empirical risk minimization function in F can approximate the best-expected risk minimization function in F well, i.e., f e can provide a good approximation to f b .”
  • Empirical risk minimization does not ensure strong generalization on test data, especially with limited training samples.

    “This approach doesn't ensure strong generalization on test data, especially with limited training samples.”

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M.
Affiliations
  • Hong Kong University of Science and Technology (HK)
  • 4Paradigm Inc., Kwun Tong, Kowloon, Hong Kong
Venue
ACM Computing Surveys
Published
2020-06-12
Volume
53
Issue
3
Pages
1-34
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
OpenAlex
W3034942609
Cited by
3,165
References
57
Citations by year
2026: 332 · 2025: 596 · 2024: 625 · 2023: 612 · 2022: 504 · 2021: 382 · 2020: 106 · 2019: 8
FWCI
157.65
Type
journal article
Language
en
Open access
closed
Locations
  • ACM Computing Surveys (publishedVersion)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
Keywords
few-shot learning, small datasets, prior knowledge, data augmentation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review