CoChat
Sign Up
Cited by 722Domain Adaptation and Few-Shot LearningAdvanced Neural Network ApplicationsMachine Learning and Algorithms

The DOI matches its Crossref record; 216 of the 219 references it cites were found in a registry or an index.

A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities

Open access
Song, Yisheng, Wang, Ting, Cai, Puyu, Mondal, Subrota K., Sahoo, Jyoti Prakash·2023·ACM Computing Surveys
DOI 10.1145/3582688
Open this paper in CoChat — free accountOpen PDF

Abstract

Few-shot learning (FSL) has emerged as an effective learning method and shows great potential. Despite the recent creative works in tackling FSL tasks, learning valid information rapidly from just a few or even zero samples remains a serious challenge. In this context, we extensively investigated 200+ FSL papers published in top journals and conferences in the past three years, aiming to present a timely and comprehensive overview of the most recent advances in FSL with a fresh perspective and to provide an impartial comparison of the strengths and weaknesses of existing work. To avoid conceptual confusion, we first elaborate and contrast a set of relevant concepts including few-shot learning, transfer learning, and meta-learning. Then, we inventively extract prior knowledge related to few-shot learning in the form of a pyramid, which summarizes and classifies previous work in detail from the perspective of challenges. Furthermore, to enrich this survey, we present in-depth analysis and insightful discussions of recent advances in each subsection. What is more, taking computer vision as an example, we highlight the important application of FSL, covering various research hotspots. Finally, we conclude the survey with unique insights into technology trends and potential future research opportunities to guide FSL follow-up research.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3582688

Cites

216 of 216 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

3 of the 219 references couldn’t be matched to a known paper.

Research works printed but not linked: 50

  1. Wenbo Zheng, Lan Yan, Chao Gou, and Fei-Yue Wang. 2021. Federated meta-learning for fraudulent credit card detection. In Proceedings of the 29th International Conference on International Joint Conference on Artificial Intelligence. 4654–4660.
  2. Adrian El Baz, Ihsan Ullah, et al. 2022. Lessons learned from the NeurIPS 2021 MetaDL challenge: Backbone fine-tuning without episodic meta-learning dominates for few-shot learning image classification. In NeurIPS 2021 Competitions and Demonstrations Track. PMLR, 80–96.
  3. Jiang Lu, Pinghua Gong, Jieping Ye, and Changshui Zhang. 2020. Learning from very few samples: A survey. arXiv preprint arXiv:2009.02653 (2020). · arXiv
  4. Luiz F. O. Chamon and Alejandro Ribeiro. 2020. Probably approximately correct constrained learning. In Proceedings of the Annual Conference on Neural Information Processing Systems. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/c291b01517f3e6797c774c306591cc32-Abstract.html.
  5. Shuo Yang, Lu Liu, and Min Xu. 2021. Free lunch for few-shot learning: Distribution calibration. In Proceedings of the 9th International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=JWOiYxMG92s.
  6. Atsushi Suzuki, Atsushi Nitanda, Jing Wang, Linchuan Xu, Kenji Yamanishi, and Marc Cavazza. 2021. Generalization error bound for hyperbolic ordinal embedding. In Proceedings of the 38th International Conference on Machine Learning. PMLR, 10011–10021. Retrieved from http://proceedings.mlr.press/v139/suzuki21a.html.
  7. Tianshi Cao, Marc T. Law, and Sanja Fidler. 2020. A theoretical analysis of the number of shots in few-shot learning. In Proceedings of the 8th International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=HkgB2TNYPS.
  8. Kevin Li, Abhishek Gupta, Ashwin Reddy, Vitchyr H. Pong, Aurick Zhou, Justin Yu, and Sergey Levine. 2021. MURAL: Meta-learning uncertainty-aware rewards for outcome-driven reinforcement learning. In Proceedings of the International Conference on Machine Learning. PMLR, 6346–6356.
  9. Eleni Triantafillou, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Utku Evci, Kelvin Xu, Ross Goroshin, Carles Gelada, Kevin Swersky, Pierre-Antoine Manzagol, and Hugo Larochelle. 2020. Meta-Dataset: A dataset of datasets for learning to learn from few examples. In Proceedings of the 8th International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=rkgAGAVKPr.
  10. Patrizio Bellan, Han van der Aa, Mauro Dragoni, Chiara Ghidini, and Simone Paolo Ponzetto. 2022. PET: A new dataset for process extraction from natural language text. arXiv e-prints (2022), arXiv–2203.
  11. 10.1016/j.cobeha.2019.04.007 · DOI
  12. Cheng Perng Phoo and Bharath Hariharan. 2021. Self-training for few-shot transfer across extreme task differences. In Proceedings of the 9th International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=O3Y56aqpChA.
  13. Ibrahim Aksu, Zhengyuan Liu, Min-Yen Kan, and Nancy Chen. 2022. N-shot learning for augmenting task-oriented dialogue state tracking. In Findings of the Association for Computational Linguistics. 1659–1671.
  14. Hang Gao, Zheng Shou, Alireza Zareian, Hanwang Zhang, and Shih-Fu Chang. 2018. Low-shot learning via covariance-preserving adversarial augmentation networks. In Proceedings of the Annual Conference on Neural Information Processing Systems. 983–993. Retrieved from https://proceedings.neurips.cc/paper/2018/hash/81448138f5f163ccdba4acc69819f280-Abstract.html.
  15. Eli Schwartz, Leonid Karlinsky, Joseph Shtok, Sivan Harary, Mattias Marder, Abhishek Kumar, Rogério Schmidt Feris, Raja Giryes, and Alexander M. Bronstein. 2018. Delta-encoder: An effective sample synthesis method for few-shot object recognition. In Proceedings of the Annual Conference on Neural Information Processing Systems. 2850–2860. Retrieved from https://proceedings.neurips.cc/paper/2018/hash/1714726c817af50457d810aae9d27a2e-Abstract.html.
  16. Wentao Cui and Yuhong Guo. 2021. Parameterless transductive feature re-representation for few-shot learning. In Proceedings of the International Conference on Machine Learning. PMLR, 2212–2221.
  17. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An image is worth 16x16 words: Transformers for image recognition at scale. In Proceedings of the 9th International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=YicbFdNTTy.
  18. Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. 2020. Generative pretraining from pixels. In Proceedings of the International Conference on Machine Learning. PMLR, 1691–1703.
  19. Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. 2019. A closer look at few-shot classification. In Proceedings of the 7th International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=HkxLXnAcFQ.
  20. Jathushan Rajasegaran, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Mubarak Shah. 2021. Self-supervised knowledge distillation for few-shot learning. In Proceedings of the 32nd British Machine Vision Conference. BMVA Press, 179. Retrieved from https://www.bmvc2021-virtualconference.com/assets/papers/0820.pdf.
  21. Carl Doersch, Ankush Gupta, and Andrew Zisserman. 2020. CrossTransformers: Spatially-aware few-shot transfer. In Proceedings of the Annual Conference on Neural Information Processing Systems. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/fa28c6cdf8dd6f41a657c3d7caa5c709-Abstract.html.
  22. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 7th International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=rJ4km2R5t7.
  23. Tianyi Zhang, Felix Wu, Arzoo Katiyar, Kilian Q. Weinberger, and Yoav Artzi. 2021. Revisiting few-sample BERT fine-tuning. In Proceedings of the 9th International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=cO1IH43yUF.
  24. Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the International Conference on Machine Learning. PMLR, 1126–1135.
  25. Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. 2019. Meta-learning with latent embedding optimization. In Proceedings of the International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=BJgklhAcK7.
  26. Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas L. Griffiths. 2018. Recasting gradient-based meta-learning as hierarchical Bayes. In Proceedings of the International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=BJ_UL-k0b.
  27. Sachin Ravi and Alex Beatson. 2018. Amortized Bayesian meta-learning. In Proceedings of the International Conference on Learning Representations.
  28. Yoonho Lee and Seungjin Choi. 2018. Gradient-based meta-learning with learned layerwise metric and subspace. In Proceedings of the International Conference on Machine Learning. PMLR, 2927–2936.
  29. Han-Jia Ye and Wei-Lun Chao. 2022. How to train your MAML to excel in few-shot classification. In Proceedings of the 10th International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=49h_IkpJtaE.
  30. Tiago Ramalho and Marta Garnelo. 2019. Adaptive posterior learning: Few-shot learning with a surprise-based memory module. In Proceedings of the 7th International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=ByeSdsC9Km.
  31. Tianqin Li, Zijie Li, Andrew Luo, Harold Rockwell, Amir Barati Farimani, and Tai Sing Lee. 2021. Prototype memory and attention mechanisms for few shot image generation. In Proceedings of the International Conference on Learning Representations.
  32. Ying-Jun Du, Xiantong Zhen, Ling Shao, and Cees G. M. Snoek. 2022. Hierarchical variational memory for few-shot learning across domains. In Proceedings of the 10th International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=i3RI65sR7N.
  33. Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. 2018. Efficient neural architecture search via parameters sharing. In Proceedings of the International Conference on Machine Learning. PMLR, 4095–4104.
  34. Gabriel Bender, Pieter-Jan Kindermans, Barret Zoph, Vijay Vasudevan, and Quoc Le. 2018. Understanding and simplifying one-shot architecture search. In Proceedings of the International Conference on Machine Learning. PMLR, 550–559.
  35. Yiyang Zhao, Linnan Wang, Yuandong Tian, Rodrigo Fonseca, and Tian Guo. 2021. Few-shot neural architecture search. In Proceedings of the International Conference on Machine Learning. PMLR, 12707–12718.
  36. Gregory Koch, Richard Zemel, Ruslan Salakhutdinov, et al. 2015. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop, Vol. 2. Lille.
  37. Kaidi Cao, Maria Brbic, and Jure Leskovec. 2021. Concept learners for few-shot learning. In Proceedings of the 9th International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=eJIJF3-LoZO.
  38. 10.1109/CVPR.2018.00131 · DOI
  39. Victor Garcia Satorras and Joan Bruna Estrach. 2018. Few-shot learning with graph neural networks. In Proceedings of the 6th International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=BJj6qGbRW.
  40. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning. PMLR, 8748–8763.
  41. Haoxiang Wang, Han Zhao, and Bo Li. 2021. Bridging multi-task learning and meta-learning: Towards efficient training and effective adaptation. In Proceedings of the 38th International Conference on Machine Learning. PMLR, 10991–11002. Retrieved from http://proceedings.mlr.press/v139/wang21ad.html.
  42. Mohamed Afham, Salman Khan, Muhammad Haris Khan, Muzammal Naseer, and Fahad Shahbaz Khan. 2021. Rich semantics improve few-shot learning. In Proceedings of the 32nd British Machine Vision Conference. BMVA Press, 152. Retrieved from https://www.bmvc2021-virtualconference.com/assets/papers/0444.pdf.
  43. Imtiaz Ziko, Jose Dolz, Eric Granger, and Ismail Ben Ayed. 2020. Laplacian regularized few-shot learning. In Proceedings of the International Conference on Machine Learning. PMLR, 11660–11670.
  44. Shell Xu Hu, Pablo Garcia Moreno, Yang Xiao, Xi Shen, Guillaume Obozinski, Neil D. Lawrence, and Andreas C. Damianou. 2020. Empirical Bayes transductive meta-learning with synthetic gradients. In Proceedings of the International Conference on Learning Representations. OpenReview.net. Retrieved from https://openreview.net/forum?id=Hkg-xgrYvH.
  45. Jin Xu, Jean-Francois Ton, Hyunjik Kim, Adam Kosiorek, and Yee Whye Teh. 2020. MetaFun: Meta-learning with iterative functional updates. In Proceedings of the International Conference on Machine Learning. PMLR, 10617–10627.
  46. Sung Whan Yoon, Jun Seo, and Jaekyun Moon. 2019. TapNet: Neural network augmented with task-adaptive projection for few-shot learning. In Proceedings of the International Conference on Machine Learning. PMLR, 7115–7123.
  47. Eunbyung Park and Junier B. Oliva. 2019. Meta-curvature. In Proceedings of the Annual Conference on Neural Information Processing Systems. 3309–3319. Retrieved from https://proceedings.neurips.cc/paper/2019/hash/57c0531e13f40b91b3b0f1a30b529a1d-Abstract.html.
  48. Xiaosong Zhang, Feng Liu, Zhiliang Peng, Zonghao Guo, Fang Wan, Xiangyang Ji, and Qixiang Ye. 2022. Integral migrating pre-trained transformer encoder-decoders for visual object detection. arXiv preprint arXiv:2205.09613 (2022). · arXiv
  49. 10.5555/3524938.3525858 · DOI
  50. Gengwei Zhang, Guoliang Kang, Yi Yang, and Yunchao Wei. 2021. Few-shot segmentation via cycle-consistent transformer. In Proceedings of the Annual Conference on Neural Information Processing Systems. 21984–21996. Retrieved from https://proceedings.neurips.cc/paper/2021/hash/b8b12f949378552c21f28deff8ba8eb6-Abstract.html.

Cited by

Sorted by citations

See all 722

Related

See all

Metadata

Authors
Song, Y., Wang, T., Cai, P., Mondal, S. K., & Sahoo, J. P.
Affiliations
  • East China Normal University (CN)
  • Michigan State University (US)
  • Macau University of Science and Technology (MO)
  • Siksha O Anusandhan University (IN)
ORCID
Venue
ACM Computing Surveys
Publisher
Association for Computing Machinery (ACM)
Published
2023-07-13
Volume
55
Issue
13s
Pages
1-40
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
DOI
10.1145/3582688
OpenAlex
W4319163914
Cited by
721
References
219
Citations by year
2026: 190 · 2025: 309 · 2024: 178 · 2023: 43
FWCI
67.87
Type
journal article
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • ACM Computing Surveys (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
National Key R&D Program of China (2021ZD0114600)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
Keywords
few-shot learning, transfer learning, meta-learning, few-shot classification, computer vision applications, research trends, future research directions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review