CoChat
Sign Up
Cited by 952Domain Adaptation and Few-Shot LearningAdvanced Neural Network ApplicationsMachine Learning and Data Classification

Rethinking Few-Shot Image Classification: A Good Embedding is All You Need?

Open access
Tian, Yonglong, Wang, Yue, Krishnan, Dilip, Tenenbaum, Joshua B., Isola, Phillip·2020·Lecture Notes in Computer Science
DOI 10.1007/978-3-030-58568-6_16
Open this paper in CoChat — free accountOpen PDF

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/978-3-030-58568-6_16

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • MAE models pre-trained using the official PyTorch implementation do not exactly match the results of a MAE trained using the original TensorFlow implementation.

    “Furthermore, it has been observed that MAE models pre-trained using the official PyTorch [63] implementation (such as ours) do not exactly match the results of a MAE trained using the original (and unavailable) TensorFlow [1] implementation † .”

    From MultiMAE: Multi-modal Multi-task Masked Autoencoders (2022) · cites this paper as [63]

    The cited paper’s Results on ImageNet derivatives (p. 6) discuss cross-entropy pre-training for few-shot classification, not MAE. Searches for MAE, masked autoencoders, PyTorch, TensorFlow, and implementation comparisons found no passage addressing the claimed difference in results. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 55

  1. [2][2] Kelsey Allen, Evan Shelhamer, Hanul Shin, and Joshua Tenenbaum. Infinite mixture prototypes for few-shot learning. In ICML , 2019. 5 -
  2. [3][3] Luca Bertinetto, Joao F Henriques, Philip HS Torr, and Andrea Vedaldi. Meta-learning with differentiable closed-form solvers. arXiv preprint arXiv:1805.08136 , 2018. 4, 5, 6 - · arXiv
  3. [4][4] Cristian Buciluˇ a, Rich Caruana, and Alexandru NiculescuMizil. Model compression. In SIGKDD , 2006. 2 -
  4. [5][5] Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Wang, and Jia-Bin Huang. A closer look at few-shot classification. In ICLR , 2019. 1 -
  5. [6][6] Yinbo Chen, Xiaolong Wang, Zhuang Liu, Huijuan Xu, and Trevor Darrell. A new meta-baseline for few-shot learning. ArXiv , abs/2003.04390, 2020. 1 - · arXiv
  6. [7][7] Kevin Clark, Minh-Thang Luong, Christopher D. Manning, and Quoc V. Le. Bam! born-again multi-task networks for natural language understanding. In ACL , 2019. 3 -
  7. [9][9] Guneet Singh Dhillon, Pratik Chaudhari, Avinash Ravichandran, and Stefano Soatto. A baseline for few-shot image classification. In ICLR , 2020. 1, 4, 5, 6 -
  8. [10][10] Simon Shaolei Du, Wei Hu, Sham M. Kakade, Jason D. Lee, and Qi Lei. Few-shot learning via learning the representation, provably. ArXiv , abs/2002.09434, 2020. 10 - · arXiv
  9. [11][11] Nikita Dvornik, Cordelia Schmid, and Julien Mairal. Diversity with cooperation: Ensemble methods for few-shot classification. In ICCV , 2019. 5 -
  10. [12][12] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Modelagnostic meta-learning for fast adaptation of deep networks. In ICML , 2017. 1, 2, 5, 6 -
  11. [13][13] Tommaso Furlanello, Zachary Chase Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. Bornagain neural networks. In ICML , 2018. 3, 4, 7 -
  12. [14][14] Spyros Gidaris and Nikos Komodakis. Dynamic few-shot visual learning without forgetting. In CVPR , 2018. 5 -
  13. [15][15] Fusheng Hao, Fengxiang He, Jun Cheng, Lei Wang, Jianzhong Cao, and Dacheng Tao. Collect and select: Semantic alignment metric learning for few-shot learning. In ICCV , 2019. 5 -
  14. [16][16] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick. Momentum contrast for unsupervised visual representation learning. ArXiv , abs/1911.05722, 2019. 6, 13 - · arXiv
  15. [18][18] Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. In NIPS Deep Learning and Representation Learning Workshop , 2015. 2, 3, 4, 10 -
  16. [19][19] Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation networks. In CVPR , 2018. 9 -
  17. [20][20] Shaoli Huang and Dacheng Tao. All you need is a good representation: A multi-level and classifier-centric representation for few-shot learning. ArXiv , abs/1911.12476, 2019. 1 - · arXiv
  18. [21][21] Muhammad Abdullah Jamal and Guo-Jun Qi. Task agnostic meta-learning for few-shot learning. In CVPR , 2019. 5 -
  19. [22][22] Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop , 2015. 2 -
  20. [26][26] Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with differentiable convex optimization. In CVPR , 2019. 1, 2, 3, 4, 5, 6 -
  21. [27][27] Aoxue Li, Tiange Luo, Tao Xiang, Weiran Huang, and Liwei Wang. Few-shot learning with global class representations. In ICCV , 2019. 5 -
  22. [28][28] Hongyang Li, David Eigen, Samuel Dodge, Matthew Zeiler, and Xiaogang Wang. Finding task-relevant features for fewshot learning by category traversal. In CVPR , 2019. 1 -
  23. [29][29] Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. A simple neural attentive meta-learner. arXiv preprint arXiv:1707.03141 , 2017. 4, 5 - · arXiv
  24. [30][30] Hossein Mobahi, Mehrdad Farajtabar, and Peter L Bartlett. Self-distillation amplifies regularization in hilbert space. arXiv preprint arXiv:2002.05715 , 2020. 3 - · arXiv
  25. [31][31] Hossein Mobahi, Mehrdad Farajtabar, and Peter L. Bartlett. Self-distillation amplifies regularization in hilbert space. ArXiv , abs/2002.05715, 2020. 10 - · arXiv
  26. [32][32] Tsendsuren Munkhdalai, Xingdi Yuan, Soroush Mehri, and Adam Trischler. Rapid adaptation with conditionally shifted neurons. arXiv preprint arXiv:1712.09926 , 2017. 5 - · arXiv
  27. [34][34] Boris Oreshkin, Pau Rodr´ ıguez L´ opez, and Alexandre Lacoste. Tadam: Task dependent adaptive metric for improved few-shot learning. In NIPS , 2018. 1, 2, 4, 5, 6 -
  28. [35][35] Zhimao Peng, Zechao Li, Junge Zhang, Yan Li, Guo-Jun Qi, and Jinhui Tang. Few-shot image recognition with knowledge transfer. In ICCV , 2019. 5 -
  29. [36][36] Mary Phuong and Christoph Lampert. Towards understanding knowledge distillation. In ICML , 2019. 10 -
  30. [37][37] Limeng Qiao, Yemin Shi, Jia Li, Yaowei Wang, Tiejun Huang, and Yonghong Tian. Transductive episodic-wise adaptive metric for few-shot learning. In ICCV , 2019. 5, 6 -
  31. [38][38] Siyuan Qiao, Chenxi Liu, Wei Shen, and Alan L. Yuille. Few-shot image recognition by predicting parameters from activations. In CVPR , 2018. 6 -
  32. [39][39] Aniruddh Raghu, Maithra Raghu, Samy Bengio, and Oriol Vinyals. Rapid learning or feature reuse? towards understanding the effectiveness of maml. arXiv preprint arXiv:1909.09157 , 2019. 1, 2, 4 - · arXiv
  33. [40][40] Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. In ICLR , 2017. 2, 5 -
  34. [41][41] Avinash Ravichandran, Rahul Bhotika, and Stefano Soatto. Few-shot learning with embedded class models and shot-free meta training. In ICCV , 2019. 4, 5, 6 -
  35. [42][42] Mengye Ren, Sachin Ravi, Eleni Triantafillou, Jake Snell, Kevin Swersky, Josh B. Tenenbaum, Hugo Larochelle, and Richard S. Zemel. Meta-learning for semi-supervised fewshot classification. In ICLR , 2018. 2, 4, 5 -
  36. [43][43] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. IJCV , 2015. 4 -
  37. [44][44] Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. Meta-learning with latent embedding optimization. In ICLR , 2019. 1, 2, 4, 5, 6 -
  38. [45][45] Tyler Scott, Karl Ridgeway, and Michael C Mozer. Adapted deep embeddings: A synthesis of methods for k-shot inductive transfer learning. In NIPS , 2018. 1 -
  39. [46][46] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In NIPS , 2017. 1, 2, 5, 6 -
  40. [47][47] Qianru Sun, Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele. Meta-transfer learning for few-shot learning. In CVPR , 2019. 5, 6 -
  41. [48][48] Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learning. In CVPR , 2018. 1, 2, 5, 6 -
  42. [49][49] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. arXiv preprint arXiv:1906.05849 , 2019. 6, 13 - · arXiv
  43. [50][50] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive representation distillation. arXiv preprint arXiv:1910.10699 , 2019. 3 - · arXiv
  44. [51][51] Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning? arXiv preprint arXiv:2005.10243 , 2020. 6 - · arXiv
  45. [52][52] Antonio Torralba, Rob Fergus, and William T. Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. TPAMI , 2008. 4 -
  46. [53][53] Eleni Triantafillou, Richard S. Zemel, and Raquel Urtasun. Few-shot learning through an information retrieval lens. In NIPS , 2017. 1 -
  47. [54][54] Eleni Triantafillou, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Utku Evci, Kelvin Xu, Ross Goroshin, Carles Gelada, Kevin Swersky, Pierre-Antoine Manzagol, et al. Meta-dataset: A dataset of datasets for learning to learn from few examples. arXiv preprint arXiv:1903.03096 , 2019. 2, 4, 9, 10 - · arXiv
  48. [55][55] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, koray kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning. In NIPS , 2016. 1, 2, 4, 5 -
  49. [56][56] Yuxiong Wang and Martial Hebert. Learning to learn: Model regression networks for easy small sample learning. In ECCV , 2016. 1 -
  50. [57][57] Yu-Xiong Wang, Ross B. Girshick, Martial Hebert, and Bharath Hariharan. Low-shot learning from imaginary data. CVPR , 2018. 1 -
  51. [58][58] Yu-Xiong Wang and Martial Hebert. Learning from small sample sets by combining unsupervised meta-training with cnns. In Advances in Neural Information Processing Systems 29 , 2016. 6 -
  52. [60][60] Ziyang Wu, Yuwei Li, Lihua Guo, and Kui Jia. Parn: Position-aware relation networks for few-shot learning. In ICCV , 2019. 5 -
  53. [62][62] Han-Jia Ye, Hexiang Hu, De-Chuan Zhan, and Fei Sha. Learning embedding adaptation for few-shot learning. CoRR , abs/1812.03664, 2018. 1 - · arXiv
  54. [63][63] Junho Yim, Donggyu Joo, Jihoon Bae, and Junmo Kim. A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. In CVPR , 2017. 3 -
  55. [64][64] Jian Zhang, Chenglong Zhao, Bingbing Ni, Minghao Xu, and Xiaokang Yang. Variational few-shot learning. In ICCV , 2019. 5

Cited by

Sorted by citations

Authors
Tian, Y., Wang, Y., Krishnan, D., Tenenbaum, J. B., & Isola, P.
Affiliations
  • Massachusetts Institute of Technology (US)
  • Google (United States) (US)
Corresponding author
  • Tian, Yonglong
  • Wang, Yue
ORCID
Venue
Lecture Notes in Computer Science
Publisher
Springer International Publishing
Published
2020
Pages
266-282
ISSN
  • 0302-9743 (ISSN-L)
  • 1611-3349
DOI
10.1007/978-3-030-58568-6_16
OpenAlex
W3108975329
Cited by
952
References
4
Citations by year
2026: 65 · 2025: 126 · 2024: 205 · 2023: 200 · 2022: 165 · 2021: 155 · 2020: 35 · 2019: 1
FWCI
53.00
Type
conference paper
Language
en
Open access
Licence
Locations
  • Lecture notes in computer science (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-sa)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
Keywords
few-shot image classification, meta-learning, self-supervised representation learning, self-distillation, linear classifiers, embedding quality

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review