Cited by 55Domain Adaptation and Few-Shot LearningAdvanced Neural Network ApplicationsAdvanced Image and Video Retrieval Techniques

Incremental Few-Shot Object Detection

Open access
Perez-Rua, Juan-Manuel, Zhu, Xiatian, Hospedales, Timothy M., Xiang, Tao·2020·2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Abstract

Existing object detection methods typically rely on the availability of abundant labelled training samples per class and offline model training in a batch mode. These requirements substantially limit their scalability to open-ended accommodation of novel classes with limited labelled training data, both in terms of model accuracy and training efficiency during deployment. We present the first study aiming to go beyond these limitations by considering the Incremental Few-Shot Detection (iFSD) problem setting, where new classes must be registered incrementally (without revisiting base classes) and with few examples. To this end we propose OpeN-ended Centre nEt (ONCE), a detector designed for incrementally learning to detect novel class objects with few examples. This is achieved by an elegant adaptation of the efficient CentreNet detector to the few-shot learning scenario, and meta-learning a class-wise code generator model for registering novel classes. ONCE fully respects the incremental learning paradigm, with novel class registration requiring only a single forward pass of few-shot training samples, and no access to base classes - thus making it suitable for deployment on embedded devices, etc. Extensive experiments conducted on both the standard object detection (COCO, PASCAL VOC) and fashion landmark detection (DeepFashion2) tasks show the feasibility of iFSD for the first time, opening an interesting and very important line of research.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr42600.2020.01386

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • For FSCIOD tasks, the data partitioning strategy by Perez-Rua et al. utilizes 20 classes overlapping with PASCAL VOC as new incremental classes and the remaining 60 as base data, with models evaluated using K ∈ {1,5,10} bounding boxes per new class.

    “For FSCIOD tasks, the data partitioning strategy by PerezRua et al. [62] is commonly used. This approach utilizes 20 classes overlapping with the PASCAL VOC dataset [63] as new incremental classes and the remaining 60 as base data. FSCIOD models under this setup are evaluated using K ∈ 1 , 5 , 10 bounding boxes per new class.”
  • Anchor-free frameworks can effectively handle incremental classes without defining anchor boxes.

    “The reason is that anchor-free frameworks can effectively handle incremental classes without defining anchor boxes.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 50

  1. [1][1] Saif Alabachi, Gita Sukthankar, and Rahul Sukthankar. Customizing object detectors for indoor robots. In ICRA , 2019. 1 -
  2. [2][2] Ankan Bansal, Karan Sikka, Gaurav Sharma, Rama Chellappa, and Ajay Divakaran. Zero-shot object detection. In ECCV , 2018. 3 -
  3. [3][3] I. Biederman. Recognition-by-components: A theory of human image understanding. Psychological Review , 92(2):115-147, 1987. 1 -
  4. [4][4] Hakan Bilen and Andrea Vedaldi. Weakly supervised deep detection networks. In CVPR , 2016. 3 -
  5. [5][5] Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. A closer look at few-shot classification. In ICLR , 2019. 2 -
  6. [6][6] Zitian Chen, Yanwei Fu, Yu-Xiong Wang, Lin Ma, Wei Liu, and Martial Hebert. Image deformation meta-networks for one-shot learning. In CVPR , 2019. 2, 7 -
  7. [7][7] Wen-Hsuan Chu, Yu-Jhe Li, Jing-Cheng Chang, and YuChiang Frank Wang. Spot and learn: A maximum-entropy patch sampler for few-shot image classification. In CVPR , 2019. 2, 7 -
  8. [8][8] Jifeng Dai, Yi Li, Kaiming He, and Jian Sun. R-fcn: Object detection via region-based fully convolutional networks. In NeurIPS , 2016. 2 -
  9. [9][9] Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. Continual learning: A comparative study on how to defy forgetting in classification tasks. arXiv preprint arXiv:1909.08383 , 2019. 1 - · arXiv
  10. [10][10] Ali Diba, Vivek Sharma, Ali Pazandeh, Hamed Pirsiavash, and Luc Van Gool. Weakly supervised cascaded convolutional networks. In CVPR , 2017. 3 -
  11. [11][11] Nikita Dvornik, Cordelia Schmid, and Julien Mairal. Diversity with cooperation: Ensemble methods for few-shot classification. In ICCV , 2019. 2, 7 -
  12. [12][12] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. IJCV , 88(2):303-338, 2010. 2, 6 -
  13. [13][13] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Modelagnostic meta-learning for fast adaptation of deep networks. In ICML , 2017. 2, 6, 7 -
  14. [14][14] Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences , 3(4):128-135, 1999. 1, 2 -
  15. [15][15] Yuying Ge, Ruimao Zhang, Xiaogang Wang, Xiaoou Tang, and Ping Luo. Deepfashion2: A versatile benchmark for detection, pose estimation, segmentation and re-identification of clothing images. In CVPR , 2019. 2, 8 -
  16. [16][16] Spyros Gidaris, Andrei Bursuc, Nikos Komodakis, Patrick Perez, and Matthieu Cord. Boosting few-shot visual learning with self-supervision. In ICCV , 2019. 2, 7 -
  17. [17][17] Spyros Gidaris and Nikos Komodakis. Generating classification weights with gnn denoising autoencoders for few-shot learning. In CVPR , 2019. 2, 7 -
  18. [18][18] Ross Girshick. Fast r-cnn. In ICCV , 2015. 2 -
  19. [19][19] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR , 2014. 2 -
  20. [20][20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. TPAMI , 37(9):1904-1916, 2015. 2 -
  21. [22][22] Bingyi Kang, Zhuang Liu, Xin Wang, Fisher Yu, Jiashi Feng, and Trevor Darrell. Few-shot object detection via feature reweighting. In ICCV , 2019. 1, 2, 4, 6, 7, 8 -
  22. [23][23] Leonid Karlinsky, Joseph Shtok, Sivan Harary, Eli Schwartz, Amit Aides, Rogerio Feris, Raja Giryes, and Alex M. Bronstein. Repmet: Representative-based metric learning for classification and few-shot object detection. In CVPR , 2019. 2 -
  23. [25][25] Hei Law and Jia Deng. Cornernet: Detecting objects as paired keypoints. In ECCV , 2018. 2, 3 -
  24. [26][26] Aoxue Li, Tiange Luo, Zhiwu Lu, Tao Xiang, and Liwei Wang. Large-scale few-shot learning: Knowledge transfer with class hierarchy. In CVPR , 2019. 2, 7 -
  25. [27][27] Aoxue Li, Tiange Luo, Tao Xiang, Weiran Huang, and Liwei Wang. Few-shot learning with global class representations. In ICCV , 2019. 2, 7 -
  26. [28][28] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll´ ar. Focal loss for dense object detection. In ICCV , 2017. 1, 2, 3 -
  27. [29][29] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ ar, and C Lawrence Zitnick. Microsoft COCO: Common objects in context. In ECCV . Springer, 2014. 2, 6 -
  28. [30][30] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In ECCV , 2016. 1, 2, 3 -
  29. [31][31] Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X. Yu. Large-scale long-tailed recognition in an open world. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2019. 1 -
  30. [34][34] Alex Nichol, Joshua Achiam, and John Schulman. On firstorder meta-learning algorithms. arXiv , 2018. 2, 6, 8 -
  31. [35][35] Andreas Opelt, Axel Pinz, and Andrew Zisserman. Incremental learning of object detectors using a visual shape alphabet. In CVPR , 2006. 3 -
  32. [36][36] Steven Pinker. How the mind works . Penguin UK, 2003. 1 -
  33. [38][38] Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. In ICLR , 2016. 2 -
  34. [39][39] Avinash Ravichandran, Rahul Bhotika, and Stefano Soatto. Few-shot learning with embedded class models and shot-free meta training. In ICCV , 2019. 2, 7 -
  35. [42][42] Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv , 2018. 1, 2, 3 -
  36. [43][43] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NeurIPS , 2015. 1, 2, 3 -
  37. [44][44] Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. One-shot learning with memory-augmented neural networks. In ICML , 2017. 2, 4 -
  38. [45][45] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR , 2015. 1 -
  39. [46][46] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In NeurIPS , 2017. 2 -
  40. [47][47] Qianru Sun, Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele. Meta-transfer learning for few-shot learning. In CVPR , 2019. 2, 7 -
  41. [48][48] Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learning. In CVPR , 2018. 2, 4 -
  42. [49][49] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In CVPR , 2015. 1 -
  43. [50][50] Pavel Tokmakov, Yu-Xiong Wang, and Martial Hebert. Learning compositional representations for few-shot recognition. In ICCV , 2019. 2, 7 -
  44. [51][51] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In NeurIPS , 2016. 2, 4, 5 -
  45. [52][52] Bin Xiao, Haiping Wu, and Yichen Wei. Simple baselines for human pose estimation and tracking. In ECCV , 2018. 3, 5, 6 -
  46. [53][53] Xiaopeng Yan, Ziliang Chen, Anni Xu, Xiaoxi Wang, Xiaodan Liang, and Liang Lin. Meta r-cnn: Towards general solver for instance-level low-shot learning. In ICCV , 2019. 1, 2, 4, 6, 7, 8 -
  47. [54][54] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan R Salakhutdinov, and Alexander J Smola. Deep sets. In NeurIPS , 2017. 6 -
  48. [55][55] Hongguang Zhang, Jing Zhang, and Piotr Koniusz. Fewshot learning via saliency-guided hallucination of samples. In CVPR , 2019. 2, 7 -
  49. [56][56] Xingyi Zhou, Dequan Wang, and Philipp Kr¨ ahenb¨ uhl. Objects as points. In arXiv , 2019. 2, 3, 5, 7 -
  50. [58][58] Pengkai Zhu, Hanxiao Wang, and Venkatesh Saligrama. Zero shot detection. IEEE TCSVT , 2019. 3

Cited by

Sorted by citations

Authors
Perez-Rua, J.-M., Zhu, X., Hospedales, T. M., & Xiang, T.
Affiliations
  • University of Edinburgh (GB)
  • University of Surrey (GB)
Venue
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2020-06
Pages
13843-13852
ISSN
1063-6919 (ISSN-L)
OpenAlex
W3011420752
Cited by
55
References
6
Citations by year
2025: 6 · 2024: 12 · 2023: 6 · 2022: 18 · 2021: 11 · 2020: 2
FWCI
3.53
Type
conference paper
Language
en
Open access
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
Keywords
incremental learning, few-shot object detection, meta-learning, embedded deployment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review