An Analysis of Scale Invariance in Object Detection - SNIP
Abstract
An analysis of different techniques for recognizing and detecting objects under extreme scale variation is presented. Scale specific and scale invariant design of detectors are compared by training them with different configurations of input data. By evaluating the performance of different network architectures for classifying small objects on ImageNet, we show that CNNs are not robust to changes in scale. Based on this analysis, we propose to train and test detectors on the same scales of an image-pyramid. Since small and large objects are difficult to recognize at smaller and larger scales respectively, we present a novel training scheme called Scale Normalization for Image Pyramids (SNIP) which selectively back-propagates the gradients of object instances of different sizes as a function of the image scale. On the COCO dataset, our single model performance is 45.7% and an ensemble of 3 networks obtains an mAP of 48.3%. We use off-the-shelf ImageNet-1000 pre-trained models and only train with bounding box supervision. Our submission won the Best Student Entry in the COCO 2017 challenge. Code will be made available at http://bit.ly/2yXVg4c.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Visual elements can vary substantially in scale, unlike word tokens in language Transformers.
“Unlike the word tokens that serve as the basic elements of processing in language Transformers, visual elements can vary substantially in scale, a problem that receives attention in tasks such as object detection [42, 53, 54].”
From Swin Transformer: Hierarchical Vision Transformer using Shifted Windows (2021) · cites this paper as [53]
SNIP builds image pyramids at both training and detection stages and only backpropagates the loss of some selected scales.
“A recent improvement is Scale Normalization for Image Pyramids (SNIP) [176], which builds image pyramids at both training and detection stages and only backpropagates the loss of some selected scales, as shown in Fig. 12.”
From Object Detection in 20 Years: A Survey (2023) · cites this paper as [176]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · see citing works
(2016) · Cited by 31,590 · see citing works
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · see citing works
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · see citing works
A Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004 · see citing works
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137 · see citing works
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157 · see citing works
Scale-space and edge detection using anisotropic diffusion
Perona, P., & Malik, J. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.56205 · Cited by 12,143 · see citing works
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858826 · Cited by 9,556 · see citing works
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916 · see citing works
Deformable Convolutional Networks
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.89 · Cited by 7,513 · see citing works
R-FCN: Object Detection via Region-based Fully Convolutional NetworksPreprint
Dai, J., Li, Y., He, K., & Sun, J. Z. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.06409 · Cited by 5,135 · see citing works
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034 · see citing works
Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors
Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., Guadarrama, S., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.351 · Cited by 2,743 · see citing works
Soft-NMS — Improving Object Detection with One Line of Code
Bodla, N., Singh, B., Chellappa, R., & Davis, L. S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.593 · Cited by 2,274 · cited 6× · see citing works
“D-RFCN obtains the best single model results on COCO and is publicly available, so we use this detector.”
MethodInfluentialSelf-citationScale-Space Theory in Computer Vision
Lindeberg, T. (1994) · Springer eBooks · doi:10.1007/978-1-4757-6465-9 · Cited by 2,185 · see citing works
A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection
Cai, Z., Fan, Q., Feris, R. S., & Vasconcelos, N. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_22 · Cited by 1,605 · see citing works
Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks
Bell, S., Zitnick, C. L., Bala, K., & Girshick, R. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.314 · Cited by 1,344 · see citing works
Scale-space filtering: A new approach to multi-scale description
Witkin, A. (2005) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1984.1172729 · Cited by 1,088 · see citing works
Object Detection via a Multi-region and Semantic Segmentation-Aware CNN Model
Gidaris, S., & Komodakis, N. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.135 · Cited by 801 · see citing works
Learning to Refine Object Segments
Pinheiro, P. O., Lin, T.-Y., Collobert, R., & Dollár, P. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_5 · Cited by 768 · see citing works
Dual Path NetworksPreprint
Chen, Y., Li, J., Xiao, H., Jin, X., Yan, S., & Feng, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.01629 · Cited by 714 · see citing works
Hu, P., & Ramanan, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.166 · Cited by 709 · see citing works
Yang, F., Choi, W., & Lin, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.234 · Cited by 608 · see citing works
SSH: Single Stage Headless Face Detector
Najibi, M., Samangouei, P., Chellappa, R., & Davis, L. S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.522 · Cited by 509 · see citing works
Self-citationObject Detection Networks on Convolutional Feature Maps
Ren, S., He, K., Girshick, R., Zhang, X., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2601099 · Cited by 454 · see citing works
Beyond Skip Connections: Top-Down Modulation for Object DetectionPreprint
Shrivastava, A., Sukthankar, R., Malik, J., & Gupta, A. K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.06851 · Cited by 315 · see citing works
A MultiPath Network for Object Detection
Zagoruyko, S., Lerer, A. K., Lin, T.-Y., Pinheiro, P. O., Gross, S., Chintala, S., & Dollár, P. (2016) · British Machine Vision Conference (BMVC) · doi:10.5244/c.30.15 · Cited by 236 · see citing works
Crafting GBD-Net for Object Detection
Zeng, X., Ouyang, W., Yan, J., Li, H., Xiao, T., Wang, K., Liu, Y., Zhou, Y., Yang, B., Wang, Z., et al. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2745563 · Cited by 142 · see citing works
Scale-aware Fast R-CNN for Pedestrian DetectionPreprint
Li, J., Liang, X., Shen, S., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.08160 · Cited by 114 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · see citing works
“Unlike the word tokens that serve as the basic elements of processing in language Transformers, visual elements can vary substantially in scale, a problem that receives attention in tasks such as object detection [38, 49, 50].”
BackgroundDeep High-Resolution Representation Learning for Visual Recognition
Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y., Liu, D., Mu, Y., Tan, M., Wang, X., et al. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2983686 · Cited by 4,894 · see citing works
Object Detection in 20 Years: A Survey
Zou, Z., Chen, K., Shi, Z., Guo, Y., & Ye, J. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3238524 · Cited by 3,689 · see citing works
“is too large or too small, its gradient will be discarded [39], [176], [177].”
InfluentialCornerNet: Detecting Objects as Paired Keypoints
Law, H., & Deng, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_45 · Cited by 3,852 · see citing works
CenterNet: Keypoint Triplets for Object Detection
Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., & Tian, Q. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00667 · Cited by 4,079 · see citing works
Deep Learning for Generic Object Detection: A Survey
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01247-4 · Cited by 2,965 · see citing works
Object detection in optical remote sensing images: A survey and a new benchmark
Li, K., Wan, G., Cheng, G., Meng, L., & Han, J. (2020) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.11.023 · Cited by 2,317 · see citing works
RepPoints: Point Set Representation for Object Detection
Yang, Z., Liu, S., Hu, H., Wang, L., & Lin, S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00975 · Cited by 1,307 · see citing works
Dai, Y., Gieseke, F., Oehmcke, S., Wu, Y., & Barnard, K. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00360 · Cited by 1,274 · see citing works
Bottom-Up Object Detection by Grouping Extreme and Center Points
Zhou, X., Zhuo, J., & Krahenbuhl, P. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00094 · Cited by 1,088 · see citing works
Recent advances in deep learning for object detection
Wu, X., Sahoo, D., & Hoi, S. C. H. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.01.085 · Cited by 1,047 · see citing works
Objects365: A Large-Scale, High-Quality Dataset for Object Detection
Shao, S., Li, Z., Zhang, T., Peng, C., Yu, G., Zhang, X., Li, J., & Sun, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00852 · Cited by 673 · see citing works
A review of object detection based on deep learning
Xiao, Y., Tian, Z., Yu, J., Zhang, Y., Liu, S., Du, S., & Lan, X. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-08976-6 · Cited by 584 · see citing works
QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection
Yang, C., Huang, Z., & Wang, N. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01330 · Cited by 634 · see citing works
Clustered Object Detection in Aerial Images
Yang, F., Fan, H., Chu, P., Blasch, E., & Ling, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00840 · Cited by 506 · see citing works
“On the contrary, inspired by the observation in [30] that objects with extreme scales may deteriorate the detection performance, we propose a ScaleNet to alleviate this issue, resulting in improvement in fine detection on each chip.”
BackgroundYolo-facev2: A scale and occlusion aware face detector
Yu, Z., Huang, H., Chen, W., Su, Y., Liu, Y., & Wang, X. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110714 · Cited by 476 · see citing works
CBNet: A Novel Composite Backbone Network Architecture for Object Detection
Liu, Y., Wang, Y., Wang, S., Liang, T., Zhao, Q., Tang, Z., & Ling, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6834 · Cited by 280 · see citing works
Deep Learning for Generic Object Detection: A Survey
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2018) · arXiv · doi:10.48550/arxiv.1809.02165 · Cited by 2,965 · see citing works
Disparity-Based Multiscale Fusion Network for Transportation Detection
Chen, J., Wang, Q., Peng, W., Xu, H., Li, X., & Xu, W. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3161977 · Cited by 166 · see citing works
Yan, J., Wang, H., Yan, M., Diao, W., Sun, X., & Li, H. (2019) · Remote Sensing · doi:10.3390/rs11030286 · Cited by 147 · see citing works
Dense and Small Object Detection in UAV Vision Based on Cascade Network
Zhang, X., Izquierdo, E., & Chandramouli, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00020 · Cited by 129 · see citing works
A vehicle detection method based on disparity segmentation
Li, S., Chen, J., Peng, W., Shi, X., & Bu, W. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14360-x · Cited by 102 · see citing works
Spatially Consistent Representation Learning
Roh, B., Shin, W., Kim, I., & Kim, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00120 · Cited by 85 · see citing works
A single-shot multi-level feature reused neural network for object detection
Wei, L., Cui, W., Hu, Z., Sun, H., & Hou, S. (2021) · The Visual Computer · doi:10.1007/s00371-019-01787-3 · Cited by 29 · see citing works
- Authors
- Singh, B., & Davis, L. S.
- Affiliations
- University of Maryland, College Park (US)
- ORCID
- Singh, Bharat
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 3578-3587
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2768489488
- Cited by
- 913
- References
- 40
- Citations by year
- 2026: 43 · 2025: 77 · 2024: 83 · 2023: 130 · 2022: 140 · 2021: 173 · 2020: 134 · 2019: 119 · 2018: 14
- FWCI
- 33.16
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- scale invariance, object detection, image pyramid, small object recognition, convolutional neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
