HyperNet: Towards Accurate Region Proposal Generation and Joint Object Detection
Open accessAbstract
Almost all of the current top-performing object detection networks employ region proposals to guide the search for object instances. State-of-the-art region proposal methods usually need several thousand proposals to get high recall, thus hurting the detection efficiency. Although the latest Region Proposal Network method gets promising detection accuracy with several hundred proposals, it still struggles in small-size object detection and precise localization (e.g., large IoU thresholds), mainly due to the coarseness of its feature maps. In this paper, we present a deep hierarchical network, namely HyperNet, for handling region proposal generation and object detection jointly. Our HyperNet is primarily based on an elaborately designed Hyper Feature which aggregates hierarchical feature maps first and then compresses them into a uniform space. The Hyper Features well incorporate deep but highly semantic, intermediate but really complementary, and shallow but naturally high-resolution features of the image, thus enabling us to construct HyperNet by sharing them both in generating proposals and detecting objects via an end-to-end joint training strategy. For the deep VGG16 model, our method achieves completely leading recall and state-of-the-art object detection accuracy on PASCAL VOC 2007 and 2012 using only 100 proposals per image. It runs with a speed of 5 fps (including all steps) on a GPU, thus having the potential for real-time processing.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The bottle-neck design has been widely used for designing lightweight networks.
“In recent years, the bottle-neck design has been widely used for designing lightweight networks [50, 133, 146-148].”
From Object Detection in 20 Years: A Survey (2023) · cites this paper as [148]
The feature map can be compressed to make it thinner, so as to speed up subsequent detection.
“One can also compress the feature map to make it thinner, so that to speed up subsequent detection [50, 148].”
From Object Detection in 20 Years: A Survey (2023) · cites this paper as [148]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · see citing works
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · see citing works
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · see citing works
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971 · see citing works
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · see citing works
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853 · see citing works
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139 · see citing works
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034 · see citing works
Edge Boxes: Locating Object Proposals from Edges
Zitnick, C. L., & Dollár, P. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_26 · Cited by 2,656 · see citing works
Fast Feature Pyramids for Object Detection
Dollár, P., Appel, R., Belongie, S., & Perona, P. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2300479 · Cited by 1,941 · see citing works
An HOG-LBP human detector with partial occlusion handling
Wang, X., Han, T. X., & Yan, S. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459207 · Cited by 1,727 · see citing works
Hypercolumns for object segmentation and fine-grained localization
Hariharan, B., Arbelaez, P., Girshick, R., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298642 · Cited by 1,511 · see citing works
Scalable Object Detection Using Deep Neural Networks
Erhan, D., Szegedy, C., Toshev, A., & Anguelov, D. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.276 · Cited by 1,161 · cited 1× · see citing works
“In MultiBox [8], region proposals are generated from a CNN model.”
MethodMultiscale Combinatorial Grouping
Arbelaez, P., Pont-Tuset, J., Barron, J., Marques, F., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.49 · Cited by 1,155 · see citing works
BING: Binarized Normed Gradients for Objectness Estimation at 300fps
Cheng, M., Zhang, Z., Lin, W.-Y., & Torr, P. H. S. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.414 · Cited by 1,042 · see citing works
Object Detection via a Multi-region and Semantic Segmentation-Aware CNN Model
Gidaris, S., & Komodakis, N. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.135 · Cited by 801 · see citing works
Alexe, B., Deselaers, T., & Ferrari, V. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540226 · Cited by 778 · see citing works
Segmentation as selective search for object recognition
van de Sande, K. E. A., Uijlings, J. R. R., Gevers, T., & Smeulders, A. W. M. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126456 · Cited by 739 · see citing works
What Makes for Effective Detection Proposals?
Hosang, J., Benenson, R., Dollar, P., & Schiele, B. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2465908 · Cited by 727 · see citing works
Object Detection Networks on Convolutional Feature Maps
Ren, S., He, K., Girshick, R., Zhang, X., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2601099 · Cited by 454 · see citing works
Constrained parametric min-cuts for automatic object segmentation
Carreira, J., & Sminchisescu, C. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540063 · Cited by 424 · see citing works
Zhang, Y., Sohn, K., Villegas, R., Pan, G., & Lee, H. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298621 · Cited by 219 · see citing works
Object Detection Using Strongly-Supervised Deformable Part Models
Azizpour, H., & Laptev, I. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33718-5_60 · Cited by 199 · see citing works
DeepBox: Learning Objectness with Convolutional Networks
Kuo, W., Hariharan, B., & Malik, J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.285 · Cited by 178 · see citing works
Category-Independent Object-Level Saliency Detection
Jia, Y., & Han, M. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.221 · Cited by 134 · see citing works
Occlusion Patterns for Object Class Detection
Pepikj, B., Stark, M. M., Gehler, P., & Schiele, B. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.422 · Cited by 131 · see citing works
DeepProposal: Hunting Objects by Cascading Deep Convolutional Layers
Ghodrati, A., Diba, A., Pedersoli, M., Tuytelaars, T., & Gool, L. V. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.296 · Cited by 124 · see citing works
Online Object Tracking with Proposal Selection
Yang, H., Alahari, K., & Schmid, C. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.354 · Cited by 106 · see citing works
Learning to Combine Mid-Level Cues for Object Proposal Generation
Lee, T., Fidler, S., & Dickinson, S. (2015) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.196 · Cited by 37 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · see citing works
Path Aggregation Network for Instance Segmentation
Liu, S., Qi, L., Qin, H., Shi, J., & Jia, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00913 · Cited by 10,011 · see citing works
Object Detection in 20 Years: A Survey
Zou, Z., Chen, K., Shi, Z., Guo, Y., & Ye, J. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3238524 · Cited by 3,689 · see citing works
“In recent years, the bottle-neck design has been widely used for designing lightweight networks [50], [133], [146], [147], [148].”
Deep Learning for Generic Object Detection: A Survey
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01247-4 · Cited by 2,965 · see citing works
Object detection in optical remote sensing images: A survey and a new benchmark
Li, K., Wan, G., Cheng, G., Meng, L., & Han, J. (2020) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.11.023 · Cited by 2,317 · see citing works
Single-Shot Refinement Neural Network for Object Detection
Zhang, S., Wen, L., Bian, X., Lei, Z., & Li, S. Z. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00442 · Cited by 1,678 · see citing works
An End-to-End Steel Surface Defect Detection Approach via Fusing Multiple Hierarchical Features
He, Y., Song, K., Meng, Q., & Yan, Y. (2020) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2019.2915404 · Cited by 1,341 · see citing works
“But this does not mean HyperNet can work well with ResNet due to the severe dimensionality decline. detection tasks, our system can provide detailed and valuable indicators for quality assessment system, such as the quantity, category, complexity, and area of a defect.”
BackgroundMethodInfluentialDense Contrastive Learning for Self-Supervised Visual Pre-Training
Wang, X., Zhang, R., Shen, C., Kong, T., & Li, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00304 · Cited by 660 · see citing works
Self-citationA review of object detection based on deep learning
Xiao, Y., Tian, Z., Yu, J., Zhang, Y., Liu, S., Du, S., & Lan, X. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-08976-6 · Cited by 584 · see citing works
QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection
Yang, C., Huang, Z., & Wang, N. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01330 · Cited by 634 · see citing works
“These methods can be mainly categorized into four types: 1) increasing the resolution of input features [31, 2, 23, 11, 28, 46, 26, 53]; 2) oversampling and strong data augmentation [31, 21, 75]; 3) incorporating context information [7, 64, 5], and 4) scale-aware training [28, 47, 48, 27].”
MethodSINet: A Scale-Insensitive Convolutional Neural Network for Fast Vehicle Detection
Hu, X., Xu, X., Xiao, Y., Chen, H., He, S., Qin, J., & Heng, P.-A. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2838132 · Cited by 341 · see citing works
What Can Help Pedestrian Detection?
Mao, J., Xiao, T., Jiang, Y., & Cao, Z. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.639 · Cited by 306 · see citing works
Auto-FPN: Automatic Network Architecture Adaptation for Object Detection Beyond Classification
Xu, A., Yao, A., Li, A., Liang, A., & Zhang, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00675 · Cited by 228 · see citing works
Self-citationDeng, Z., Sun, H., Zhou, S., Zhao, J., & Zou, H. (2017) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2017.2694890 · Cited by 195 · see citing works
Ding, P., Zhang, Y., Deng, W.-J., Jia, P., & Kuijper, A. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2018.05.005 · Cited by 174 · see citing works
Object Detection and Classification by Decision-Level Fusion for Intelligent Vehicle Systems
Oh, S.-I., & Kang, H.-B. (2017) · Sensors · doi:10.3390/s17010207 · Cited by 120 · see citing works
Simultaneously Detecting and Counting Dense Vehicles From Drone Images
Li, W., Li, H., Wu, Q., Chen, X., & Ngan, K. N. (2019) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2019.2899548 · Cited by 96 · see citing works
Hsu, C.-Y., Li, W., & Wang, S. (2021) · Remote Sensing · doi:10.3390/rs13112116 · Cited by 53 · see citing works
A single-shot multi-level feature reused neural network for object detection
Wei, L., Cui, W., Hu, Z., Sun, H., & Hou, S. (2021) · The Visual Computer · doi:10.1007/s00371-019-01787-3 · Cited by 29 · see citing works
Object Localization Based on Proposal Fusion
Tang, S., Li, Y., Deng, L., & Zhang, Y. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2729786 · Cited by 25 · see citing works
A Review on Object Detection Based on Deep Convolutional Neural Networks for Autonomous Driving
Lu, J., Tang, S., Wang, J., Zhu, H., & Wang, Y. (2019) · 2019 Chinese Control And Decision Conference (CCDC) · doi:10.1109/ccdc.2019.8832398 · Cited by 17 · see citing works
An Efficient Object Detection Algorithm Based on Compressed Networks
Li, J., Peng, K., & Chang, C.-C. (2018) · Symmetry · doi:10.3390/sym10070235 · Cited by 6 · see citing works
- Authors
- Kong, T., Yao, A., Chen, Y., & Sun, F.
- Affiliations
- Tsinghua University (CN)
- Intel (United States) (US)
- Venue
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2016-06
- Pages
- 845-853
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2962992847
- Cited by
- 934
- References
- 37
- Citations by year
- 2026: 15 · 2025: 32 · 2024: 78 · 2023: 92 · 2022: 85 · 2021: 140 · 2020: 124 · 2019: 151 · 2018: 139 · 2017: 71 · 2016: 7
- FWCI
- 42.58
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (91420302, 61210013, 91520201, 61327809)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- region proposal generation, HyperNet, hierarchical feature aggregation, feature map compression, small object detection, end-to-end learning, region proposal network, VGG16, real-time object detection, Pascal VOC
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
