Object Localization Based on Proposal Fusion
The DOI matches its Crossref record; 58 of the 61 references it cites were found in a registry or an index.
Abstract
Traditional regression framework of object locali-zation such as Overfeat often suffers from the problem of inaccurate scoring due to the separate scoring of classification network and regression network upon inconsistent regions. To tackle this problem, in this paper, we propose a novel object localization framework based on multiple complementary region proposal methods from the view of classification rather than regression. On top of our framework, we first combine multiple complementary region proposals during both training and testing as a means of data augmentation to generate more dense and reliable proposals for fusion, then achieve optimal compromise between complexity and efficiency through category clustering for bounding box sharing among similar categories, and finally propose a dense proposal fusion approach to merge dense region proposals near true object for fine-tuning of the final bounding box's coordinates and updating the confidence of fused proposals for final decision. Extensive experiments on the well-known large scale ILSVRC 2015 LOC dataset verify the effectiveness of our object localization framework.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 17×
“Assuming that an image set consists of c categories (for ILSVRC LOC set, c = 1000), c networks have to be trained, which implies that a huge number of training samples are required.”
BackgroundMethodInfluentialHistograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · cited 6×
“PASCAL VOC [35] which contains 20 categories is usually the first choice for object detection task.”
BackgroundMethodInfluentialSpatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853
Least Squares Support Vector Machine Classifiers
Suykens, J. A. K., & Vandewalle, J. (1999) · Neural Processing Letters · doi:10.1023/a:1018628609742 · Cited by 9,362
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
Selective Search for Object Recognition
Uijlings, J. R. R., van de Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0620-5 · Cited by 5,980
R-FCN: Object Detection via Region-based Fully Convolutional Networks
Dai, J., Li, Y., He, K., & Sun, J. Z. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.06409 · Cited by 3,149
Self-citationSpatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034
Region-based convolutional networks for accurate object detection and segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2437384 · Cited by 2,912
Training Region-Based Object Detectors with Online Hard Example Mining
Shrivastava, A., Gupta, A., & Girshick, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.89 · Cited by 2,686 · cited 1×
“In recent years, due to the high quality of region proposals, most of the state-of-the-art methods [28]–[34] combine region proposal methods and deep CNN to achieve very exciting performances for object detection.”
BackgroundOverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680 · cited 13×
“[16] proposed “Overfeat” approach which uses deep CNN to solve localization problem and gains good performance with a 29.”
BackgroundMethodInfluentialEdge Boxes: Locating Object Proposals from Edges
Zitnick, C. L., & Dollár, P. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_26 · Cited by 2,656
Chameleon: hierarchical clustering using dynamic modeling
Karypis, G., Han, E.-H., & Kumar, V. (1999) · Computer · doi:10.1109/2.781637 · Cited by 2,097
Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks
Bell, S., Zitnick, C. L., Bala, K., & Girshick, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.314 · Cited by 1,292
A Trainable System for Object Detection
Papageorgiou, C., & Poggio, T. (2000) · International Journal of Computer Vision · doi:10.1023/a:1008162616689 · Cited by 1,289 · cited 1×
“[18] and a large variety of manually-designed features such as SIFT [19], BOF [20]–[21], and HOG [22].”
BackgroundMeasuring the Objectness of Image Windows
Alexe, B., Deselaers, T., & Ferrari, V. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.28 · Cited by 1,186
Scalable Object Detection Using Deep Neural Networks
Erhan, D., Szegedy, C., Toshev, A., & Anguelov, D. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.276 · Cited by 1,161 · cited 1×
“CNN proposal methods consist of MultiBox [54], RPN [12], and HyperNet [32].”
MethodMultiscale Combinatorial Grouping
Arbelaez, P., Pont-Tuset, J., Barron, J., Marques, F., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.49 · Cited by 1,155
BING: Binarized Normed Gradients for Objectness Estimation at 300fps
Cheng, M.-M., Zhang, Z., Lin, W.-Y., & Torr, P. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.414 · Cited by 1,050
HyperNet: Towards Accurate Region Proposal Generation and Joint Object Detection
Kong, T., Yao, A., Chen, Y., & Sun, F. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.98 · Cited by 934 · cited 1×
“CNN proposal methods consist of MultiBox [54], RPN [12], and HyperNet [32].”
MethodObject Detection via a Multi-region and Semantic Segmentation-Aware CNN Model
Gidaris, S., & Komodakis, N. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.135 · Cited by 801
Alexe, B., Deselaers, T., & Ferrari, V. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540226 · Cited by 776
Segmentation as selective search for object recognition
van de Sande, K. E. A., Uijlings, J. R. R., Gevers, T., & Smeulders, A. W. M. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126456 · Cited by 739 · cited 1×
“Grouping proposal methods include methods such as Selective Search [25], [48], CPMC [49], and MCG [50], [51].”
MethodWhat Makes for Effective Detection Proposals?
Hosang, J., Benenson, R., Dollar, P., & Schiele, B. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2465908 · Cited by 727 · cited 2×
“avoiding exhaustive sliding window search across images [47].”
BackgroundMultiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation
Pont-Tuset, J., Arbelaez, P., T.Barron, J., Marques, F., & Malik, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2537320 · Cited by 581
Object Detection Networks on Convolutional Feature Maps
Ren, S., He, K., Girshick, R., Zhang, X., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2601099 · Cited by 454
Object Detection Networks on Convolutional Feature Maps
Ren, S., He, K., Girshick, R., Zhang, X., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1504.06066 · Cited by 454
Constrained parametric min-cuts for automatic object segmentation
Carreira, J., & Sminchisescu, C. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540063 · Cited by 420
Robust Object Detection via Soft Cascade
Bourdev, L., & Brandt, J. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.310 · Cited by 319
Combining efficient object localization and image classification
Harzallah, H., Jurie, F., & Schmid, C. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459257 · Cited by 295
A MultiPath Network for Object Detection
Zagoruyko, S., Lerer, A., Lin, T.-Y., Pinheiro, P., Gross, S., Chintala, S., & Dollar, P. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.15 · Cited by 228
Visual Classification by $\ell _1$<alternatives> </alternatives>-Hypergraph Modeling
Wang, M., Liu, X., & Wu, X. (2015) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2015.2415497 · Cited by 172
AttentionNet: Aggregating Weak Directions for Accurate Object Detection
Yoo, D., Park, S., Lee, J.-Y., Paek, A. S., & Kweon, I. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.305 · Cited by 171
On the Design of Cascades of Boosted Ensembles for Face Detection
Brubaker, S. C., Wu, J., Sun, J., Mullin, M. D., & Rehg, J. M. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-007-0060-1 · Cited by 170
Image Caption with Global-Local Attention
Li, L., Tang, S., Deng, L., Zhang, Y., & Tian, Q. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11236 · Cited by 155
Self-citationContextual Bag-of-Words for Visual Categorization
Li, T., Mei, T., Kweon, I.-S., & Hua, X. (2010) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2010.2041828 · Cited by 148
A comparative study of data fusion for RGB-D based visual recognition
Sànchez-Riera, J., Hua, K., Hsiao, Y.-S., Lim, T., Hidayati, S. C., & Cheng, W.-H. (2015) · Pattern Recognition Letters · doi:10.1016/j.patrec.2015.12.006 · Cited by 65
Sparse Ensemble Learning for Concept Detection
Tang, S., Zheng, Y.-T., Wang, Y., & Chua, T. (2011) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2011.2168198 · Cited by 62
Self-citationVideo Annotation Through Search and Graph Reinforcement Mining
Moxley, E., Mei, T., & Manjunath, B. S. (2010) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2010.2041101 · Cited by 62
Learning and Recognition of On-Premise Signs From Weakly Labeled Street View Images
Tsai, T.-H., Cheng, W.-H., You, C.-W., Hu, M., Tsui, A. W., & Chi, H.-Y. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2298982 · Cited by 48
TRECVID 2007 High-Level Feature Extraction By MCG-ICT-CAS *
Tang, S., Zhang, Y., Li, J., Cai, M. L. N., Zhang, X., Tao, K., Tan, L., Xu, S., & Ran, Y.-Y. (2007) · Cited by 43
Self-citationGlobal-residual and Local-boundary Refinement Networks for Rectifying Scene Parsing Predictions
Zhang, R., Tang, S., Lin, M., Li, J., & Yan, S. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/479 · Cited by 33
Self-citationAn efficient concept detection system via sparse ensemble learning
Tang, S., Zhang, Y., Xu, Z.-X., Li, H., Zheng, Y.-T., & Li, J. (2015) · Neurocomputing · doi:10.1016/j.neucom.2014.09.100 · Cited by 12
Self-citationMOSRO: Enabling Mobile Sensing for Real-Scene Objects with Grid Based Structured Output Learning
Chi, H.-Y., Cheng, W.-H., Chen⋆, M.-S., & Tsui, A. W. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-04114-8_18 · Cited by 7
Research works printed but not linked: 1
- lin, 2013, Network in network, CoRR
Cited by
CGNet: A Light-Weight Context Guided Network for Semantic Segmentation
Wu, T., Tang, S., Zhang, R., Cao, J., & Zhang, Y. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3042065 · Cited by 859
Self-citationGLA: Global–Local Attention for Image Description
Li, L., Tang, S., Zhang, Y., Deng, L., & Tian, Q. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2751140 · Cited by 140
“task than fundamental perceptual tasks, such as image classification task [1]–[4], object localization [5] and detection task”
BackgroundSelf-citationMulti-Encoder Towards Effective Anomaly Detection in Videos
Fang, Z., Zhou, J. T., Xiao, Y., Li, Y., & Yang, F. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3037538 · Cited by 66
Self-citationObject-difference drived graph convolutional networks for visual question answering
Zhu, X., Mao, Z., Chen, Z., Li, Y., Wang, Z., & Wang, B. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-08790-0 · Cited by 41
Self-citation3D IoU-Net: IoU Guided 3D Object Detector for Point CloudsPreprint
Li, J., Luo, S., Zhu, Z., Dai, H., Krylov, A. S., Ding, Y. S., & Shao, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.04962 · Cited by 62
“Different from 2D object detection [4]–[6], which only locates the object on the image plane, 3D object detection”
Metadata
- Authors
- Tang, S., Li, Y., Deng, L., & Zhang, Y.
- Affiliations
- Chinese Academy of Sciences (CN)
- Venue
- IEEE Transactions on Multimedia
- Published
- 2017-09
- Volume
- 19
- Issue
- 9
- Pages
- 2105-2116
- ISSN
- 1520-9210 (ISSN-L)
- 1941-0077
- OpenAlex
- W2738890255
- Cited by
- 25
- References
- 61
- Citations by year
- 2025: 1 · 2023: 2 · 2022: 3 · 2021: 4 · 2020: 7 · 2019: 2 · 2018: 4 · 2017: 2
- FWCI
- 1.70
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Multimedia (publishedVersion)
- Funders
- National Natural Science Foundation of China (61525206, 61572472)
- National Key Research and Development Program of China (2016YFB0800403)
- Beijing Natural Science Foundation (4152050)
- Beijing Advanced Innovation Center for Imaging Technology (BAICIT-2016009)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- object localization, bounding box regression, category clustering, confidence fusion, data augmentation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.