What Makes for Effective Detection Proposals?
Open accessThe DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Abstract
Current top performing object detectors employ detection proposals to guide the search for objects, thereby avoiding exhaustive sliding window search across images. Despite the popularity and widespread use of detection proposals, it is unclear which trade-offs are made when using them during object detection. We provide an in-depth analysis of twelve proposal methods along with four baselines regarding proposal repeatability, ground truth annotation recall on PASCAL, ImageNet, and MS COCO, and their impact on DPM, R-CNN, and Fast R-CNN detection performance. Our analysis shows that for object detection improving proposal localisation accuracy is as important as improving recall. We introduce a novel metric, the average recall (AR), which rewards both high recall and good localisation and correlates surprisingly well with detection performance. Our findings show common strengths and weaknesses of existing methods, and provide insights and metrics for selecting and tuning proposal methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It is noteworthy that the Recall-to-IoU metric is just loosely [19], [20], [21] related to the ultimate detection accuracy.”
From Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2017) · cites this paper as [20]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In the case of sliding window based detectors, the imbalance between backgrounds and objects could be as extreme as 10 7 :1 [71].”
From Object Detection in 20 Years: A Survey (2023) · cites this paper as [71]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139
Efficient Graph-Based Image Segmentation
Felzenszwalb, P. F., & Huttenlocher, D. P. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000022288.19776.77 · Cited by 6,176
Selective Search for Object Recognition
Uijlings, J. R. R., van de Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0620-5 · Cited by 5,980
Contour Detection and Hierarchical Image Segmentation
Arbeláez, P., Maire, M., Fowlkes, C., & Malik, J. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.161 · Cited by 5,454
A comparison of affine region detectors
Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., & Gool, L. V. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-3848-x · Cited by 2,939
Edge Boxes: Locating Object Proposals from Edges
Zitnick, C. L., & Dollár, P. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_26 · Cited by 2,656
Self-citationMicrosoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C. L., & Dollár, P. (2014) · arXiv · doi:10.48550/arxiv.1405.0312 · Cited by 2,579
Self-citationA Trainable System for Object Detection
Papageorgiou, C., & Poggio, T. (2000) · International Journal of Computer Vision · doi:10.1023/a:1008162616689 · Cited by 1,289
Local Invariant Feature Detectors: A Survey
Tuytelaars, T., & Mikolajczyk, K. (2007) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000017 · Cited by 1,186
Measuring the Objectness of Image Windows
Alexe, B., Deselaers, T., & Ferrari, V. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.28 · Cited by 1,186
Scalable Object Detection Using Deep Neural Networks
Erhan, D., Szegedy, C., Toshev, A., & Anguelov, D. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.276 · Cited by 1,161 · cited 1×
“The simplest such approach would be to directly use the output of any hierarchical image segmentation algorithm, e.g. Gu et al. [34] use the segmentation produced by gPb [35].”
MethodMultiscale Combinatorial Grouping
Arbelaez, P., Pont-Tuset, J., Barron, J., Marques, F., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.49 · Cited by 1,155
BING: Binarized Normed Gradients for Objectness Estimation at 300fps
Cheng, M.-M., Zhang, Z., Lin, W.-Y., & Torr, P. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.414 · Cited by 1,050
Fast Edge Detection Using Structured Forests
Dollar, P., & Zitnick, C. L. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2377715 · Cited by 962
Structured Forests for Fast Edge Detection
Dollar, P., & Zitnick, C. L. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.231 · Cited by 899
Alexe, B., Deselaers, T., & Ferrari, V. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540226 · Cited by 776
Segmentation as selective search for object recognition
van de Sande, K. E. A., Uijlings, J. R. R., Gevers, T., & Smeulders, A. W. M. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126456 · Cited by 739 · cited 3×
“While in this work we focus on the impact of proposals on detection, proposals have applications beyond object detection, as we discuss in §VI.”
BackgroundSharing Visual Features for Multiclass and Multiview Object Detection
Torralba, A., Murphy, K. P., & Freeman, W. T. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1055 · Cited by 700
Learning to Segment Object Candidates
Pinheiro, P. O., Collobert, R., & Dollár, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06204 · Cited by 648
CPMC: Automatic Object Segmentation Using Constrained Parametric Min-Cuts
Carreira, J., & Sminchisescu, C. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.231 · Cited by 592
Diagnosing Error in Object Detectors
Hoiem, D., Chodpathumwan, Y., & Dai, Q. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33712-3_25 · Cited by 520
SEEDS: Superpixels Extracted via Energy-Driven Sampling
Van den Bergh, M., Boix, X., Roig, G., de Capitani, B., & Van Gool, L. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33786-4_2 · Cited by 473
Category Independent Object Proposals
Endres, I., & Hoiem, D. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15555-0_42 · Cited by 432
Sketch Tokens: A Learned Mid-level Representation for Contour and Object Detection
Lim, J. J., Zitnick, C. L., & Dollar, P. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.406 · Cited by 427
Constrained parametric min-cuts for automatic object segmentation
Carreira, J., & Sminchisescu, C. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540063 · Cited by 420
Gu, C., Lim, J. J., Arbelaez, P., & Malik, J. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206727 · Cited by 347
Fusing generic objectness and visual saliency for salient object detection
Chang, K.-Y., Liu, T.-L., Chen, H.-T., & Lai, S.-H. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126333 · Cited by 345
Krähenbühl, P., & Koltun, V. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_47 · Cited by 336
Scalable, High-Quality Object Detection
Szegedy, C., Reed, S., Erhan, D., Anguelov, D., & Ioffe, S. (2014) · arXiv · doi:10.48550/arxiv.1412.1441 · Cited by 333
Robust Object Detection via Soft Cascade
Bourdev, L., & Brandt, J. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.310 · Cited by 319
Regionlets for Generic Object Detection
Wang, X., Yang, M., Zhu, S., & Lin, Y. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.10 · Cited by 316
Discriminatively Trained Sparse Code Gradients for Contour Detection
Ren, X., & Bo, L. (2012) · neural information processing systems · Cited by 296
Combining efficient object localization and image classification
Harzallah, H., Jurie, F., & Schmid, C. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459257 · Cited by 295
Vicente, S., Rother, C., & Kolmogorov, V. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995530 · Cited by 292
Improving Spatial Support for Objects via Multiple Segmentations
Malisiewicz, T., & Efros, A. A. (2007) · Procedings of the British Machine Vision Conference 2007 · doi:10.5244/c.21.55 · Cited by 280
Prime Object Proposals with Randomized Prim's Algorithm
Manen, S., Guillaumin, M., & Gool, L. V. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.315 · Cited by 278
Girshick, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.08083 · Cited by 276
SEEDS: Superpixels Extracted Via Energy-Driven Sampling
Van den Bergh, M., Boix, X., Roig, G., & Van Gool, L. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0744-2 · Cited by 253
Crosstalk Cascades for Frame-Rate Pedestrian Detection
Dollár, P., Appel, R., & Kienzle, W. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33709-3_46 · Cited by 222
Category-Independent Object Proposals with Diverse Ranking
Endres, I., & Hoiem, D. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.122 · Cited by 221
How good are detection proposals, really?
Hosang, J., Benenson, R., & Schiele, B. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.24 · Cited by 215
Self-citationLearning to segment moving objects in videos
Fragkiadaki, K., Arbelaez, P., Felsen, P., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299035 · Cited by 193
Co-localization in Real-World Images
Tang, K., Joulin, A., Li, L.-J., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.190 · Cited by 184
DeepBox: Learning Objectness with Convolutional Networks
Kuo, W., Hariharan, B., & Malik, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.285 · Cited by 176
ImageNet Auto-Annotation with Segmentation Propagation
Guillaumin, M., Kuettel, D., & Ferrari, V. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-014-0713-9 · Cited by 172
Learning a category independent object detection cascade
Rahtu, E., Kannala, J., & Blaschko, M. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126351 · Cited by 163
Generating Object Segmentation Proposals Using Global and Local Search
Rantalankila, P., Kannala, J., & Rahtu, E. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.310 · Cited by 159
Salient object detection by composition
Feng, J., Wei, Y., Tao, L., Zhang, C., & Sun, J. Z. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126348 · Cited by 148
Segmentation Driven Object Detection with Fisher Vectors
Cinbiş, R. G., Verbeek, J. J., & Schmid, C. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.369 · Cited by 133
Tuytelaars, T. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539911 · Cited by 122
Proposal generation for object detection using cascaded ranking SVMs
Zhang, Z., Warrell, J., & Torr, P. H. S. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995411 · Cited by 107
Online Video SEEDS for Temporal Window Objectness
den Bergh, M. V., Roig, G., Boix, X., Manén, S., & Gool, L. V. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.54 · Cited by 76
Shape Sharing for Object Segmentation
Kim, J., & Grauman, K. L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33786-4_33 · Cited by 75
Object-Proposal Evaluation Protocol is ‘Gameable’
Chavali, N., Agrawal, H., Mahendru, A., & Batra, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.97 · Cited by 65
Zhao, Q., & Liu, Z. (2014) · British Machine Vision Conference (BMVC) · doi:10.5244/c.28.23 · Cited by 35
Training Deformable Part Models with Decorrelated Features
Girshick, R., & Malik, J. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.375 · Cited by 34
DeepBox: Learning Objectness with Convolutional NetworksPreprint
Kuo, W., Hariharan, B., & Malik, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.02146 · Cited by 30
Fast Edge Detection Using Structured Forests
Neelsidhi Atlantis, P., & Nair, B. (2016) · INTERNATIONAL JOURNAL OF EMERGING TRENDS IN SCIENCE AND TECHNOLOGY · doi:10.18535/ijetst/v3i08.06 · Cited by 29
Non Maximal Suppression in Cascaded Ranking Models
Blaschko, M. B., Kannala, J., & Rahtu, E. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-38886-6_39 · Cited by 18
An Efficient Shared Multi-Class Detection Cascade
Zehnder, P., Koller-Meier, E., & Gool, L. V. (2008) · British Machine Vision Conference (BMVC) · doi:10.5244/c.22.80 · Cited by 15
Research works printed but not linked: 1
- humayun, 0, RIGOR: Recycling inference in graph cuts for generating object regions, Proc IEEE Conf Comput Vis Pattern Recog, 336
Cited by
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
“In the following, we train Fast R-CNN using 2k RPN proposals, but evaluate different numbers of proposals at test-time.”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
“Many of the commonly used bounding box proposal methods [28, 54] first generate a set of overlapping segments, e.”
MethodSelf-citation
Metadata
- Authors
- Hosang, J., Benenson, R., Dollar, P., & Schiele, B.
- Affiliations
- Max Planck Institute for Informatics (DE)
- ORCID
- Schiele, Bernt
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2016-04-01
- Volume
- 38
- Issue
- 4
- Pages
- 814-830
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W1958328135
- Cited by
- 727
- References
- 68
- Citations by year
- 2026: 9 · 2025: 27 · 2024: 34 · 2023: 36 · 2022: 57 · 2021: 58 · 2020: 63 · 2019: 95 · 2018: 93 · 2017: 126 · 2016: 98
- FWCI
- 100.45
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- object detection, PASCAL dataset, ImageNet, MS COCO, DPM, R-CNN, Fast R-CNN, sliding window search
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.