Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Open accessAbstract
State-of-the-art object detection networks depend on region proposal algorithms to hypothesize object locations. Advances like SPPnet and Fast R-CNN have reduced the running time of these detection networks, exposing region proposal computation as a bottleneck. In this work, we introduce a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals. An RPN is a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position. The RPN is trained end-to-end to generate high-quality region proposals, which are used by Fast R-CNN for detection. We further merge RPN and Fast R-CNN into a single network by sharing their convolutional features---using the recently popular terminology of neural networks with 'attention' mechanisms, the RPN component tells the unified network where to look. For the very deep VGG-16 model, our detection system has a frame rate of 5fps (including all steps) on a GPU, while achieving state-of-the-art object detection accuracy on PASCAL VOC 2007, 2012, and MS COCO datasets with only 300 proposals per image. In ILSVRC and COCO 2015 competitions, Faster R-CNN and RPN are the foundations of the 1st-place winning entries in several tracks. Code has been made publicly available.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Existing mask proposal methods are based on densely sampled image pyramids, making them computationally expensive.
“Existing mask proposal methods [27, 28, 4] are based on densely sampled image pyramids ( e.g ., scaled by 2 {-2:0 . 5:1 } in [27, 28]), making them computationally expensive.”
From Feature Pyramid Networks for Object Detection (2017) · cites this paper as [27]
The Introduction says prevalent object-detection methods use image or filter pyramids, and §3.3 says avoiding those pyramids saves running time. The paper does not establish that existing mask proposal methods rely on densely sampled image pyramids or document the specified scale sampling. (LLM, conf=0.86)
Experimental — accuracy not yet measuredSharing features increases train time by 1.5× (using 4-step training [29]).
“Sharing features increases train time by 1.5 × (using 4-step training [29]), but reduces test time.”
From Feature Pyramid Networks for Object Detection (2017) · cites this paper as [29]
The “4-Step Alternating Training” section describes using four steps to learn shared features. However, our copy does not report that sharing features increases training time by 1.5×; Section 3.2 instead compares the training time of approximate joint training with alternating training, a different comparison. No page numbers are available in our copy. (LLM, conf=0.84)
Experimental — accuracy not yet measuredOn COCO, DETR needs 500 epochs to converge, around 10 to 20 times slower than Faster R-CNN.
“on the COCO(Lin et al., 2014) benchmark, DETR needs 500 epochs to converge, which is around 10 to 20 times slower than Faster R-CNN (Ren et al., 2015).”
From Deformable DETR: Deformable Transformers for End-to-End Object Detection (2020) · cites this paper as Ren et al. (2015)
Section 4.2 (Experiments on MS COCO; page unavailable) reports Faster R-CNN’s COCO training schedule, providing a basis for the comparator. It does not report DETR’s 500 epochs or establish that DETR converges 10–20 times more slowly. (LLM, conf=0.84)
Experimental — accuracy not yet measuredPrevious long-term tracking methods usually consist of multiple components, such as base trackers, target verification modules, and global detectors.
“Specifically, previous methods usually consist of multiple components, such as base trackers [9, 50], target verification modules [21], and global detectors [40, 19].”
From Learning Spatio-Temporal Transformer for Visual Tracking (2021) · cites this paper as [40]
We outperform the accuracy of DeepMask, SharpMask, and InstanceFCN by over 8.3 points AR.
“We outperform the accuracy of these approaches by over 8.3 points AR.”
From Feature Pyramid Networks for Object Detection (2017) · cites this paper as [27]
The Analysis of Recall-to-IoU section compares RPN proposals with Selective Search and EdgeBoxes, not DeepMask, SharpMask, or InstanceFCN. The passages found for those names and for average recall (AR) do not report an 8.3-point advantage; our copy does not address the claimed comparison. (LLM, conf=0.76)
Experimental — accuracy not yet measured
For 'fully convolutional' models such as GoogleNets and ResNets, there are no hidden fc layers for building regionwise classifiers.
“However, for 'fully convolutional' models such as GoogleNets [18] and ResNets [17], there are no hidden fc layers for building regionwise classifiers.”
From Object Detection Networks on Convolutional Feature Maps (2017) · cites this paper as [17]
Combining with orthogonal improvements, the results in [17] secured the 1st place in MS COCO and ImageNet 2015 challenges.
“Combining with orthogonal improvements, the results in [17] secured the 1st place in MS COCO and ImageNet 2015 challenges.”
From Object Detection Networks on Convolutional Feature Maps (2017) · cites this paper as [17]
The 101-layer ResNet is released by the authors of [17], with center crop top-1 error of 23.6%.
“The 101-layer ResNet is released by the authors of [17], with center crop top-1 error of 23.6%.”
From Object Detection Networks on Convolutional Feature Maps (2017) · cites this paper as [17]
The above system (27.2% AP and 48.4% AP@0.5) is the foundation of the detection system in the ResNet paper.
“The above system (27.2% AP and 48.4% AP@0.5) is the foundation of the detection system in the ResNet paper [17].”
From Object Detection Networks on Convolutional Feature Maps (2017) · cites this paper as [17]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 45 · Cited by 70,812
Research works printed but not linked: 36
- K. He, X. Zhang, S. Ren, and J. Sun, "Spatial pyramid pooling in deep convolutional networks for visual recognition," in European Conference on Computer Vision (ECCV), 2014.
- R. Girshick, "Fast R-CNN," in IEEE International Conference on Computer Vision (ICCV), 2015.
- K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in International Conference on Learning Representations (ICLR), 2015.
- J. R. Uijlings, K. E. van de Sande, T. Gevers, and A. W. Smeul- ders, "Selective search for object recognition," International Journal of Computer Vision (IJCV), 2013.
- R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic seg- mentation," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
- C. L. Zitnick and P. Dollár, "Edge boxes: Locating object proposals from edges," in European Conference on Computer Vision (ECCV), 2014.
- J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
- P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ra- manan, "Object detection with discriminatively trained part- based models," IEEE Transactions on Pattern Analysis and Ma- chine Intelligence (TPAMI), 2010.
- P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, "Overfeat: Integrated recognition, localization and detection using convolutional networks," in International Conference on Learning Representations (ICLR), 2014.
- S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards real-time object detection with region proposal networks," in Neural Information Processing Systems (NIPS), 2015.
- T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ra- manan, P. Dollár, and C. L. Zitnick, "Microsoft COCO: Com- mon Objects in Context," in European Conference on Computer Vision (ECCV), 2014.
- S. Song and J. Xiao, "Deep sliding shapes for amodal 3d object detection in rgb-d images," arXiv:1511.02300, 2015. · arXiv
- J. Zhu, X. Chen, and A. L. Yuille, "DeePM: A deep part-based model for object detection and semantic part localization," arXiv:1511.07131, 2015. · arXiv
- J. Dai, K. He, and J. Sun, "Instance-aware semantic segmenta- tion via multi-task network cascades," arXiv:1512.04412, 2015. · arXiv
- J. Johnson, A. Karpathy, and L. Fei-Fei, "Densecap: Fully convolutional localization networks for dense captioning," arXiv:1511.07571, 2015. · arXiv
- D. Kislyuk, Y. Liu, D. Liu, E. Tzeng, and Y. Jing, "Human cu- ration and convnets: Powering item-to-item recommendations on pinterest," arXiv:1511.04003, 2015. · arXiv
- K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," arXiv:1512.03385, 2015. · arXiv
- J. Hosang, R. Benenson, and B. Schiele, "How good are de- tection proposals, really?" in British Machine Vision Conference (BMVC), 2014.
- J. Hosang, R. Benenson, P. Dollár, and B. Schiele, "What makes for effective detection proposals?" IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2015.
- N. Chavali, H. Agrawal, A. Mahendru, and D. Batra, "Object-Proposal Evaluation Protocol is 'Gameable'," arXiv: 1505.05836, 2015. · arXiv
- J. Carreira and C. Sminchisescu, "CPMC: Automatic ob- ject segmentation using constrained parametric min-cuts," IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2012.
- P. Arbeláez, J. Pont-Tuset, J. T. Barron, F. Marques, and J. Malik, "Multiscale combinatorial grouping," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
- B. Alexe, T. Deselaers, and V. Ferrari, "Measuring the object- ness of image windows," IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2012.
- C. Szegedy, A. Toshev, and D. Erhan, "Deep neural networks for object detection," in Neural Information Processing Systems (NIPS), 2013.
- D. Erhan, C. Szegedy, A. Toshev, and D. Anguelov, "Scalable object detection using deep neural networks," in IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2014.
- C. Szegedy, S. Reed, D. Erhan, and D. Anguelov, "Scalable, high-quality object detection," arXiv:1412.1441 (v1), 2015. · arXiv
- P. O. Pinheiro, R. Collobert, and P. Dollar, "Learning to segment object candidates," in Neural Information Processing Systems (NIPS), 2015.
- J. Dai, K. He, and J. Sun, "Convolutional feature masking for joint object and stuff segmentation," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
- S. Ren, K. He, R. Girshick, X. Zhang, and J. Sun, "Ob- ject detection networks on convolutional feature maps," arXiv:1504.06066, 2015. · arXiv
- J. K. Chorowski, D. Bahdanau, D. Serdyuk, K. Cho, and Y. Bengio, "Attention-based models for speech recognition," in Neural Information Processing Systems (NIPS), 2015.
- M. D. Zeiler and R. Fergus, "Visualizing and understanding convolutional neural networks," in European Conference on Computer Vision (ECCV), 2014.
- V. Nair and G. E. Hinton, "Rectified linear units improve restricted boltzmann machines," in International Conference on Machine Learning (ICML), 2010.
- C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, and A. Rabinovich, "Going deeper with convo- lutions," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
- O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, "ImageNet Large Scale Visual Recognition Challenge," in International Journal of Computer Vision (IJCV), 2015.
- Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Gir- shick, S. Guadarrama, and T. Darrell, "Caffe: Convolutional architecture for fast feature embedding," arXiv:1408.5093, 2014. · arXiv
- K. Lenc and A. Vedaldi, "R-CNN minus R," in British Machine Vision Conference (BMVC), 2015.
- Authors
- Ren, S., He, K., Girshick, R., & Sun, J.
- Affiliations
- University of Science and Technology of China (CN)
- Meta (United States) (US)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- Volume
- 39
- Issue
- 6
- Pages
- 1137-1149
- ISSN
- 0162-8828 (ISSN-L)
- arXiv
- 1506.01497
- OpenAlex
- W639708223
- Cited by
- 70,812
- References
- 45
- Citations by year
- 2026: 4605 · 2025: 8652 · 2024: 9390 · 2023: 9585 · 2022: 9034 · 2021: 9713 · 2020: 7552 · 2019: 6316 · 2018: 3709 · 2017: 1730 · 2016: 474 · 2015: 34 · 2014: 3 · 2012: 4
- FWCI
- 2766.93
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- neural information processing systems
- Funders
- Microsoft Research
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- Faster R-CNN, region proposal network, real-time object detection, Fast R-CNN, bounding box regression, VGG16, attention mechanism, end-to-end learning, Pascal VOC, MS COCO
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
