segDeepM: Exploiting segmentation and context in deep neural networks for object detection
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
In this paper, we propose an approach that exploits object segmentation in order to improve the accuracy of object detection. We frame the problem as inference in a Markov Random Field, in which each detection hypothesis scores object appearance as well as contextual information using Convolutional Neural Networks, and allows the hypothesis to choose and score a segment out of a large pool of accurate object segmentation proposals. This enables the detector to incorporate additional evidence when it is available and thus results in more accurate detections. Our experiments show an improvement of 4.1% in mAP over the R-CNN baseline on PASCAL VOC 2010, and 3.4% over the current state-of-the-art, demonstrating the power of our approach.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853
Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Krähenbühl, P., & Koltun, V. (2012) · arXiv · doi:10.48550/arxiv.1210.5644 · Cited by 2,908
Going Deeper with Convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2014) · arXiv · doi:10.48550/arxiv.1409.4842 · Cited by 1,461
The Role of Context for Object Detection and Semantic Segmentation in the Wild
Mottaghi, R., Chen, X., Liu, X., Cho, N.-G., Lee, S.-W., Fidler, S., Urtasun, R., & Yuille, A. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.119 · Cited by 1,438 · cited 2×
“It is also different from [19] in that the model chooses contextual segments, and does not score context in a fixed segmentation window.”
BackgroundMethodSelf-citationSegmentation as selective search for object recognition
van de Sande, K. E. A., Uijlings, J. R. R., Gevers, T., & Smeulders, A. W. M. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126456 · Cited by 739 · cited 4×
“This strategy reduces the dependency on the initial candidate boxes obtained by [23] and enables our approach to recover from the potentially bad initial localiza-”
BackgroundMethodInfluentialDetect What You Can: Detecting and Representing Objects Using Holistic Models and Body Parts
Chen, X., Mottaghi, R., Liu, X., Fidler, S., Urtasun, R., & Yuille, A. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.254 · Cited by 605
Self-citationDiagnosing Error in Object Detectors
Hoiem, D., Chodpathumwan, Y., & Dai, Q. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33712-3_25 · Cited by 520
Towards Unified Object Detection and Semantic Segmentation
Dong, J., Chen, Q., Yan, S., & Yuille, A. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_20 · Cited by 70
Cited by
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
“uk:8080/leaderboard (accessed April 18, 2015) SegDeepM [25] achieves a higher mAP than Fast R-CNN (67.”
Object Detection With Deep Learning: A Review
Zhao, Z.-Q., Zheng, P., Xu, S.-T., & Wu, X. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2876865 · Cited by 5,545
“proposed the SegDeepM to exploit object segmentation which reduces the dependency on initial candidate boxes with Markov Random Field [106].”
MethodDeep learning for computer vision: A brief review
Voulodimos, A., Doulamis, N., Doulamis, A., & Protopapadakis, E. (2018) · Computational Intelligence and Neuroscience · doi:10.1155/2018/7068349 · Cited by 3,403
“Figure 4: Object detection results comparison from [66].”
BackgroundResultDeep Learning for Generic Object Detection: A Survey
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2018) · arXiv · doi:10.48550/arxiv.1809.02165 · Cited by 2,965
Deep Learning for Generic Object Detection: A Survey
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01247-4 · Cited by 2,965
“G lo ba lC on te xt SegDeepM [250] SS+CMPC VGG16 RCNN VOC10 VOC12 − CVPR15 Use an additional feature extracted from an enlarged object proposal as context information; Frame the detection problem as inference in a Markov Random Field.”
MethodDeep learning for visual understanding: A review
Guo, Y., Liu, Y., Oerlemans, A., Lao, S., Wu, S., & Lew, M. S. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.09.116 · Cited by 2,504
“RCNNs improved the benchmark by a large margin, and became the base model for many other promising algorithms [164,177,178,183,185].”
BackgroundMethodInfluentialObject detection in optical remote sensing images: A survey and a new benchmark
Li, K., Wan, G., Cheng, G., Meng, L., & Han, J. (2020) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.11.023 · Cited by 2,317
“…hard negative mining (Lin et al., 2017c; Shrivastava et al., 2016), contextual information fusion (Bell et al., 2016; Gidaris and Komodakis, 2015; Shrivastava and Gupta, 2016; Zhu et al., 2015b), modeling object deformations (Mordan et al., 2018; Ouyang et al., 2017; Xu et al., 2017), and so on.”
BackgroundA Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection
Cai, Z., Fan, Q., Feris, R. S., & Vasconcelos, N. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_22 · Cited by 1,605
“Context information has been modeled by a recurrent neural network in [26] and acquired from multiple regions around the object location in [7,5,27].”
BackgroundMethodRecent advances in deep learning for object detection
Wu, X., Sahoo, D., & Hoi, S. C. H. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.01.085 · Cited by 1,047
Monocular 3D Object Detection for Autonomous Driving
Chen, X., Kundu, K., Zhang, Z., Ma, H., Fidler, S., & Urtasun, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.236 · Cited by 1,019
Self-citationObject Detection via a Multi-region and Semantic Segmentation-Aware CNN Model
Gidaris, S., & Komodakis, N. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.135 · Cited by 801
(2015) · Cited by 656
Learning to Segment Object Candidates
Pinheiro, P. O., Collobert, R., & Dollár, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06204 · Cited by 648
Yang, F., Choi, W., & Lin, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.234 · Cited by 608
“With the exceptional power on image classification, CNN has been applied to object detection and achieves promising results [34, 14, 7, 44, 13, 27].”
Recent advances in small object detection based on deep learning: A review
Tong, K., Wu, Y., & Zhou, F. (2020) · Image and Vision Computing · doi:10.1016/j.imavis.2020.103910 · Cited by 592
A review of object detection based on deep learning
Xiao, Y., Tian, Z., Yu, J., Zhang, Y., Liu, S., Du, S., & Lan, X. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-08976-6 · Cited by 584
“Thus the stage-wise training of SVM+bounding-box regression (stage-wise training [66, 81, 294]) can be transformed into multi-task training [20, 177].”
BackgroundMethodFull-Resolution Residual Networks for Semantic Segmentation in Street Scenes
Pohlen, T., Hermans, A., Mathias, M., & Leibe, B. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.353 · Cited by 584
Deep Count: Fruit Counting Based on Deep Simulated Learning
Rahnemoonfar, M., & Sheppard, C. (2017) · Sensors · doi:10.3390/s17040905 · Cited by 520
Rotation-Insensitive and Context-Augmented Object Detection in Remote Sensing Images
Li, K., Cheng, G., Bu, S., & You, X. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2778300 · Cited by 439
3D Object Proposals Using Stereo Imagery for Accurate Object Class Detection
Chen, X., Kundu, K., Zhu, Y., Ma, H., Fidler, S., & Urtasun, R. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2706685 · Cited by 421
Self-citationCheng, G., Han, J., Zhou, P., & Xu, D. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2867198 · Cited by 408
MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features
Chen, L.-C., Hermans, A., Papandreou, G., Schroff, F., Wang, P., & Adam, H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00422 · Cited by 373
Weakly Supervised Object Localization and Detection: A Survey
Zhang, D., Han, J., Cheng, G., & Yang, M.-H. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3074313 · Cited by 277
Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data
Prakash, A., Boochoon, S., Brophy, M., Acuna, D., Cameracci, E., State, G., Shapira, O., & Birchfield, S. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794443 · Cited by 266
Cheng, G., Zhou, P., & Han, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.315 · Cited by 175
LocNet: Improving Localization Accuracy for Object Detection
Gidaris, S., & Komodakis, N. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.92 · Cited by 154
Contextual Priming and Feedback for Faster R-CNN
Shrivastava, A., & Gupta, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_20 · Cited by 153
Object-Proposal Evaluation Protocol is ‘Gameable’
Chavali, N., Agrawal, H., Mahendru, A., & Batra, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.97 · Cited by 65
StuffNet: Using ‘Stuff’ to Improve Object Detection
Brahmbhatt, S., Christensen, H. I., & Hays, J. (2017) · 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2017.109 · Cited by 24
Metadata
- Authors
- Zhu, Y., Urtasun, R., Salakhutdinov, R., & Fidler, S.
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 4703-4711
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1952506714
- Cited by
- 129
- References
- 9
- FWCI
- 9.42
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.