Monocular Object Instance Segmentation and Depth Ordering with CNNs
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
In this paper we tackle the problem of instance-level segmentation and depth ordering from a single monocular image. Towards this goal, we take advantage of convolutional neural nets and train them to directly predict instance-level segmentations where the instance ID encodes the depth ordering within image patches. To provide a coherent single explanation of an image we develop a Markov random field which takes as input the predictions of convolutional neural nets applied at overlapping patches of different resolutions, as well as the output of a connected component algorithm. It aims to predict accurate instance-level segmentation and depth ordering. We demonstrate the effectiveness of our approach on the challenging KITTI benchmark and show good performance on both tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
Self-citationDeep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213 · cited 2×
“We refer the reader to Fig.”
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386
Deep Visual-Semantic Alignments for Generating Image Descriptions
Karpathy, A., & Fei-Fei, L. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2598339 · Cited by 1,048
Parsing IKEA Objects: Fine Pose Estimation
Lim, J. J., Pirsiavash, H., & Torralba, A. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.372 · Cited by 249 · cited 1×
“Online object catalogs can be used to assign physical size to the CAD models based on their semantic class, giving rise to a 3D interpretation of the scene.”
MethodRecovering Occlusion Boundaries from a Single Image
Hoiem, D., Stein, A. N., Efros, A. A., & Hebert, M. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408985 · Cited by 206 · cited 1×
“Our work is related to a line of work that aims at predicting depth ordering using weak object information as well as mid-level cues such as junctions and boundaries [15, 17, 19].”
BackgroundFast Image Scanning with Deep Max-Pooling Convolutional Neural Networks
Giusti, A., Cireşan, D. C., Masci, J., Gambardella, L. M., & Schmidhuber, J. (2013) · arXiv · doi:10.48550/arxiv.1302.1700 · Cited by 48
Cited by
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
“Following this line, [9, 37, 40, 63, 81] propose different approaches that combine the strengths of CNNs and Conditional Random Fields (CRFs).”
BackgroundMethodPath Aggregation Network for Instance Segmentation
Liu, S., Qi, L., Qin, H., Shi, J., & Jia, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00913 · Cited by 10,011
Deep Ordinal Regression Network for Monocular Depth Estimation
Fu, H., Gong, M., Wang, C., Batmanghelich, K., & Tao, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00214 · Cited by 1,991
Hybrid Task Cascade for Instance Segmentation
Chen, K., Pang, J., Wang, J., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Shi, J., Ouyang, W., et al. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00511 · Cited by 1,587
“[46, 45] propose to predict instance labels based on local patches and integrate the local results with an MRF.”
ResultAdvances in Computer Vision-Based Civil Infrastructure Inspection and Monitoring
Spencer, B. F., Hoskere, V., & Narazaki, Y. (2019) · Engineering · doi:10.1016/j.eng.2018.11.030 · Cited by 1,282
Monocular 3D Object Detection for Autonomous Driving
Chen, X., Kundu, K., Zhang, Z., Ma, H., Fidler, S., & Urtasun, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.236 · Cited by 1,019
Self-citationTowards End-to-End Lane Detection: an Instance Segmentation Approach
Neven, D., Brabandere, B. D., Georgoulis, S., Proesmans, M., & Gool, L. V. (2018) · 2018 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2018.8500547 · Cited by 849
Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art
Janai, J., Güney, F., Behl, A., & Geiger, A. (2020) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000079 · Cited by 643
Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation
Cheng, B., Collins, M. D., Zhu, Y., Liu, T., Huang, T. S., Adam, H., & Chen, L.-C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01249 · Cited by 635
Papandreou, G., Zhu, T., Chen, L.-C., Gidaris, S., Tompson, J., & Murphy, K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_17 · Cited by 621
Meta R-CNN: Towards General Solver for Instance-Level Low-Shot Learning
Yan, X., Chen, Z., Xu, A., Wang, X., Liang, X., & Lin, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00967 · Cited by 616
Deep Watershed Transform for Instance Segmentation
Bai, M., & Urtasun, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.305 · Cited by 539
Self-citationDF-Net: Unsupervised Joint Learning of Depth and Flow Using Cross-Task Consistency
Zou, Y., Luo, Z., & Huang, J.-B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_3 · Cited by 469
PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation
Jiang, L., Zhao, H., Shi, S., Liu, S., Fu, C.-W., & Jia, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00492 · Cited by 460
Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++
Acuna, D., Ling, H., Kar, A., & Fidler, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00096 · Cited by 414
Self-citationSemantic Instance Segmentation with a Discriminative Loss Function
Brabandere, B. D., Neven, D., & Gool, L. V. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.02551 · Cited by 392
MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features
Chen, L.-C., Hermans, A., Papandreou, G., Schroff, F., Wang, P., & Adam, H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00422 · Cited by 373
Annotating Object Instances with a Polygon-RNN
Castrejon, L., Kundu, K., Urtasun, R., & Fidler, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.477 · Cited by 355
Self-citationRecurrent Instance Segmentation
Romera-Paredes, B., & Torr, P. H. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_19 · Cited by 311
End-to-End Instance Segmentation with Recurrent Attention
Ren, M., & Zemel, R. S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.39 · Cited by 309
Weakly Supervised Instance Segmentation Using Class Peak Response
Zhou, Y., Zhu, Y., Ye, Q., Qiu, Q., & Jiao, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00399 · Cited by 296
SGN: Sequential Grouping Networks for Instance Segmentation
Liu, S., Jia, J., Fidler, S., & Urtasun, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.378 · Cited by 269
Self-citationPerspective-Adaptive Convolutions for Scene Parsing
Zhang, R., Tang, S., Zhang, Y., Li, J., & Yan, S. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2890637 · Cited by 25
Duran, O., & Turan, B. (2022) · Measurement · doi:10.1016/j.measurement.2022.110732 · Cited by 25
Metadata
- Authors
- Zhang, Z., Schwing, A. G., Fidler, S., & Urtasun, R.
- Venue
- 2015 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2015-12
- Pages
- 2614-2622
- OpenAlex
- W260801291
- Cited by
- 152
- References
- 9
- FWCI
- 8.31
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.