Iterative Instance Segmentation
Open accessAbstract
Existing methods for pixel-wise labelling tasks generally disregard the underlying structure of labellings, often leading to predictions that are visually implausible. While incorporating structure into the model should improve prediction quality, doing so is challenging - manually specifying the form of structural constraints may be impractical and inference often becomes intractable even if structural constraints are given. We sidestep this problem by reducing structured prediction to a sequence of unconstrained prediction problems and demonstrate that this approach is capable of automatically discovering priors on shape, contiguity of region predictions and smoothness of region contours from data without any a priori specification. On the instance segmentation task, this method outperforms the state-of-the-art, achieving a mean APr of 63:6% at 50% overlap and 43:3% at 70% overlap.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · see citing works
Self-citationGirshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971 · see citing works
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · see citing works
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853 · see citing works
GrabCut: Interactive foreground extraction using iterated graph cuts
Rother, C., Kolmogorov, V., & Blake, A. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015720 · Cited by 5,703 · see citing works
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034 · see citing works
Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Krähenbühl, P., & Koltun, V. (2012) · arXiv · doi:10.48550/arxiv.1210.5644 · Cited by 2,908 · see citing works
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · see citing works
Salakhutdinov, R., & Hinton, G. E. (2009) · Cited by 1,890 · see citing works
Semantic contours from inverse detectors
Hariharan, B., Arbeláez, P., Bourdev, L. D., Maji, S., & Malik, J. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126343 · Cited by 1,714 · see citing works
Self-citationHypercolumns for object segmentation and fine-grained localization
Hariharan, B., Arbelaez, P., Girshick, R., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298642 · Cited by 1,511 · see citing works
Self-citationJoint Training of a Convolutional Network and a Graphical Model for Human Pose EstimationPreprint
Tompson, J., Jain, A., LeCun, Y., & Bregler, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.2984 · Cited by 1,317 · see citing works
Simultaneous Detection and Segmentation
Hariharan, B., Arbeláez, P., Girshick, R., & Malik, J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10584-0_20 · Cited by 1,176 · see citing works
Self-citationMultiscale Combinatorial Grouping
Arbelaez, P., Pont-Tuset, J., Barron, J., Marques, F., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.49 · Cited by 1,155 · see citing works
Self-citationArticulated Human Detection with Flexible Mixtures of Parts
Yang, Y., & Ramanan, D. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.261 · Cited by 843 · see citing works
Human Pose Estimation with Iterative Error Feedback
Carreira, J., Agrawal, P., Fragkiadaki, K., & Malik, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.512 · Cited by 805 · see citing works
Self-citationRecurrent Convolutional Neural Networks for Scene Labeling
Pinheiro, P. O., & Collobert, R. (2014) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 621 · see citing works
Auto-Context and Its Application to High-Level Vision Tasks and 3D Brain Image Segmentation
Tu, Z., & Bai, X. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.186 · Cited by 578 · see citing works
Semantic Segmentation with Second-Order Pooling
Carreira, J., Caseiro, R., Batista, J. P., & Sminchisescu, C. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33786-4_32 · Cited by 461 · see citing works
Convolutional feature masking for joint object and stuff segmentation
Dai, J., He, K., & Sun, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299025 · Cited by 425 · see citing works
Detecting People Using Mutually Consistent Poselet Activations
Bourdev, L. D., Maji, S., Brox, T., & Malik, J. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15567-3_13 · Cited by 369 · see citing works
Self-citationClass-Specific, Top-Down Segmentation
Borenstein, E., & Ullman, S. (2002) · Lecture notes in computer science · doi:10.1007/3-540-47967-8_8 · Cited by 367 · see citing works
Semantic segmentation using regions and parts
Arbeláez, P., Hariharan, B., Gu, C., Gupta, S., Bourdev, L. D., & Malik, J. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248077 · Cited by 274 · see citing works
Self-citationLayered Object Models for Image Segmentation
Yang, Y., Hallman, S., Ramanan, D., & Fowlkes, C. C. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.208 · Cited by 142 · see citing works
The Shape Boltzmann Machine: A Strong Model of Object Shape
Eslami, S. M. A., Heess, N., Williams, C. K. I., & Winn, J. M. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0669-1 · Cited by 126 · see citing works
Parkhi, O., Vedaldi, A., Jawahar, C. V., & Zisserman, A. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126398 · Cited by 113 · see citing works
Object segmentation by alignment of poselet activations to image contours
Brox, T., Bourdev, L. D., Maji, S., & Malik, J. (2011) · doi:10.1109/cvpr.2011.5995659 · Cited by 101 · see citing works
Self-citationMaji, S., Vishnoi, N. K., & Malik, J. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995630 · Cited by 90 · see citing works
Self-citationLearning message-passing inference machines for structured prediction
Ross, S., Munoz, D., Hebert, M., & Bagnell, J. A. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995724 · Cited by 87 · see citing works
A Principled Deep Random Field Model for Image Segmentation
Kohli, P., Osokin, A., & Jegelka, S. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.257 · Cited by 61 · see citing works
Object detection and segmentation from joint embedding of parts and pixels
Maire, M., Yu, S. X., & Perona, P. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126490 · Cited by 53 · see citing works
Exploring Compositional High Order Pattern Potentials for Structured Output Learning
Li, Y., Tarlow, D., & Zemel, R. S. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.14 · Cited by 33 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes
Neuhold, G., Ollmann, T., Bulo, S. R., & Kontschieder, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.534 · Cited by 1,434 · see citing works
Fully Convolutional Instance-Aware Semantic Segmentation
Li, Y., Qi, H., Dai, J., Ji, X., & Wei, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.472 · Cited by 1,116 · see citing works
Oktay, O., Ferrante, E., Kamnitsas, K., Heinrich, M., Bai, W., Caballero, J., Cook, S. A., de Marvao, A., Dawes, T., O'Regan, D. P., et al. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2743464 · Cited by 751 · see citing works
“Besides variational models, cascaded convolutional architectures [27], [37] have been shown to discover priors on shape and structure in label space without any a priori specification.”
MethodEnd-to-End Instance Segmentation with Recurrent Attention
Ren, M., & Zemel, R. S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.39 · Cited by 309 · see citing works
Contextual Priming and Feedback for Faster R-CNN
Shrivastava, A., & Gupta, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_20 · Cited by 153 · see citing works
Recurrent Scene Parsing with Perspective Understanding in the Loop
Kong, S., & Fowlkes, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00106 · Cited by 114 · see citing works
- Authors
- Li, K., Hariharan, B., & Malik, J.
- Affiliations
- University of California, Berkeley (US)
- Meta (United States) (US)
- Venue
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2016-06
- Pages
- 3659-3667
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2963971305
- Cited by
- 152
- References
- 34
- Citations by year
- 2025: 5 · 2024: 9 · 2023: 15 · 2022: 11 · 2021: 18 · 2020: 14 · 2019: 20 · 2018: 25 · 2017: 28 · 2016: 7
- FWCI
- 9.13
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Nvidia
- Multidisciplinary University Research Initiative (N00014-09-1-1051)
- Natural Sciences and Engineering Research Council of Canada
- Office of Naval Research (N000141410671, N00014‐09‐1‐1051, N00014)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- instance segmentation, iterative refinement, structured prediction, shape priors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
