The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Describing the scene as a whole: Joint object detection, scene classification and semantic segmentation
Abstract
In this paper we propose an approach to holistic scene understanding that reasons jointly about regions, location, class and spatial extent of objects, presence of a class in the image, as well as the scene type. Learning and inference in our model are efficient as we reason at the segment level, and introduce auxiliary variables that allow us to decompose the inherent high-order potentials into pairwise potentials between a few variables with small number of states (at most the number of classes). Inference is done via a convergent message-passing algorithm, which, unlike graph-cuts inference, has no submodularity restrictions and does not require potential specific moves. We believe this is very important, as it allows us to encode our ideas and prior knowledge about the problem without the need to change the inference engine every time we introduce a new potential. Our approach outperforms the state-of-the-art on the MSRC-21 benchmark, while being much faster. Importantly, our holistic model is able to improve performance in all tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853
Contour Detection and Hierarchical Image Segmentation
Arbeláez, P., Maire, M., Fowlkes, C., & Malik, J. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.161 · Cited by 5,454
SUN database: Large-scale scene recognition from abbey to zoo
Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., & Torralba, A. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539970 · Cited by 3,283
Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Krähenbühl, P., & Koltun, V. (2012) · arXiv · doi:10.48550/arxiv.1210.5644 · Cited by 2,908
Approximating discrete probability distributions with dependence trees
Chow, C., & Liu, C. (1968) · IEEE Transactions on Information Theory · doi:10.1109/tit.1968.1054142 · Cited by 1,720
Semantic texton forests for image categorization and segmentation
Shotton, J., Johnson, M., & Cipolla, R. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587503 · Cited by 1,014
Image segmentation with a bounding box prior
Lempitsky, V., Kohli, P., Rother, C., & Sharp, T. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459262 · Cited by 390
Gu, C., Lim, J. J., Arbelaez, P., & Malik, J. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206727 · Cited by 347
What, Where and How Many? Combining Object Detectors and CRFs
Ladický, Ľ., Sturgess, P., Alahari, K., Russell, C., & Torr, P. H. S. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_31 · Cited by 310
Graph Cut Based Inference with Co-occurrence Statistics
Ladicky, L., Russell, C., Kohli, P., & Torr, P. H. S. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15555-0_18 · Cited by 308
Contextual models for object detection using boosted random fields
Torralba, A. B., Murphy, K. P., & Freeman, W. T. (2004) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 301
Learning hierarchical models of scenes, objects, and parts
Sudderth, E. B., Torralba, A., Freeman, W. T., & Willsky, A. S. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.137 · Cited by 297
Improving Spatial Support for Objects via Multiple Segmentations
Malisiewicz, T., & Efros, A. A. (2007) · Procedings of the British Machine Vision Conference 2007 · doi:10.5244/c.21.55 · Cited by 280
Estimating Spatial Layout of Rooms using Volumetric Reasoning about Objects and Surfaces
Lee, D. C., Gupta, A., Hebert, M., & Takeo Kanade, A. (2018) · doi:10.1184/r1/6554765 · Cited by 252
P3 & Beyond: Solving Energies with Higher Order Cliques
Kohli, P., Kumar, M. P., & Torr, P. H. S. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383204 · Cited by 223
A hierarchical field framework for unified context-based classification
Kumar, S., & Hebert, M. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.9 · Cited by 223
Region-Based Segmentation and Object Detection
Professor of Computer Science and Engineering, C. H. P. D. F. J., Raju, K. B., Saikiran, B., & M. TECH in Computer Science and Engineering, F. G. I. O. E. &. T. (2019) · International Journal of Innovative Technology and Exploring Engineering · doi:10.35940/ijitee.k1353.0981119 · Cited by 170
Harmony potentials for joint classification and segmentation
Gonfaus, J. M., Boix, X., van de Weijer, J., Bagdanov, A. D., Serrat, J., & Gonzàlez, J. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540048 · Cited by 154
Cascaded Classification Models: Combining Models for Holistic Scene Understanding
Heitz, G., Gould, S. J., Saxena, A., & Koller, D. (2008) · neural information processing systems · Cited by 148
Region-based Segmentation and Object Detection
Gould, S., Gao, T., & Koller, D. (2009) · Cited by 112
A Dynamic Conditional Random Field Model for Joint Labeling of Object and Scene Classes
Wojek, C., & Schiele, B. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88693-8_54 · Cited by 110
Object segmentation by alignment of poselet activations to image contours
Brox, T., Bourdev, L. D., Maji, S., & Malik, J. (2011) · doi:10.1109/cvpr.2011.5995659 · Cited by 101
Layered object detection for multi-class segmentation
Yang, Y., Hallman, S., Ramanan, D., & Fowlkes, C. C. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540070 · Cited by 96
Distributed message passing for large scale graphical models
Schwing, A. G., Hazan, T., Pollefeys, M., & Urtasun, R. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995642 · Cited by 67
Self-citationCombining appearance models and Markov Random Fields for category level object segmentation
Larlus, D., & Jurie, F. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587453 · Cited by 65
A Primal-Dual Message-Passing Algorithm for Approximated Large Scale Structured Prediction
Hazan, T., & Urtasun, R. (2010) · Cited by 60
Self-citationSolving Energies with Higher Order Cliques
Kohli, P., Kumar, M., & Torr, P. H. S. (2007) · Cited by 36
Efficient scale space auto-context for image segmentation and labeling
Jiang, J., & Zhuowen Tu, A. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206761 · Cited by 34
Efficient scale space auto-context for image segmentation and labeling
Jiang, J., & Zhuowen Tu, A. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206761 · Cited by 7
Cited by
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
Kirillov, A., He, K., Girshick, R., Rother, C., & Dollár, P. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00963 · Cited by 1,659
The Role of Context for Object Detection and Semantic Segmentation in the Wild
Mottaghi, R., Chen, X., Liu, X., Cho, N.-G., Lee, S.-W., Fidler, S., Urtasun, R., & Yuille, A. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.119 · Cited by 1,438
Self-citationPanoptic Feature Pyramid Networks
Kirillov, A., Girshick, R., He, K., & Dollár, P. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00656 · Cited by 1,460
Metadata
- Authors
- Yao, J., Fidler, S., & Urtasun, R.
- Venue
- 2012 IEEE Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2012-06
- Pages
- 702-709
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2137881638
- Cited by
- 365
- References
- 30
- FWCI
- 23.93
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.