The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Towards total scene understanding: Classification, annotation and segmentation in an automatic framework
Abstract
Given an image, we propose a hierarchical generative model that classifies the overall scene, recognizes and segments each object component, as well as annotates the image with a list of tags. To our knowledge, this is the first model that performs all three tasks in one coherent framework. For instance, a scene of a `polo game' consists of several visual objects such as `human', `horse', `grass', etc. In addition, it can be further annotated with a list of more abstract (e.g. `dusk') or visually less salient (e.g. `saddle') tags. Our generative model jointly explains images through a visual model and a textual model. Visually relevant objects are represented by regions and patches, while visually irrelevant textual annotations are influenced directly by the overall scene class. We propose a fully automatic learning framework that is able to learn robust scene models from noisy Web data such as images and user tags from Flickr.com. We demonstrate the effectiveness of our framework by automatically classifying, annotating and segmenting images from eight classes depicting sport scenes. In all three tasks, our model significantly outperforms state-of-the-art algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Object recognition from local scale-invariant features
Lowe, D. G. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790410 · Cited by 16,472
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015
Efficient Graph-Based Image Segmentation
Felzenszwalb, P. F., & Huttenlocher, D. P. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000022288.19776.77 · Cited by 6,176
A Bayesian Hierarchical Model for Learning Natural Scene Categories
Li, F.-F., & Perona, P. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.16 · Cited by 3,613
Markov Chain Sampling Methods for Dirichlet Process Mixture Models
Neal, R. M. (2000) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2000.10474879 · Cited by 2,288
Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary
Duygulu, P., Barnard, K., de Freitas, J. F. G., & Forsyth, D. A. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-47979-1_7 · Cited by 1,608
Automatic linguistic indexing of pictures by a statistical modeling approach
Li, J., & Wang, J. Z. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1227984 · Cited by 1,087
Blei, D. M., & Jordan, M. I. (2003) · Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval · doi:10.1145/860435.860460 · Cited by 1,054
Putting Objects in Perspective
Hoiem, D., Efros, A. A., & Hebert, M. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-008-0137-5 · Cited by 805
What, where and who? Classifying events by scene and object recognition
Li, L.-J., & Fei-Fei, L. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408872 · Cited by 781
Self-citationLearning object categories from Google's image search
Fergus, R., Fei-Fei, L., Perona, P., & Zisserman, A. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.142 · Cited by 700
Self-citationUsing Multiple Segmentations to Discover Objects and their Extent in Image Collections
Russell, B. C., Freeman, W. T., Efros, A. A., Sivic, J., & Zisserman, A. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.326 · Cited by 617
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303322533214 · Cited by 615
Rabinovich, A., Vedaldi, A., Galleguillos, C., Wiewiora, E., & Belongie, S. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408986 · Cited by 594
Image Parsing: Unifying Segmentation, Detection, and Recognition
Tu, Z., Chen, X., Yuille, A. L., & Zhu, S.-C. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-6642-x · Cited by 498
Correlative multi-label video annotation
Qi, G.-J., Hua, X.-S., Rui, Y., Tang, J., Mei, T., & Zhang, H.-J. (2007) · Proceedings of the 15th ACM international conference on Multimedia · doi:10.1145/1291233.1291245 · Cited by 477
Blei, D. M., & Jordan, M. I. (2003) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/860458.860460 · Cited by 376
Cao, L., & Fei-Fei, L. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408965 · Cited by 360
Self-citationUsing the Forest to See the Trees: A Graphical Model Relating Features, Objects, and Scenes
Murphy, K. P., Torralba, A. B., & Freeman, W. T. (2003) · neural information processing systems · Cited by 352
Kumar, M. P., Torr, P. H. S., & Zisserman, A. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.249 · Cited by 326
Learning hierarchical models of scenes, objects, and parts
Sudderth, E. B., Torralba, A., Freeman, W. T., & Willsky, A. S. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.137 · Cited by 297
Spatial Latent Dirichlet Allocation
Wang, X., & Grimson, W. E. L. (2007) · neural information processing systems · Cited by 287
A Statistical Model for General Contextual Object Recognition
Carbonetto, P., de Freitas, N., & Barnard, K. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24670-1_27 · Cited by 261
A Visual Category Filter for Google Images
Fergus, R., Perona, P., & Zisserman, A. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24670-1_19 · Cited by 258
Putting Objects in Perspective
Hoiem, D., Efros, A. A., & Hebert, M. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.232 · Cited by 258
Real-time computerized annotation of pictures
Li, J., & Wang, J. Z. (2006) · Proceedings of the 14th ACM international conference on Multimedia · doi:10.1145/1180639.1180841 · Cited by 196
OPTIMOL: automatic Online Picture collecTion via Incremental MOdel Learning
Li, L.-J., Wang, G., & Fei-Fei, L. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383048 · Cited by 194
Self-citationHarvesting Image Databases from the Web
Schroff, F., Criminisi, A., & Zisserman, A. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4409099 · Cited by 192
Berg, T. L., & Forsyth, D. A. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.57 · Cited by 181
Beyond Nouns: Exploiting Prepositions and Comparative Adjectives for Learning Visual Classifiers
Gupta, A., & Davis, L. S. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88682-2_3 · Cited by 167
Context and Hierarchy in a Probabilistic Image Model
Jin, Y., & Geman, S. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.86 · Cited by 163
Recognition by association via learning per-exemplar distances
Malisiewicz, T., & Efros, A. A. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587462 · Cited by 152
Combining Class-Specific Fragments for Object Classification
Sali, E., & Ullman, S. (1999) · British Machine Vision Conference (BMVC) · doi:10.5244/c.13.21 · Cited by 35
What do we see when we glance at a scene?
Fei-Fei, L., Koch, C., Iyer, A., & Perona, P. (2004) · Journal of Vision · doi:10.1167/4.8.863 · Cited by 18
Self-citation
Cited by
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Self-citationYFCC100M: the new data in multimedia research
Thomee, B., Shamma, D. A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., & Li, L.-J. (2016) · Communications of the ACM · doi:10.1145/2812802 · Cited by 1,541
Self-citationEvery Picture Tells a Story: Generating Sentences from Images
Farhadi, A., Hejrati, M., Sadeghi, M. A., Young, P., Rashtchian, C., Hockenmaier, J., & Forsyth, D. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_2 · Cited by 1,204
(2011) · Cited by 1,192
Image retrieval using scene graphs
Johnson, J., Krishna, R., Stark, M., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298990 · Cited by 1,101
Self-citation
Metadata
- Authors
- Li, L.-J., Socher, R., & Fei-Fei, L.
- Venue
- 2009 IEEE Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2009-06
- Pages
- 2036-2043
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2106624428
- Cited by
- 484
- References
- 34
- FWCI
- 43.34
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.