The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Supervised Learning of Semantic Classes for Image Annotation and Retrieval
Open accessAbstract
A probabilistic formulation for semantic image annotation and retrieval is proposed. Annotation and retrieval are posed as classification problems where each class is defined as the group of database images labeled with a common semantic label. It is shown that, by establishing this one-to-one correspondence between semantic labels and semantic classes, a minimum probability of error annotation and retrieval are feasible with algorithms that are 1) conceptually simple, 2) computationally efficient, and 3) do not require prior semantic segmentation of training images. In particular, images are represented as bags of localized feature vectors, a mixture density estimated for each image, and the mixtures associated with all images annotated with a common semantic label pooled into a density estimate for the corresponding semantic class. This pooling is justified by a multiple instance learning argument and performed efficiently with a hierarchical extension of expectation-maximization. The benefits of the supervised formulation over the more complex, and currently popular, joint modeling of semantic label and visual feature distributions are illustrated through theoretical arguments and extensive experiments. The supervised formulation is shown to achieve higher accuracy than various previously published methods at a fraction of their computational cost. Finally, the proposed method is shown to be fairly robust to parameter tuning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5195 · Cited by 19,370
Russell, E. L., Chiang, L. H., & Braatz, R. D. (2000) · Advances in Industrial Control · doi:10.1007/978-1-4471-0409-4_3 · Cited by 7,862
Content-based image retrieval at the end of the early years
Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., & Jain, R. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895972 · Cited by 6,056
Texture features for browsing and retrieval of image data
Manjunath, B. S., & Ma, W. Y. (1996) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.531803 · Cited by 3,727
Solving the multiple instance problem with axis-parallel rectangles
Dietterich, T. G., Lathrop, R. H., & Lozano-Pérez, T. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(96)00034-3 · Cited by 2,866
Stricker, M. A., & Orengo, M. (1995) · SPIE Proceedings · doi:10.1117/12.205308 · Cited by 1,747
QBIC project: querying images by content, using color, texture, and shape:
Niblack, C. W., Barber, R., Equitz, W., Flickner, M. D., Glasman, E. H., Petkovic, D., Yanker, P., Faloutsos, C., & Taubin, G. (1993) · SPIE Proceedings · doi:10.1117/12.143648 · Cited by 1,747
VisualSEEk: a fully automated content-based image query system
Smith, J. R., & Chang, S. (1996) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/244130.244151 · Cited by 1,745
A metric for distributions with applications to image databases
Rubner, Y., Tomasi, C., & Guibas, L. J. (2002) · Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) · doi:10.1109/iccv.1998.710701 · Cited by 1,726
Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary
Duygulu, P., Barnard, K., de Freitas, J. F. G., & Forsyth, D. A. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-47979-1_7 · Cited by 1,608
Photobook: Content-based manipulation of image databases
Pentland, A., Picard, R. W., & Sclaroff, S. (1996) · International Journal of Computer Vision · doi:10.1007/bf00123143 · Cited by 1,562
A Framework for Multiple-Instance Learning
Maron, O., & Lozano‐Pérez, T. (1997) · Cited by 1,232
Automatic linguistic indexing of pictures by a statistical modeling approach
Li, J., & Wang, J. Z. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1227984 · Cited by 1,087
Blei, D. M., & Jordan, M. I. (2003) · Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval · doi:10.1145/860435.860460 · Cited by 1,054
Image retrieval using color and shape
Jain, A. K., & Vailaya, A. (1996) · Pattern Recognition · doi:10.1016/0031-3203(95)00160-3 · Cited by 912
Efficient color histogram indexing for quadratic form distance functions
Hafner, J., Sawhney, H. S., Equitz, W., Flickner, M., & Niblack, W. (1995) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.391417 · Cited by 833
A Model for Learning the Semantics of Pictures
Lavrenko, V. P., Manmatha, R., & Jeon, J. (2003) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 680
Indoor-outdoor image classification
Szummer, M., & Picard, R. W. (2002) · Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database · doi:10.1109/caivd.1998.646032 · Cited by 639
Multiple-Instance Learning for Natural Scene Classification
Maron, O., & Ratan, A. L. (1998) · Cited by 572
Learning the semantics of words and pictures
Barnard, K., & Forsyth, D. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937654 · Cited by 524
Periodicity, directionality, and randomness: Wold features for image modeling and retrieval
Liu, F., & Picard, R. W. (1996) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.506794 · Cited by 524
ON IMAGE CLASSIFICATION: CITY IMAGES VS. LANDSCAPES
VAILAYA, A., JAIN, A., & ZHANG, H. J. (1998) · Pattern Recognition · doi:10.1016/s0031-3203(98)00079-x · Cited by 388
Blei, D. M., & Jordan, M. I. (2003) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/860458.860460 · Cited by 359
Visually searching the Web for content
Smith, J. R., & Chang, S.-F. (1997) · IEEE Multimedia · doi:10.1109/93.621578 · Cited by 350
Spatial Color Indexing and Applications
Huang, J., Ravi Kumar, S., Mitra, M., Zhu, W.-J., & Zabih, R. (1999) · International Journal of Computer Vision · doi:10.1023/a:1008108327226 · Cited by 271
A Statistical Model for General Contextual Object Recognition
Carbonetto, P., de Freitas, N., & Barnard, K. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24670-1_27 · Cited by 261
Local Feedback in Full-Text Retrieval Systems
Attar, R., & Fraenkel, A. S. (1977) · Journal of the ACM · doi:10.1145/322017.322021 · Cited by 230
Forsyth, D. A., & Fleck, M. M. (2002) · Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1997.609399 · Cited by 173
On image classification: city vs. landscape
Vailaya, A., Jain, A., & Zhang, H. J. (2002) · Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173) · doi:10.1109/ivl.1998.694464 · Cited by 163
A Note on Learning from Multiple-Instance Examples
Blum, A. L., & Kalai, A. T. (1998) · Machine Learning · doi:10.1023/a:1007402410823 · Cited by 115
GCap: Graph-based Automatic Image Captioning
Pan, J.-Y., Yang, H.-J., Faloutsos, C., & Duygulu, P. (2005) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvpr.2004.353 · Cited by 115
Consistent line clusters for building recognition in CBIR
Li, Y., & Shapiro, L. G. (2003) · doi:10.1109/icpr.2002.1048195 · Cited by 86
Minimum Probability of Error Image Retrieval
Vasconcelos, N. M. (2004) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2004.831125 · Cited by 69
Self-citationContent-based retrieval from image databases: current solutions and future directions
Vasconcelos, N. M., & Kunt, M. (2002) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2001.958037 · Cited by 54
Self-citationScalable discriminant feature selection for image retrieval and recognition
Vasconcelos, N. M., & Vasconcelos, M. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315242 · Cited by 53
Self-citationA database centric view of semantic image annotation and retrieval
Carneiro, G., & Vasconcelos, N. M. (2005) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1076034.1076129 · Cited by 51
Self-citationLibrary-based coding: a representation for efficient video compression and retrieval
Vasconcelos, N. M., & Lippman, A. (2002) · DCC · doi:10.1109/dcc.1997.581989 · Cited by 29
Self-citationA Constrained Semi-supervised Learning Approach to Data Association
Kück, H., Carbonetto, P. S., & de Freitas, N. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24672-5_1 · Cited by 20
Cited by
Image-Based Recommendations on Styles and Substitutes
McAuley, J., Targett, C., Shi, Q., & van den Hengel, A. (2015) · Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/2766462.2767755 · Cited by 2,280
A new approach to cross-modal multimedia retrieval
Rasiwasia, N., Costa Pereira, J., Coviello, E., Doyle, G., Lanckriet, G. R. G., Levy, R., & Vasconcelos, N. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1873987 · Cited by 1,334
Self-citationEvaluating bag-of-visual-words representations in scene classification
Yang, J., Jiang, Y.-G., Hauptmann, A. G., & Ngo, C.-W. (2007) · Proceedings of the international workshop on Workshop on multimedia information retrieval · doi:10.1145/1290082.1290111 · Cited by 825
TagProp: Discriminative metric learning in nearest neighbor models for image auto-annotation
Guillaumin, M., Mensink, T., Verbeek, J., & Schmid, C. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459266 · Cited by 673
Real-Time Computerized Annotation of Pictures
Li, J., & Wang, J. Z. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70847 · Cited by 506
Metadata
- Authors
- Carneiro, G., Chan, A. B., Moreno, P. J., & Vasconcelos, N.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2007-03
- Volume
- 29
- Issue
- 3
- Pages
- 394-410
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2125238156
- Cited by
- 859
- References
- 39
- FWCI
- 98.19
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.