The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Modeling annotated data
Abstract
We consider the problem of modeling annotated data---data with multiple types where the instance of one type (such as a caption) serves as a description of the other type (such as an image). We describe three hierarchical probabilistic mixture models which aim to describe such data, culminating in correspondence latent Dirichlet allocation, a latent variable model that is effective at modeling the joint distribution of both types and the conditional distribution of the annotation given the primary type. We conduct experiments on the Corel database of images and captions, assessing performance in terms of held-out likelihood, automatic annotation, and text-based image retrieval.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
(1972) · Cited by 17,806
Normalized cuts and image segmentation
Shi, J., & Malik, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.868688 · Cited by 15,502
An Introduction to Variational Methods for Graphical Models
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,791
Self-citationA Language Modeling Approach to Information Retrieval
Ponte, J. M., & Croft, W. B. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130368 · Cited by 2,503
SIMPLIcity: semantics-sensitive integrated matching for picture libraries
Wang, J. Z., Li, J., & Wiederhold, G. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.955109 · Cited by 2,196
A language modeling approach to information retrieval
Ponte, J. M., & Croft, W. B. (1998) · Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/290941.291008 · Cited by 1,821
Parametric empirical Bayes inference: theory and applications
Morris, C. N. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477920 · Cited by 1,448 · cited 1×
“With K draws from the prior distribution, however, we are in a good position to take the empirical Bayes perspective [9] and compute a maximum likelihood estimate of η.”
MethodAutomatic image annotation and retrieval using cross-media relevance models
Jeon, J., Lavrenko, V., & Manmatha, R. (2003) · Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval · doi:10.1145/860435.860459 · Cited by 1,139
A Variational Baysian Framework for Graphical Models
Attias, H. T. (1999) · neural information processing systems · Cited by 689
Cited by
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Blei, D. M. (2012) · Communications of the ACM · doi:10.1145/2133806.2133826 · Cited by 5,516
Self-citationImage retrieval: Ideas, influences, and trends of the new age
Datta, R., Joshi, D., Li, J., & Wang, J. Z. (2008) · ACM Computing Surveys · doi:10.1145/1348246.1348248 · Cited by 3,040
Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey
Jelodar, H., Wang, Y., Yuan, C., Feng, X., Jiang, X., Li, Y., & Zhao, L. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6894-4 · Cited by 1,961
A new approach to cross-modal multimedia retrieval
Rasiwasia, N., Costa Pereira, J., Coviello, E., Doyle, G., Lanckriet, G. R. G., Levy, R., & Vasconcelos, N. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1873987 · Cited by 1,334
Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics
Hodosh, M., Young, P., & Hockenmaier, J. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3994 · Cited by 1,314
Blei, D. M., & McAuliffe, J. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1003.0783 · Cited by 1,259
Self-citationAutomatic image annotation and retrieval using cross-media relevance models
Jeon, J., Lavrenko, V., & Manmatha, R. (2003) · Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval · doi:10.1145/860435.860459 · Cited by 1,139
A correlated topic model of Science
Blei, D. M., & Lafferty, J. (2018) · doi:10.1184/r1/6587330.v1 · Cited by 1,088
Self-citationA correlated topic model of Science
Blei, D. M., & Lafferty, J. D. (2007) · The Annals of Applied Statistics · doi:10.1214/07-aoas114 · Cited by 953
Self-citationSupervised Learning of Semantic Classes for Image Annotation and Retrieval
Carneiro, G., Chan, A. B., Moreno, P. J., & Vasconcelos, N. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.61 · Cited by 859
The nested chinese restaurant process and bayesian nonparametric inference of topic hierarchies
Blei, D. M., Griffiths, T. L., & Jordan, M. I. (2010) · Journal of the ACM · doi:10.1145/1667053.1667056 · Cited by 646
Self-citationIdentifying miRNAs, targets and functions
Liu, B., Li, J., & Cairns, M. J. (2014) · Briefings in Bioinformatics · doi:10.1093/bib/bbs075 · Cited by 563
Cross-modal Retrieval with Correspondence Autoencoder
Feng, F., Wang, X., & Li, R. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654902 · Cited by 560
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics
Gong, Y., Ke, Q., Isard, M., & Lazebnik, S. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-013-0658-4 · Cited by 540
Simultaneous image classification and annotation
Chong, W., Blei, D., & Li, F.-F. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206800 · Cited by 510
Self-citationLi, L.-J., Socher, R., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206718 · Cited by 484
Semantic Annotation and Retrieval of Music and Sound Effects
Turnbull, D., Barrington, L., Torres, D., & Lanckriet, G. (2008) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2007.913750 · Cited by 483
A New Baseline for Image Annotation
Makadia, A., Pavlovic, V., & Kumar, S. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88690-7_24 · Cited by 436
Content-based image retrieval: approaches and trends of the new age
Datta, R., Li, J., & Wang, J. Z. (2005) · Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval · doi:10.1145/1101826.1101866 · Cited by 435
Automatic image annotation and retrieval using cross-media relevance models
Jeon, J., Lavrenko, V. P., & Manmatha, R. (2003) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/860458.860459 · Cited by 417
The Emerging Trends of Multi-Label Learning
Liu, W., Wang, H., Shen, X., & Tsang, I. W. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3119334 · Cited by 400
A heuristic approach to determine an appropriate number of topics in topic modeling
Zhao, W., Chen, J. J., Perkins, R., Liu, Z., Ge, W., Ding, Y., & Zou, W. (2015) · BMC Bioinformatics · doi:10.1186/1471-2105-16-s13-s8 · Cited by 382
An Overview of Cross-Media Retrieval: Concepts, Methodologies, Benchmarks, and Challenges
Peng, Y., Huang, X., & Zhao, Y. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2705068 · Cited by 341
Scene Summarization for Online Image Collections
Simon, I., Snavely, N., & Seitz, S. M. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408863 · Cited by 339
Blei, D., Carin, L., & Dunson, D. (2010) · IEEE Signal Processing Magazine · doi:10.1109/msp.2010.938079 · Cited by 335
Self-citationMimno, D., Wallach, H. M., Naradowsky, J., Smith, D. A., & McCallum, A. (2009) · Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing Volume 2 - EMNLP '09 · doi:10.3115/1699571.1699627 · Cited by 328
Joint Feature Selection and Subspace Learning for Cross-Modal Retrieval
Wang, K., He, R., Wang, L., Wang, W., & Tan, T. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2505311 · Cited by 318
Topic Models Conditioned on Arbitrary Features with Dirichlet-multinomial Regression
Mimno, D., & McCallum, A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.3278 · Cited by 312
Das, P., Xu, C., Doell, R. F., & Corso, J. J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.340 · Cited by 282
A Statistical Model for General Contextual Object Recognition
Carbonetto, P., de Freitas, N., & Barnard, K. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24670-1_27 · Cited by 261
Semantic Annotation of Satellite Images Using Latent Dirichlet Allocation
Lienou, M., Maitre, H., & Datcu, M. (2010) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2009.2023536 · Cited by 259
A Multi-View Deep Learning Framework for EEG Seizure Detection
Yuan, Y., Xun, G., Jia, K., & Zhang, A. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2018.2871678 · Cited by 258
Hierarchical relational models for document networks
Chang, J., & Blei, D. M. (2010) · The Annals of Applied Statistics · doi:10.1214/09-aoas309 · Cited by 255
Self-citationA survey of multimodal deep generative models
Suzuki, M., & Matsuo, Y. (2022) · Advanced Robotics · doi:10.1080/01691864.2022.2035253 · Cited by 109
“Other research involves extending topic models such as latent Dirichlet allocation (LDA) [9] to multimodal input [8, 58].”
BackgroundMethodInterleaved text/image Deep Mining on a large-scale radiology database
Shin, H.-C., Lu, L., Kim, L., Seff, A., Yao, J., & Summers, R. M. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298712 · Cited by 94
Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models
Shi, Y., Siddharth, N., Paige, B., & Torr, P. H. S. (2019) · arXiv · doi:10.48550/arxiv.1911.03393 · Cited by 186
“We also observe that latent-variable mixture models have previously been applied to generative models targetting multi-modal topic-modelling (Barnard et al., 2003; Blei and Jordan, 2003).”
Method
Metadata
- Authors
- Blei, D. M., & Jordan, M. I.
- Venue
- Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
- Publisher
- ACM
- Published
- 2003-07-28
- Pages
- 127-134
- OpenAlex
- W2020842694
- Cited by
- 1,054
- References
- 13
- FWCI
- 28.38
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.